Re: [Pharo-project] Long method can not be compiled using #compile: aString

Dec. 21, 2009


      You might want to have a look at SmaCC and how it generates the
scanner code: a huge state machine implemented in a single method that
consumes the source code of a programming language. To compile the
method SmaCC repeatedly applies the "extract method" refactoring to
reduce the code into nicely sized (but still efficient) methods.

Lukas

2009/12/21 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>:
...
2009/12/20 Igor Stasenko <siguctua@gmail.com>:
...
2009/12/20 Marcus Denker <marcus.denker@inria.fr>:
...
On Dec 20, 2009, at 8:44 PM, Bart Gauquie wrote:
...
Dear all,
I'm developing code which generates a method content, compiles it and adds it to an existing class.
I compile the message using #compile: message:
for instance:
In general, everything is limited to a fairly low quantity: number of temps, number of ivars, number of
literals.
Especially that the literal frame is limted to some small number (255 or something) can be easily be a problem.
(we ran into that already when using bytesurgeon on large existing methods when reifying all sends, for example).
To get around this, place all literals into one (usually array):
#( a b c d e ..... )
For instance, the above:
Â self addMMEFrom: 1054@512 to: 1037@504.
Â self addMMEFrom: 651@559 to: 643@579.
can be easily replaced by something like:
#( (1054 512 1037 504)
Â  Â ..... put as many as you want here.... no limit!!!
Â  Â (651 559 643 579) ) do: [:each |
self addMMEFrom: (each first @ each second) to: (each third @ each fourth)
]
...
Â  Â  Â  Â Marcus
_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Hehe, when generating code for evaluating monster symbolic
expressions, I bumped into many of these limitations.
Using a literal Array was not enough because even the indices would
consume a literal slot. So i had to decompose indices into
operations... Â Oh, and even some message sends had to be transformed
in #perform: sends (not shown here)...
Here is a fragment of code for allocating a literal slot (in
Objectworks, an ancestor of VW).
Nicolas
CodeStream subclass: #ExtendedCodeStream
Â  Â  Â  Â instanceVariableNames: 'reduceLiterals'
Â  Â  Â  Â classVariableNames: 'MaxLiteralIndex '
Â  Â  Â  Â poolDictionaries: 'OpcodePool '
Â  Â  Â  Â category: 'System-Compiler'
ExtendedCodeStream>>pushReducedConstant: lit
Â  Â  Â  Â "Handle MaxLiteralsNumber"
Â  Â  Â  Â | classIndex newClassArray litIndex |
Â  Â  Â  Â self push.
Â  Â  Â  Â lit == nil
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadNil.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â lit == true
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadTrue.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â lit == false
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadFalse.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â lit isInteger
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [(lit >= 0 and: [lit <= 2])
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadZero + lit.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â (lit >= 0 and: [lit <= 255])
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadByte with: lit.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â (lit >= -32768 and: [lit < 32768])
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â nextPut: OpLoadTwoBytes
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â with: ((lit bitShift: -8)
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â bitAnd: 255)
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â with: (lit bitAnd: 255).
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self]].
Â  Â  Â  Â ((lit isMemberOf: Character)
Â  Â  Â  Â  Â  Â  Â  Â and: [lit asInteger between: 0 and: 255])
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadCharacter with: lit asInteger.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â classIndex := literals at: lit class
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifAbsent:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ["Add the literal class array to the collection"
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â literalCollection addLast: Array new.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â literals at: lit class put: literalCollection size - 1].
Â  Â  Â  Â classIndex > MaxLiteralIndex
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [Errorscript cr; show: 'compilation failure: too many literal classes...'.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â self class literalLimitSignal raiseWith: topNode body].
Â  Â  Â  Â classIndex <= MaxLoadLiteral
Â  Â  Â  Â  Â  Â  Â  Â ifTrue: [code nextPut: OpLoadLiteral + classIndex]
Â  Â  Â  Â  Â  Â  Â  Â ifFalse: [code nextPut: OpXLoadLiteral with: classIndex].
Â  Â  Â  Â newClassArray := literalCollection at: classIndex + 1.
Â  Â  Â  Â litIndex := newClassArray indexOf: lit ifAbsent: [0].
Â  Â  Â  Â litIndex = 0
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [newClassArray := newClassArray copyWith: lit.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â literalCollection at: classIndex + 1 put: newClassArray.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â litIndex := newClassArray size].
Â  Â  Â  Â self pushBigIndex: litIndex.
Â  Â  Â  Â self sendNoCheck: #at: numArgs: 1
ExtendedCodeStream>>pushBigIndex: index
Â  Â  Â  Â "i do not want the index to figure in literalCollection
Â  Â  Â  Â so it must remain < 32768"
Â  Â  Â  Â | q r |
Â  Â  Â  Â index < 32768 ifTrue: [^self pushConstant: index].
Â  Â  Â  Â q := index // 32768.
Â  Â  Â  Â r := index - (q * 32768).
Â  Â  Â  Â self pushConstant: 32767; pushConstant: 1; sendNoCheck: #+ numArgs: 1.
Â  Â  Â  Â q = 1 ifFalse: [self pushBigIndex: q; sendNoCheck: #* numArgs: 1].
Â  Â  Â  Â r = 0 ifFalse: [self pushConstant: r; sendNoCheck: #+ numArgs: 1].
ExtendedCodeStream>>pushStatic: binding
Â  Â  Â  Â "Static variables do normally consume a literal slot.
Â  Â  Â  Â Arrange to economize this precious ressource"
Â  Â  Â  Â reduceLiterals
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [self pushReducedConstant: binding.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â self sendNoCheck: #value numArgs: 0]
Â  Â  Â  Â  Â  Â  Â  Â ifFalse: [super pushStatic: binding]
ExtendedCodeStream>>pushConstant: lit
Â  Â  Â  Â "Handle MaxLiteralsIndex"
Â  Â  Â  Â reduceLiterals ifTrue: [^self pushReducedConstant: lit].
Â  Â  Â  Â super pushConstant: lit.
ExtendedCodeStream>>testLiteralsSize
Â  Â  Â  Â "Restart compilation if literal limit is exceeded"
Â  Â  Â  Â literalCollection size > MaxLiteralIndex ifTrue: [reduceLiterals
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifTrue: [self class literalLimitSignal raiseWith: topNode body]
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifFalse: [self doReduceLiterals; restartCompilation]]
_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- 
Lukas Renggli
http://www.lukas-renggli.ch