Re: [Pharo-project] Long method can not be compiled using #compile: aString

Dec. 21, 2009


      2009/12/21 Lukas Renggli <renggli@gmail.com>:
...
You might want to have a look at SmaCC and how it generates the
scanner code: a huge state machine implemented in a single method that
consumes the source code of a programming language. To compile the
method SmaCC repeatedly applies the "extract method" refactoring to
reduce the code into nicely sized (but still efficient) methods.
Lukas
Of course, that would have been a nice option too (more general).
Above code is from 1994, not sure what was the status of SmaCC at that time...
In case of symbolic expressions, more agressive transformation could
occur, like common subexpression elimination...
(run on symbolic expression tree rather than AST, because knowing
there would be no side effect).

Nicolas
...
2009/12/21 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>:
...
2009/12/20 Igor Stasenko <siguctua@gmail.com>:
...
2009/12/20 Marcus Denker <marcus.denker@inria.fr>:
...
On Dec 20, 2009, at 8:44 PM, Bart Gauquie wrote:
...
Dear all,
I'm developing code which generates a method content, compiles it and adds it to an existing class.
I compile the message using #compile: message:
for instance:
In general, everything is limited to a fairly low quantity: number of temps, number of ivars, number of
literals.
Especially that the literal frame is limted to some small number (255 or something) can be easily be a problem.
(we ran into that already when using bytesurgeon on large existing methods when reifying all sends, for example).
To get around this, place all literals into one (usually array):
#( a b c d e ..... )
For instance, the above:
Â self addMMEFrom: 1054@512 to: 1037@504.
Â self addMMEFrom: 651@559 to: 643@579.
can be easily replaced by something like:
#( (1054 512 1037 504)
Â  Â ..... put as many as you want here.... no limit!!!
Â  Â (651 559 643 579) ) do: [:each |
self addMMEFrom: (each first @ each second) to: (each third @ each fourth)
]
...
Â  Â  Â  Â Marcus
_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Hehe, when generating code for evaluating monster symbolic
expressions, I bumped into many of these limitations.
Using a literal Array was not enough because even the indices would
consume a literal slot. So i had to decompose indices into
operations... Â Oh, and even some message sends had to be transformed
in #perform: sends (not shown here)...
Here is a fragment of code for allocating a literal slot (in
Objectworks, an ancestor of VW).
Nicolas
CodeStream subclass: #ExtendedCodeStream
Â  Â  Â  Â instanceVariableNames: 'reduceLiterals'
Â  Â  Â  Â classVariableNames: 'MaxLiteralIndex '
Â  Â  Â  Â poolDictionaries: 'OpcodePool '
Â  Â  Â  Â category: 'System-Compiler'
ExtendedCodeStream>>pushReducedConstant: lit
Â  Â  Â  Â "Handle MaxLiteralsNumber"
Â  Â  Â  Â | classIndex newClassArray litIndex |
Â  Â  Â  Â self push.
Â  Â  Â  Â lit == nil
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadNil.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â lit == true
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadTrue.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â lit == false
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadFalse.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â lit isInteger
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [(lit >= 0 and: [lit <= 2])
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadZero + lit.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â (lit >= 0 and: [lit <= 255])
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadByte with: lit.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â (lit >= -32768 and: [lit < 32768])
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â nextPut: OpLoadTwoBytes
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â with: ((lit bitShift: -8)
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â bitAnd: 255)
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â with: (lit bitAnd: 255).
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self]].
Â  Â  Â  Â ((lit isMemberOf: Character)
Â  Â  Â  Â  Â  Â  Â  Â and: [lit asInteger between: 0 and: 255])
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [code nextPut: OpLoadCharacter with: lit asInteger.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ^self].
Â  Â  Â  Â classIndex := literals at: lit class
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifAbsent:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ["Add the literal class array to the collection"
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â literalCollection addLast: Array new.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â literals at: lit class put: literalCollection size - 1].
Â  Â  Â  Â classIndex > MaxLiteralIndex
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [Errorscript cr; show: 'compilation failure: too many literal classes...'.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â self class literalLimitSignal raiseWith: topNode body].
Â  Â  Â  Â classIndex <= MaxLoadLiteral
Â  Â  Â  Â  Â  Â  Â  Â ifTrue: [code nextPut: OpLoadLiteral + classIndex]
Â  Â  Â  Â  Â  Â  Â  Â ifFalse: [code nextPut: OpXLoadLiteral with: classIndex].
Â  Â  Â  Â newClassArray := literalCollection at: classIndex + 1.
Â  Â  Â  Â litIndex := newClassArray indexOf: lit ifAbsent: [0].
Â  Â  Â  Â litIndex = 0
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [newClassArray := newClassArray copyWith: lit.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â literalCollection at: classIndex + 1 put: newClassArray.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â litIndex := newClassArray size].
Â  Â  Â  Â self pushBigIndex: litIndex.
Â  Â  Â  Â self sendNoCheck: #at: numArgs: 1
ExtendedCodeStream>>pushBigIndex: index
Â  Â  Â  Â "i do not want the index to figure in literalCollection
Â  Â  Â  Â so it must remain < 32768"
Â  Â  Â  Â | q r |
Â  Â  Â  Â index < 32768 ifTrue: [^self pushConstant: index].
Â  Â  Â  Â q := index // 32768.
Â  Â  Â  Â r := index - (q * 32768).
Â  Â  Â  Â self pushConstant: 32767; pushConstant: 1; sendNoCheck: #+ numArgs: 1.
Â  Â  Â  Â q = 1 ifFalse: [self pushBigIndex: q; sendNoCheck: #* numArgs: 1].
Â  Â  Â  Â r = 0 ifFalse: [self pushConstant: r; sendNoCheck: #+ numArgs: 1].
ExtendedCodeStream>>pushStatic: binding
Â  Â  Â  Â "Static variables do normally consume a literal slot.
Â  Â  Â  Â Arrange to economize this precious ressource"
Â  Â  Â  Â reduceLiterals
Â  Â  Â  Â  Â  Â  Â  Â ifTrue:
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â [self pushReducedConstant: binding.
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â self sendNoCheck: #value numArgs: 0]
Â  Â  Â  Â  Â  Â  Â  Â ifFalse: [super pushStatic: binding]
ExtendedCodeStream>>pushConstant: lit
Â  Â  Â  Â "Handle MaxLiteralsIndex"
Â  Â  Â  Â reduceLiterals ifTrue: [^self pushReducedConstant: lit].
Â  Â  Â  Â super pushConstant: lit.
ExtendedCodeStream>>testLiteralsSize
Â  Â  Â  Â "Restart compilation if literal limit is exceeded"
Â  Â  Â  Â literalCollection size > MaxLiteralIndex ifTrue: [reduceLiterals
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifTrue: [self class literalLimitSignal raiseWith: topNode body]
Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â ifFalse: [self doReduceLiterals; restartCompilation]]
_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
--
Lukas Renggli
http://www.lukas-renggli.ch
_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project