[Pharo-project] Opal decompiler: I've been a big good ... I've been a bit bad ...
For whoever might care: I rewrote / fixed the Opal decompiler. Sorry about me being a bit bad... my only test is this: CompiledMethod allInstances collect: [ :m | (OCBytecodeDecompiler new decompile: m) compiledMethod ] At least on the helvetia image that doesn't give any errors :) On to hooking it in with my new classbuilder and MethodModification model to apply bytecode rewriting for hyperfast class changes. I'll be back in an hour ;P cheers, Toon
On 27 March 2011 19:14, Toon Verwaest <toon.verwaest@gmail.com> wrote:
For whoever might care: I rewrote / fixed the Opal decompiler.
Sorry about me being a bit bad... my only test is this:
CompiledMethod allInstances collect: [ :m | Â Â (OCBytecodeDecompiler new decompile: m) compiledMethod ]
At least on the helvetia image that doesn't give any errors :)
On to hooking it in with my new classbuilder and MethodModification model to apply bytecode rewriting for hyperfast class changes.
I'll be back in an hour ;P and bring a new compiler/decompiler for us this time :)
cheers, Toon
-- Best regards, Igor Stasenko AKA sig.
Does anybody know what I'm supposed to transform accesses to removed instance variables into? Updating indexes of instance variables already works. And what data do I have to copy over to the new method so that it keeps a pointer to the original source, the class pointer and the selector? cheers, Toon On 03/27/2011 07:33 PM, Igor Stasenko wrote:
On 27 March 2011 19:14, Toon Verwaest<toon.verwaest@gmail.com> wrote:
For whoever might care: I rewrote / fixed the Opal decompiler.
Sorry about me being a bit bad... my only test is this:
CompiledMethod allInstances collect: [ :m | (OCBytecodeDecompiler new decompile: m) compiledMethod ]
At least on the helvetia image that doesn't give any errors :)
On to hooking it in with my new classbuilder and MethodModification model to apply bytecode rewriting for hyperfast class changes. I'll be back in an hour ;P
and bring a new compiler/decompiler for us this time :)
cheers, Toon
On 27 March 2011 19:46, Toon Verwaest <toon.verwaest@gmail.com> wrote:
Does anybody know what I'm supposed to transform accesses to removed instance variables into? Updating indexes of instance variables already works.
I guess it would be cool to transform it to message like 'self undefinedVariableAccess: #varName' and: 'self undefinedVariableAccess: #varName value: val' for assignments. Then later, when given method will run, it will trigger an error and developer can fix it in debugger. (Unless, of course you simply raise an error during (re)compilation when given ivar removed from class).
And what data do I have to copy over to the new method so that it keeps a pointer to the original source, the class pointer and the selector?
If you create a copy of method, copy the method's trailer. trailer := oldMethod trailer. newMethod := CompiledMethod newBytes: ... trailerBytes: ***trailer*** nArgs: ... nTemps: .. nStack: ... nLits: ... primitive: ...
cheers, Toon
On 03/27/2011 07:33 PM, Igor Stasenko wrote:
On 27 March 2011 19:14, Toon Verwaest<toon.verwaest@gmail.com> Â wrote:
For whoever might care: I rewrote / fixed the Opal decompiler.
Sorry about me being a bit bad... my only test is this:
CompiledMethod allInstances collect: [ :m | Â Â (OCBytecodeDecompiler new decompile: m) compiledMethod ]
At least on the helvetia image that doesn't give any errors :)
On to hooking it in with my new classbuilder and MethodModification model to apply bytecode rewriting for hyperfast class changes.
 I'll be back in an hour ;P
and bring a new compiler/decompiler for us this time :)
cheers, Toon
-- Best regards, Igor Stasenko AKA sig.
I guess it would be cool to transform it to message like
'self undefinedVariableAccess: #varName' and: 'self undefinedVariableAccess: #varName value: val' for assignments.
Then later, when given method will run, it will trigger an error and developer can fix it in debugger.
(Unless, of course you simply raise an error during (re)compilation when given ivar removed from class). That's the plan yes. Niko is working on that part now. I guess for now we might keep it on Eliot's proposal of just transforming to an undefined field access; but we'll see. One of the ideas is of course to collect the methods that get affected by a change and report back to the user; just like with all other types of errors that can occur during class modification. We have a pretty cool model already so it's becoming increasingly easy to install such hooks. I hate self error: 'your class is invalid because of something somewhere', and we'll get that all fixed! If you create a copy of method, copy the method's trailer.
trailer := oldMethod trailer.
newMethod := CompiledMethod newBytes: ... trailerBytes: ***trailer*** nArgs: ... nTemps: .. nStack: ... nLits: ... primitive: ... It seems that the opal compiler has a compiledMethodWith: trailer. Useful ;) I just had to set the methodClass: and selector: myself. Fair enough.
@Eliot thanks for the hints. I'll take it into account to make the recompilation complete. If Niko doesn't finish it this evening I'll probably do it tomorrow... I'm in the need of a small break :) Ignoring the fact that it doesn't support removing instance variables yet, it would very well already. Pretty cool to see methods updated on bytecode level without slow recompilation. Aaah... rewriting the classbuilder is starting to pay off :) And then on to stateful traits. Just for credits: I'm working together with Camillo Bruni and Niko Schwarz on this. cheers, Toon
(Unless, of course you simply raise an error during (re)compilation when given ivar removed from class).
That's the plan yes. Niko is working on that part now. I guess for now we might keep it on Eliot's proposal of just transforming to an undefined field access; but we'll see. One of the ideas is of course to collect the methods that get affected by a change and report back to the user; just like with all other types of errors that can occur during class modification. We have a pretty cool model already so it's becoming increasingly easy to install such hooks. I hate self error: 'your class is invalid because of something somewhere', and we'll get that all fixed!
I would love to have real object that contains compilation error (instead of Transcript show:) so that we can build tool to fix them :)
If you create a copy of method, copy the method's trailer.
trailer := oldMethod trailer.
newMethod := CompiledMethod newBytes: ... trailerBytes: ***trailer*** nArgs: ... nTemps: .. nStack: ... nLits: ... primitive: ... It seems that the opal compiler has a compiledMethodWith: trailer. Useful ;) I just had to set the methodClass: and selector: myself. Fair enough.
@Eliot thanks for the hints. I'll take it into account to make the recompilation complete. If Niko doesn't finish it this evening I'll probably do it tomorrow... I'm in the need of a small break :)
Ignoring the fact that it doesn't support removing instance variables yet, it would very well already. Pretty cool to see methods updated on bytecode level without slow recompilation. Aaah... rewriting the classbuilder is starting to pay off :) And then on to stateful traits.
Just for credits: I'm working together with Camillo Bruni and Niko Schwarz on this.
cheers, Toon
2011/3/28 Stéphane Ducasse <stephane.ducasse@inria.fr>:
(Unless, of course you simply raise an error during (re)compilation when given ivar removed from class).
That's the plan yes. Niko is working on that part now. I guess for now we might keep it on Eliot's proposal of just transforming to an undefined field access; but we'll see. One of the ideas is of course to collect the methods that get affected by a change and report back to the user; just like with all other types of errors that can occur during class modification. We have a pretty cool model already so it's becoming increasingly easy to install such hooks. I hate self error: 'your class is invalid because of something somewhere', and we'll get that all fixed!
I would love to have real object that contains compilation error (instead of Transcript show:) so that we can build tool to fix them :)
Various errors can occur at different stages, hence be generated from different classes in current (old) Compiler: - errors at parsing (syntax errors) - errors at name resolving (shadowing, undeclared ...) - errors at code generation (byte code limits ...) Several alternatives APi are possible for handling these errors: - Compiler signal all warning/errors by raising Exceptions, Compiler invocations are wrapped with a handler - Compiler signal all warnings/errors by sending a message to a reified errorHandler inst. var. The errorHandler can be instanciated with different classes for interactive / non interactive The errorHandler knows itself how to interact with user thru TextEditor, menus, or transcript... There is already a mixture of these strategies in current (old) Compiler. Except that compiler talks directly to a requestor and tries to handle interaction by itself if interactive. IMHO, it's better to write all your requirements before thinking of an architecture. Nicolas
To keep you up to date... I've already implemented bytecode rewriting that inserts an error. Basically when you have something like instVarAt: 1 where 1 is now removed, this gets translated to: pushConstant: (RemovedField new name: #old name) send: invalidRead pop pushConstant: nil and at:put: to pushConstant: (RemovedField new originalSlot: oldSlotObject) send: invalidWrite pop Then these messages will at runtime give you an error. Also since my MethodFieldUpdater sees these methods as it is rewriting it, it can collect them (doesn't do it yet though.. but that's 2 minutes of work) and present them to the user. Since we insert these magic constant object recompile time, the decompiler can find them again when recompiling for the next class update. When it finds them, it has all the meta-data it needs to figure out what slot access it was. Now that the method is being recompiled it can find out if there's a new slot that matches the old name; and put the corresponding field-access back into the method. At the moment the compiler already finds the slot accesses but doesn't yet rewrite the IR back to a field-access. But that's semi-easy too. The IR just doesn't provide an API yet for finding objects around a certain IR node and removing them ... so I got bored :) So in short, we have all the infrastructure to provide good runtime errors, to collect methods at compiletime; and to reintroduce slots after they were previously removed and reattach the accesses properly. And it's all done outside of the decompiler using our classbuilder + classmodification + field modification + method & instance modification maps. And so yes, I also want to have proper meta-info everywhere; proper objects everywhere; so that we can build proper tooling to solve it. The new classbuilder is fully built around this idea. It builds up modification maps exactly so that it can find conflicts early, and present them to the developer for fixing; BEFORE applying the modification to the classes. Once this fully works I can post an image somewhere that can give you a sneak preview. At the moment it's still a bit intertwined with Helvetia for the slot-related experiment; but that we'll extract after the paper is written so that we can submit a clean model to Pharo, ... if Pharo cares ;) cheers, Toon On 03/28/2011 09:20 PM, Stéphane Ducasse wrote:
(Unless, of course you simply raise an error during (re)compilation when given ivar removed from class).
That's the plan yes. Niko is working on that part now. I guess for now we might keep it on Eliot's proposal of just transforming to an undefined field access; but we'll see. One of the ideas is of course to collect the methods that get affected by a change and report back to the user; just like with all other types of errors that can occur during class modification. We have a pretty cool model already so it's becoming increasingly easy to install such hooks. I hate self error: 'your class is invalid because of something somewhere', and we'll get that all fixed! I would love to have real object that contains compilation error (instead of Transcript show:) so that we can build tool to fix them :)
If you create a copy of method, copy the method's trailer.
trailer := oldMethod trailer.
newMethod := CompiledMethod newBytes: ... trailerBytes: ***trailer*** nArgs: ... nTemps: .. nStack: ... nLits: ... primitive: ... It seems that the opal compiler has a compiledMethodWith: trailer. Useful ;) I just had to set the methodClass: and selector: myself. Fair enough.
@Eliot thanks for the hints. I'll take it into account to make the recompilation complete. If Niko doesn't finish it this evening I'll probably do it tomorrow... I'm in the need of a small break :)
Ignoring the fact that it doesn't support removing instance variables yet, it would very well already. Pretty cool to see methods updated on bytecode level without slow recompilation. Aaah... rewriting the classbuilder is starting to pay off :) And then on to stateful traits.
Just for credits: I'm working together with Camillo Bruni and Niko Schwarz on this.
cheers, Toon
On Mar 29, 2011, at 12:24 AM, Toon Verwaest wrote:
To keep you up to date... I've already implemented bytecode rewriting that inserts an error.
Basically when you have something like
instVarAt: 1
where 1 is now removed, this gets translated to:
pushConstant: (RemovedField new name: #old name) send: invalidRead pop pushConstant: nil
and at:put: to
pushConstant: (RemovedField new originalSlot: oldSlotObject) send: invalidWrite pop
Then these messages will at runtime give you an error. Also since my MethodFieldUpdater sees these methods as it is rewriting it, it can collect them (doesn't do it yet though.. but that's 2 minutes of work) and present them to the user.
Since we insert these magic constant object recompile time, the decompiler can find them again when recompiling for the next class update. When it finds them, it has all the meta-data it needs to figure out what slot access it was. Now that the method is being recompiled it can find out if there's a new slot that matches the old name; and put the corresponding field-access back into the method. At the moment the compiler already finds the slot accesses but doesn't yet rewrite the IR back to a field-access. But that's semi-easy too. The IR just doesn't provide an API yet for finding objects around a certain IR node and removing them ... so I got bored :)
So in short, we have all the infrastructure to provide good runtime errors, to collect methods at compiletime; and to reintroduce slots after they were previously removed and reattach the accesses properly. And it's all done outside of the decompiler using our classbuilder + classmodification + field modification + method & instance modification maps.
And so yes, I also want to have proper meta-info everywhere; proper objects everywhere; so that we can build proper tooling to solve it. The new classbuilder is fully built around this idea. It builds up modification maps exactly so that it can find conflicts early, and present them to the developer for fixing; BEFORE applying the modification to the classes.
Once this fully works I can post an image somewhere that can give you a sneak preview. At the moment it's still a bit intertwined with Helvetia for the slot-related experiment; but that we'll extract after the paper is written so that we can submit a clean model to Pharo, ... if Pharo cares ;)
oh yes it does... we all want that :). First class slots is a dream.
cheers, Toon
On 03/28/2011 09:20 PM, Stéphane Ducasse wrote:
(Unless, of course you simply raise an error during (re)compilation when given ivar removed from class).
That's the plan yes. Niko is working on that part now. I guess for now we might keep it on Eliot's proposal of just transforming to an undefined field access; but we'll see. One of the ideas is of course to collect the methods that get affected by a change and report back to the user; just like with all other types of errors that can occur during class modification. We have a pretty cool model already so it's becoming increasingly easy to install such hooks. I hate self error: 'your class is invalid because of something somewhere', and we'll get that all fixed! I would love to have real object that contains compilation error (instead of Transcript show:) so that we can build tool to fix them :)
If you create a copy of method, copy the method's trailer.
trailer := oldMethod trailer.
newMethod := CompiledMethod newBytes: ... trailerBytes: ***trailer*** nArgs: ... nTemps: .. nStack: ... nLits: ... primitive: ... It seems that the opal compiler has a compiledMethodWith: trailer. Useful ;) I just had to set the methodClass: and selector: myself. Fair enough.
@Eliot thanks for the hints. I'll take it into account to make the recompilation complete. If Niko doesn't finish it this evening I'll probably do it tomorrow... I'm in the need of a small break :)
Ignoring the fact that it doesn't support removing instance variables yet, it would very well already. Pretty cool to see methods updated on bytecode level without slow recompilation. Aaah... rewriting the classbuilder is starting to pay off :) And then on to stateful traits.
Just for credits: I'm working together with Camillo Bruni and Niko Schwarz on this.
cheers, Toon
On Sun, Mar 27, 2011 at 10:46 AM, Toon Verwaest <toon.verwaest@gmail.com>wrote:
Does anybody know what I'm supposed to transform accesses to removed instance variables into? Updating indexes of instance variables already works.
In the current model it should convert into a reference to an entry in Undeclared. i.e. it should do exactly what removing an inst var from a class def does. The sequence is delete an inst var from a class, reshape the instances of the class without the inst var, recompile the methods of the class. In the last step any references to the inst var name in source will not be resolved in the class definition and will hence result in being compiled as undeclared var refs. There are edge-cases. If the inst var name was masking an inst var of the same name in a superclass it would get resolved to that inst var. But the current class builder (rightly) disallows this and so it can be discounted. If the inst var name was masking a global of the same name in a class pool or shared pool it would get resolved to that. So the correct approach is to bind names with the same semantics that the compiler uses to binds names. best, Eliot
And what data do I have to copy over to the new method so that it keeps a pointer to the original source, the class pointer and the selector?
cheers, Toon
On 03/27/2011 07:33 PM, Igor Stasenko wrote:
On 27 March 2011 19:14, Toon Verwaest<toon.verwaest@gmail.com> wrote:
For whoever might care: I rewrote / fixed the Opal decompiler.
Sorry about me being a bit bad... my only test is this:
CompiledMethod allInstances collect: [ :m | (OCBytecodeDecompiler new decompile: m) compiledMethod ]
At least on the helvetia image that doesn't give any errors :)
On to hooking it in with my new classbuilder and MethodModification model to apply bytecode rewriting for hyperfast class changes.
I'll be back in an hour ;P
and bring a new compiler/decompiler for us this time :)
cheers,
Toon
Hi toon this is a good news. I would really like to see Opal taking off the ground. Stef On Mar 27, 2011, at 7:14 PM, Toon Verwaest wrote:
For whoever might care: I rewrote / fixed the Opal decompiler.
Sorry about me being a bit bad... my only test is this:
CompiledMethod allInstances collect: [ :m | (OCBytecodeDecompiler new decompile: m) compiledMethod ]
At least on the helvetia image that doesn't give any errors :)
On to hooking it in with my new classbuilder and MethodModification model to apply bytecode rewriting for hyperfast class changes. I'll be back in an hour ;P
cheers, Toon
participants (5)
-
Eliot Miranda -
Igor Stasenko -
Nicolas Cellier -
Stéphane Ducasse -
Toon Verwaest