Reify RBLiteralNode and RBBlockNode args/return with MetaLink?
Hi, I am currently playing with MetaLinks in Pharo 7 and stumbled upon some issues. For example, I found that instrumenting message sends may break compilation (see https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compil... <https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compil...>). Two other issues also arised: i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes. ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals. iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends? I am grateful for any pointers towards a solution. BTW, Marcus' lecture slides give a great intro to metalinks (http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf <http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf>). Cheers, Manuel
Hi, Thanks! I will check all of them⦠it might very well be that they are not implemented / there are bugs. I will add them to the bug list. (the problem is that each of them takes effort⦠write test, stare at byte-code⦠so it will take some time)
On 19 Nov 2018, at 09:30, Manuel Leuenberger <leuenberger@inf.unibe.ch> wrote:
Hi,
I am currently playing with MetaLinks in Pharo 7 and stumbled upon some issues. For example, I found that instrumenting message sends may break compilation (see https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compil... <https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compil...>). Two other issues also arised:
i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes. ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals. iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?
I am grateful for any pointers towards a solution. BTW, Marcus' lecture slides give a great intro to metalinks (http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf <http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf>).
Cheers, Manuel
I already have a fix for i) i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? I will commit it later today. Marcus
On 19 Nov 2018, at 22:14, Manuel Leuenberger <leuenberger@inf.unibe.ch> wrote:
Thanks for picking this up into the backlog. I am not blocked if I don't have them, I was just curious. I am quite intrigued with the idea of reyfing the runtime semantics, this opens a whole lot of opportunities for tools based on dynamic data.
On Nov 19, 2018 17:40, Marcus Denker <marcus.denker@inria.fr> wrote: Hi,
Thanks! I will check all of them⦠it might very well be that they are not implemented / there are bugs.
I will add them to the bug list.
(the problem is that each of them takes effort⦠write test, stare at byte-code⦠so it will take some time)
On 19 Nov 2018, at 09:30, Manuel Leuenberger <leuenberger@inf.unibe.ch <mailto:leuenberger@inf.unibe.ch>> wrote:
Hi,
I am currently playing with MetaLinks in Pharo 7 and stumbled upon some issues. For example, I found that instrumenting message sends may break compilation (see https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compil... <https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compil...>). Two other issues also arised:
i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes. ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals. iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?
I am grateful for any pointers towards a solution. BTW, Marcus' lecture slides give a great intro to metalinks (http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf <http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf>).
Cheers, Manuel
i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes.
This is now fixed. All subclasses of RBValueNode with the exception of Cascade now support #value. (Cascade: see below).
ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals.
I added issue tracker entries: https://pharo.fogbugz.com/f/cases/22704/Block-Execution-reify-value-in-after <https://pharo.fogbugz.com/f/cases/22704/Block-Execution-reify-value-in-after> https://pharo.fogbugz.com/f/cases/22703/Block-execution-arguments-reificatio... <https://pharo.fogbugz.com/f/cases/22703/Block-execution-arguments-reificatio...> The story is that we for now unified links on BlockNode to be the same as a link on the block body (as we have with RBMethodNode). But this is wrong: a link on a block (e.g. ) [1] should fire when the block *definition* is evaluated. Only a link on the body should fire on execution. If we link on the body, it will be possible to reify the #value (which is the value of the last statement). So this is some work (and a non-backward compatible change), but will be done till next week I hope.
iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?
https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCas... <https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCas...> It turns out this is just not implemented. Single message sends *inside* should work (as it calls the same code as a normal send to emit code). But even that needs tests. Cascade itself needs some code for links, then it will be easy to add reifications for #receiver and #value (and #operation). I will do that today I hope. Marcus
iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?
https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCas... <https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCas...>
It turns out this is just not implemented. Single message sends *inside* should work (as it calls the same code as a normal send to emit code). But even that needs tests.
Cascade itself needs some code for links, then it will be easy to add reifications for #receiver and #value (and #operation).
Done: https://github.com/pharo-project/pharo/pull/2018 <https://github.com/pharo-project/pharo/pull/2018> Marcus
Awesome, thanks for the effort Marcus! I was wondering if it would be useful to use matrix-testing for MetaLinks. I imagine having example methods which employ all features of the Smalltalk syntax (syntax on a postcard?), for which all intermediate results of expressions are known, then apply MetaLinks in all possible combinations (arguments x control) and check the reifications and side-effects against the oracle. Then the tests would generate a matrix of which combinations work and which don't. This should cover most of the possible executions. Cheers, Manuel
On 28 Nov 2018, at 11:55, Marcus Denker <marcus.denker@inria.fr> wrote:
iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?
https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCas... <https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCas...>
It turns out this is just not implemented. Single message sends *inside* should work (as it calls the same code as a normal send to emit code). But even that needs tests.
Cascade itself needs some code for links, then it will be easy to add reifications for #receiver and #value (and #operation).
Done:
https://github.com/pharo-project/pharo/pull/2018 <https://github.com/pharo-project/pharo/pull/2018>
Marcus
On 28 Nov 2018, at 21:37, Manuel Leuenberger <leuenberger@inf.unibe.ch> wrote:
Awesome, thanks for the effort Marcus!
I was wondering if it would be useful to use matrix-testing for MetaLinks. I imagine having example methods which employ all features of the Smalltalk syntax (syntax on a postcard?), for which all intermediate results of expressions are known, then apply MetaLinks in all possible combinations (arguments x control) and check the reifications and side-effects against the oracle. Then the tests would generate a matrix of which combinations work and which don't. This should cover most of the possible executions.
Yes, that would be nice. Marcus
Hi Manuel, would that extend to testing what happens if you add many, many metalinks in a method? I tried to rewrite the automated code tracer I had to use metalinks, and I gave up once I discovered that going over a certain number of metalinks in a method would stop working. Thierry Le jeu. 29 nov. 2018 à 09:03, Marcus Denker <marcus.denker@inria.fr> a écrit :
On 28 Nov 2018, at 21:37, Manuel Leuenberger <leuenberger@inf.unibe.ch> wrote:
Awesome, thanks for the effort Marcus!
I was wondering if it would be useful to use matrix-testing for MetaLinks. I imagine having example methods which employ all features of the Smalltalk syntax (syntax on a postcard?), for which all intermediate results of expressions are known, then apply MetaLinks in all possible combinations (arguments x control) and check the reifications and side-effects against the oracle. Then the tests would generate a matrix of which combinations work and which don't. This should cover most of the possible executions.
Yes, that would be nice.
Marcus
On 29 Nov 2018, at 10:45, Thierry Goubier <thierry.goubier@gmail.com> wrote:
Hi Manuel,
would that extend to testing what happens if you add many, many metalinks in a method?
yes, it would be nice to test that.
I tried to rewrite the automated code tracer I had to use metalinks, and I gave up once I discovered that going over a certain number of metalinks in a method would stop working.
If you add a lot of code, you need to use the new byte code set as the old one is extremely limited for everything (number of temps, jump offsetâ¦) Marcus
Le jeu. 29 nov. 2018 à 11:15, Marcus Denker <marcus.denker@inria.fr> a écrit :
On 29 Nov 2018, at 10:45, Thierry Goubier <thierry.goubier@gmail.com> wrote:
Hi Manuel,
would that extend to testing what happens if you add many, many metalinks in a method?
yes, it would be nice to test that.
I tried to rewrite the automated code tracer I had to use metalinks, and I gave up once I discovered that going over a certain number of metalinks in a method would stop working.
If you add a lot of code, you need to use the new byte code set as the old one is extremely limited for everything (number of temps, jump offsetâ¦)
Well, it failed before that limit; once the context was complex enough, it would fail in various ways, such as being unable to bind to the proper temporary, writing incorrect byte sequences, and so on As anyway, it was injecting the same code that my tracer was already doing, but in a less debuggable way, so I sort of gave up and kept MetaLink use to simpler things, such as a type checker that inject type checks on arguments based on the type inferred from arguments names. Now, when I see how you're improving metalinks in this thread, I'd be interested in a higher view (source-level) of the transformations you're doing for implementing metalinks: I think they then could be reused in different contexts (i.e. refactorings, for example), or generalised outside Smalltalk itself. It may probably already been expressed that way, but I haven't looked into the metalink code itself. Thierry
Marcus
On 29 Nov 2018, at 12:55, Thierry Goubier <thierry.goubier@gmail.com> wrote:
Le jeu. 29 nov. 2018 à 11:15, Marcus Denker <marcus.denker@inria.fr> a écrit :
On 29 Nov 2018, at 10:45, Thierry Goubier <thierry.goubier@gmail.com> wrote:
Hi Manuel,
would that extend to testing what happens if you add many, many metalinks in a method?
yes, it would be nice to test that.
I tried to rewrite the automated code tracer I had to use metalinks, and I gave up once I discovered that going over a certain number of metalinks in a method would stop working.
If you add a lot of code, you need to use the new byte code set as the old one is extremely limited for everything (number of temps, jump offsetâ¦)
Well, it failed before that limit; once the context was complex enough, it would fail in various ways, such as being unable to bind to the proper temporary, writing incorrect byte sequences, and so on
I think the changes we did will make that betterâ¦
As anyway, it was injecting the same code that my tracer was already doing, but in a less debuggable way, so I sort of gave up and kept MetaLink use to simpler things, such as a type checker that inject type checks on arguments based on the type inferred from arguments names.
Now, when I see how you're improving metalinks in this thread, I'd be interested in a higher view (source-level) of the transformations you're doing for implementing metalinks: I think they then could be reused in different contexts (i.e. refactorings, for example), or generalised outside Smalltalk itself. It may probably already been expressed that way, but I haven't looked into the metalink code itself.
For now it is implemented as a special compiler (with âself emitAfter: aNodeâ) added in overriden methods of the main compiler. I made the mistake of doing some stuff (after links, for example) directly in the IR level, that did not work. Now this is fixed for all but the links for Sequences (to be done later). Steven did a version that is based on AST transformation only, we will explore that direction further in the future. Marcus
Le ven. 30 nov. 2018 à 10:31, Marcus Denker <marcus.denker@inria.fr> a écrit :
On 29 Nov 2018, at 12:55, Thierry Goubier <thierry.goubier@gmail.com> wrote:
Le jeu. 29 nov. 2018 à 11:15, Marcus Denker <marcus.denker@inria.fr> a écrit :
On 29 Nov 2018, at 10:45, Thierry Goubier <thierry.goubier@gmail.com> wrote:
Hi Manuel,
would that extend to testing what happens if you add many, many metalinks in a method?
yes, it would be nice to test that.
I tried to rewrite the automated code tracer I had to use metalinks, and I gave up once I discovered that going over a certain number of metalinks in a method would stop working.
If you add a lot of code, you need to use the new byte code set as the old one is extremely limited for everything (number of temps, jump offsetâ¦)
Well, it failed before that limit; once the context was complex enough, it would fail in various ways, such as being unable to bind to the proper temporary, writing incorrect byte sequences, and so on
I think the changes we did will make that betterâ¦
As anyway, it was injecting the same code that my tracer was already doing, but in a less debuggable way, so I sort of gave up and kept MetaLink use to simpler things, such as a type checker that inject type checks on arguments based on the type inferred from arguments names.
Now, when I see how you're improving metalinks in this thread, I'd be interested in a higher view (source-level) of the transformations you're doing for implementing metalinks: I think they then could be reused in different contexts (i.e. refactorings, for example), or generalised outside Smalltalk itself. It may probably already been expressed that way, but I haven't looked into the metalink code itself.
For now it is implemented as a special compiler (with âself emitAfter: aNodeâ) added in overriden methods of the main compiler.
I made the mistake of doing some stuff (after links, for example) directly in the IR level, that did not work.
Now this is fixed for all but the links for Sequences (to be done later).
Steven did a version that is based on AST transformation only, we will explore that direction further in the future.
Interesting, so this would be the one I'd like to see (because I implemented with AST transforms). Would you have a link to his work? Mine is on github, and the version I did at UBO was on the VisualWorks AST. Thierry
Marcus
Steven did a version that is based on AST transformation only, we will explore that direction further in the future.
Interesting, so this would be the one I'd like to see (because I implemented with AST transforms). Would you have a link to his work? Mine is on github, and the version I did at UBO was on the VisualWorks AST.
https://github.com/StevenCostiou/Reflectivity-Flatteners <https://github.com/StevenCostiou/Reflectivity-Flatteners> I want to look at that for real after finishing some things in the current version. Marcus
On 28 Nov 2018, at 09:38, Marcus Denker <marcus.denker@inria.fr> wrote:
i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes.
This is now fixed. All subclasses of RBValueNode with the exception of Cascade now support #value. (Cascade: see below).
ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals.
I added issue tracker entries: ... https://pharo.fogbugz.com/f/cases/22703/Block-execution-arguments-reificatio... <https://pharo.fogbugz.com/f/cases/22703/Block-execution-arguments-reificatio...>
I added a first fix and simple tests for this, it is in the latest version. -> when putting a link on a block body, one can reify #arguments TODO: => clean up the situation block definition vs block body => test in real world situations (might not yet work for more complex cases, e.g. nested blocks) => support #value in after links. Marcus
participants (3)
-
Manuel Leuenberger -
Marcus Denker -
Thierry Goubier