[Pharo-project] How to easily intercept messages sent to ALL objects ?
Hi folks. I want to intercept (and do something) for all objects. I know few approaches, for example: 1) using method wrappers (implementing run: aSelector with: arguments in: aReceiver) I can wrap all compiled methods of all classes and there I have the receiver. 2) become all objects to some kind of hacky object that stores the original object and that redefines the doesNotUnderstand: But in 1) I need to change all CompiledMehtod of all classes. in 2) I have to become all the objects to another object Then my question is, is there an easier way to intercept all messages send to all objects from the image side (not going to vm) and with a lower cost ? And if going to vm, how ? Thanks. Mariano
Hi folks. I want to intercept (and do something) for all objects. I know few approaches, for example:
hi Mariano, I tried something similar, but I haven't found a satisfactory solution.
1) using method wrappers (implementing run: aSelector with: arguments in: aReceiver) I can wrap all compiled methods of all classes and there I have the receiver.
You can wrap most of the compiled methods in the system. Not all of them, so you will have find out which one you can safely wrap. Wrapping and unwrapping all classes takes a bit of times (1-2 seconds on my machine). I therefore used MessageTally to find out what I exactly need to wrap. This is not perfect (MessageTally uses a sampling profiling after all). But it is fine for now.
2) become all objects to some kind of hacky object that stores the original object and that redefines the doesNotUnderstand:
If you succeed, let me know. Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
On Thu, Mar 25, 2010 at 7:31 PM, Alexandre Bergel <alexandre@bergel.eu>wrote:
Hi folks. I want to intercept (and do something) for all objects. I know
few approaches, for example:
hi Mariano,
I tried something similar, but I haven't found a satisfactory solution.
1) using method wrappers (implementing run: aSelector with: arguments in:
aReceiver) I can wrap all compiled methods of all classes and there I have the receiver.
You can wrap most of the compiled methods in the system. Not all of them, so you will have find out which one you can safely wrap.
I also noticed this too. There are some methods or classes that are so "kernel" that cannot be wrapped. And of course, the class wrapper in itself. So...do you have a list to share of the classes/methods that should not be wrapped ? In my case, for example: #new #changed #initialize #drawOn: #doesNotUnderstand: but I am not even sure. Wrapping and unwrapping all classes takes a bit of times (1-2 seconds on my
machine).
is not bad, at all.
I therefore used MessageTally to find out what I exactly need to wrap. This is not perfect (MessageTally uses a sampling profiling after all). But it is fine for now.
I didn't understand you here. What I don't understand is how MessageTally may be of help here.
2) become all objects to some kind of hacky object that stores the
original object and that redefines the doesNotUnderstand:
If you succeed, let me know.
Sure. I have been playing (no more than 2 days) and analyzing an implementation of MethodWrappers, the TestCoverage and CUIS' Protocol catcher. I just wrote some notes for myself in a simple text file. Actually to write down what I understood and not to loose it. I attached just in case you are interested. Ahhh I forgot. I am putting all these little tools here: http://www.squeaksource.com/ObjectMetaTools in case you want to take a look. Cheers Mariano
Cheers, Alexandre
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
I also noticed this too. There are some methods or classes that are so "kernel" that cannot be wrapped. And of course, the class wrapper in itself.
So...do you have a list to share of the classes/methods that should not be wrapped ?
In my case, for example: #new #changed #initialize #drawOn: #doesNotUnderstand:
For my particular case, I do not instrument the packages coming with a core image. I tried to identify the dangerous subset of it, but apparently I run into some not easy-to-reproduce problem: running twice the image instrumentation block on a method different from the previous try.
I didn't understand you here. What I don't understand is how MessageTally may be of help here.
In order to avoid a complete image instrumentation, I instrument only what is necessary: what is actually executed for a particular scenario. Message tally is handy because you can easily infer what are the involved packages when executing a piece of code. I use this when profiling code: I do a message tally on the code to profile before doing a full-fledged instrumentation.
Sure. I have been playing (no more than 2 days) and analyzing an implementation of MethodWrappers, the TestCoverage and CUIS' Protocol catcher. I just wrote some notes for myself in a simple text file. Actually to write down what I understood and not to loose it. I attached just in case you are interested.
Ahhh I forgot. I am putting all these little tools here: http://www.squeaksource.com/ObjectMetaTools in case you want to take a look.
This was on my todo list. I will have a look at it soon. Cheers, Alexandre NB: Mail apparently cannot recognize your answer and I reply to your email. -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
On 2010-03-25, at 9:50 AM, Mariano Martinez Peck wrote:
And if going to vm, how ?
Thanks.
See http://bugs.squeak.org/view.php?id=4882 setVMStatsTraceMessageSendLevels Some user VM assembly is required. -- =========================================================================== John M. McIntosh <johnmci@smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On Mar 25, 2010, at 5:50 PM, Mariano Martinez Peck wrote:
Hi folks. I want to intercept (and do something) for all objects. I know few approaches, for example:
1) using method wrappers (implementing run: aSelector with: arguments in: aReceiver) I can wrap all compiled methods of all classes and there I have the receiver.
2) become all objects to some kind of hacky object that stores the original object and that redefines the doesNotUnderstand:
But in 1) I need to change all CompiledMehtod of all classes. in 2) I have to become all the objects to another object
Then my question is, is there an easier way to intercept all messages send to all objects from the image side (not going to vm) and with a lower cost ?
No. So the MOP (meta object protocol), the "reflective API" of Smalltalk does not allow for "reifying" message receive. Historically, after Reflection was discovered and people started to generalize it, of cource they added that. For example, there is CODA: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.6885&rep=rep1&t... which made the whole "message send" explicit and interceptable. This means it is decomposed into Send, Receive, Lookup, Apply. Very nice. Very general. One can override Lookup when one wants to do e.g. Multiple inheritance. Or Apply, when the language of the method is not smalltalk. Or Receive, if you want to hook into "does an object get a message?". Futures, non-blocking messages. All easy to be done. But: it's slow. Dog slow. So people started to think about: how to speed up? Sadly the idea of optimizing at runtime (partially evaluating the MOP) has seen not much work. What was done is "partial reflection": only reifiy those opereation at exactly those spots that one is interested in. This was done in MetaClassTalk (it checks if the MetaClass overrides message sending, else it uses normal bytecodes) and to the extreme in Reflex. Now if you want to change reflective behavior *per object* the other important thing is that you need to be able to define changed behavior at the level of Objects vs. Classes. CLOS style MOPS like MetaClassTalk allow *only* for a per-Class-change. Enter Eric Tanter's Relex. Reflex generalized the MOP idea to not be bound to classes/metaclasses but you can select *per instruction* what to reify. And that reification is only done when needed, keeping the code you are not interested in fast. (Reflex is what happens when you bring the lessons of AOP back into MOPs. That is, the good parts of AOP). http://portal.acm.org/citation.cfm?id=949309 Of course, message receive is not easy to achive other than reifying method-execute on all methods+doesNotUnderstand. This can be made a bit more efficient for all the objects of the class you are not interested in by using Object-specific Behavior (the anonymous subclass trick). Phillippe did that in TreeNurse... But all in all, this is more of a hack. Now for message receive, one could imagine a partial reification scheme: a tag bit on the object header, if it is there, the VM calls a receive method instead of the method that would originally be called. So you would pay, in all casses, one bit-check on message send *and* you need space to put this flag. Would anyone want to pay that? In general, I would like to experiment with a MOP that can reify message receive in a meaningful *and efficient* way. But the current VM can't do it. In addition, there is always the problem that reflection leads to loops (your code will use the code it will used on). http://scg.unibe.ch/archive/papers/Denk08bMetaContextLNBIP.pdf Wich needs to be solved, too, if you ever want to be able to use reflection on everything (even the things you implement reflectively). Alternatively, one could use a proxy. Now if you don't use a subclass of ProtoObject, but something with VM support int the Style of Adrian Lienhard's Aliases: http://scg.unibe.ch/archive/papers/Lien08bBackInTimeDebugging.pdf or the Handle (we need a better word!!!) stuff that JB is working on, maybe this would not be too bad. The Alias is *hidden*, you can put it (eventually, after all engineering done) on any object (even Integers)... and than, after you swap out something, you need a proxy anyway for the "outpointer".
And if going to vm, how ?
The OOPSLA paper of 2008 on tolerating memory leaks has scheme where they use the GC of Jikes. So compared to LOOM, they have a modern GC without an object table, and it seems much simpler. Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
On Fri, Mar 26, 2010 at 1:15 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Mar 25, 2010, at 5:50 PM, Mariano Martinez Peck wrote:
Hi folks. I want to intercept (and do something) for all objects. I know few approaches, for example:
1) using method wrappers (implementing run: aSelector with: arguments in: aReceiver) I can wrap all compiled methods of all classes and there I have the receiver.
2) become all objects to some kind of hacky object that stores the original object and that redefines the doesNotUnderstand:
But in 1) I need to change all CompiledMehtod of all classes. in 2) I have to become all the objects to another object
Then my question is, is there an easier way to intercept all messages send to all objects from the image side (not going to vm) and with a lower cost ?
No.
So the MOP (meta object protocol), the "reflective API" of Smalltalk does not allow for "reifying" message receive.
Historically, after Reflection was discovered and people started to generalize it, of cource they added that. For example, there is CODA:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.6885&rep=rep1&t...
which made the whole "message send" explicit and interceptable.
This means it is decomposed into Send, Receive, Lookup, Apply. Very nice. Very general. One can override Lookup when one wants to do e.g. Multiple inheritance. Or Apply, when the language of the method is not smalltalk. Or Receive, if you want to hook into "does an object get a message?". Futures, non-blocking messages. All easy to be done.
But: it's slow. Dog slow. So people started to think about: how to speed up? Sadly the idea of optimizing at runtime (partially evaluating the MOP) has seen not much work. What was done is "partial reflection": only reifiy those opereation at exactly those spots that one is interested in. This was done in MetaClassTalk (it checks if the MetaClass overrides message sending, else it uses normal bytecodes) and to the extreme in Reflex.
Now if you want to change reflective behavior *per object* the other important thing is that you need to be able to define changed behavior at the level of Objects vs. Classes. CLOS style MOPS like MetaClassTalk allow *only* for a per-Class-change.
Enter Eric Tanter's Relex. Reflex generalized the MOP idea to not be bound to classes/metaclasses but you can select *per instruction* what to reify. And that reification is only done when needed, keeping the code you are not interested in fast. (Reflex is what happens when you bring the lessons of AOP back into MOPs. That is, the good parts of AOP).
http://portal.acm.org/citation.cfm?id=949309
Of course, message receive is not easy to achive other than reifying method-execute on all methods+doesNotUnderstand. This can be made a bit more efficient for all the objects of the class you are not interested in by using Object-specific Behavior (the anonymous subclass trick). Phillippe did that in TreeNurse... But all in all, this is more of a hack.
Now for message receive, one could imagine a partial reification scheme: a tag bit on the object header, if it is there, the VM calls a receive method instead of the method that would originally be called. So you would pay, in all casses, one bit-check on message send *and* you need space to put this flag. Would anyone want to pay that?
Perhaps we can be cleverer. What could we do if we introduced delegation into the language? Then we can add wrappers around objects and put the interception into the wrappers. This way there is no additional check. If one wants to intercept references to an object one creates a DNU wrapper and becomes the two objects. An execution model of delegation is that there are two slots for the receiver in a context, a state slot through which inst vars are accessed, and a self slot to which messages are sent. Normal sends merely duplicate the receiver into both slots. Delegating sends take state as an additional argument. An alternative model is to dispense with direct inst var access (assuming the JIT will inline simple accessors in PICs, or simply by virtue of the use of machine code render accessors affordable) and provide accessors for delegators that indirect through the delegate.
In general, I would like to experiment with a MOP that can reify message receive in a meaningful *and efficient* way. But the current VM can't do it. In addition, there is always the problem that reflection leads to loops (your code will use the code it will used on).
http://scg.unibe.ch/archive/papers/Denk08bMetaContextLNBIP.pdf
Wich needs to be solved, too, if you ever want to be able to use reflection on everything (even the things you implement reflectively).
Alternatively, one could use a proxy. Now if you don't use a subclass of ProtoObject, but something with VM support int the Style of Adrian Lienhard's Aliases:
http://scg.unibe.ch/archive/papers/Lien08bBackInTimeDebugging.pdf
or the Handle (we need a better word!!!) stuff that JB is working on, maybe this would not be too bad. The Alias is *hidden*, you can put it (eventually, after all engineering done) on any object (even Integers)... and than, after you swap out something, you need a proxy anyway for the "outpointer".
And if going to vm, how ?
The OOPSLA paper of 2008 on tolerating memory leaks has scheme where they use the GC of Jikes. So compared to LOOM, they have a modern GC without an object table, and it seems much simpler.
Marcus
-- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Now for message receive, one could imagine a partial reification scheme: a tag bit on the object header, if it is there, the VM calls a receive method instead of the method that would originally be called. So you would pay, in all casses, one bit-check on message send *and* you need space to put this flag. Would anyone want to pay that?
Perhaps we can be cleverer. What could we do if we introduced delegation into the language? Then we can add wrappers around objects and put the interception into the wrappers. This way there is no additional check. If one wants to intercept references to an object one creates a DNU wrapper and becomes the two objects.
An execution model of delegation is that there are two slots for the receiver in a context, a state slot through which inst vars are accessed, and a self slot to which messages are sent. Normal sends merely duplicate the receiver into both slots. Delegating sends take state as an additional argument.
An alternative model is to dispense with direct inst var access (assuming the JIT will inline simple accessors in PICs, or simply by virtue of the use of machine code render accessors affordable) and provide accessors for delegators that indirect through the delegate.
Eliot indeed this kind of schem looks interesting. I would love a system where we can really experiment with that kind of infrastructural changes that open a lot of new space. Stef
On Mar 26, 2010, at 6:12 PM, Eliot Miranda wrote:
Perhaps we can be cleverer. What could we do if we introduced delegation into the language? Then we can add wrappers around objects and put the interception into the wrappers. This way there is no additional check. If one wants to intercept references to an object one creates a DNU wrapper and becomes the two objects.
An execution model of delegation is that there are two slots for the receiver in a context, a state slot through which inst vars are accessed, and a self slot to which messages are sent. Normal sends merely duplicate the receiver into both slots. Delegating sends take state as an additional argument.
An alternative model is to dispense with direct inst var access (assuming the JIT will inline simple accessors in PICs, or simply by virtue of the use of machine code render accessors affordable) and provide accessors for delegators that indirect through the delegate.
Yes, that would be interesting to look at. Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
Hi, first post, sorry for breaking any rules. ;-) On Sat, Mar 27, 2010 at 6:17 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
Perhaps we can be cleverer. Â What could we do if we introduced delegation into the language? Â Then we can add wrappers around objects and put the interception into the wrappers. Â This way there is no additional check. Â If one wants to intercept references to an object one creates a DNU wrapper and becomes the two objects.
An execution model of delegation is that there are two slots for the receiver in a context, a state slot through which inst vars are accessed, and a self slot to which messages are sent. Â Normal sends merely duplicate the receiver into both slots. Â Delegating sends take state as an additional argument.
An alternative model is to dispense with direct inst var access (assuming the JIT will inline simple accessors in PICs, or simply by virtue of the use of machine code render accessors affordable) and provide accessors for delegators that indirect through the delegate.
Yes, that would be interesting to look at.
I can offer a bunch of code and papers on such a system. Hans Schippers, Dirk Janssens (University Antwerpen, Belgium), Robert Hirschfeld and myself implemented a thing we called delMDSOC. It started out as a machine model for aspect-oriented programming but turned out to be much more interesting. It's been implemented using COLA. *All* requests to objects are messages in this model, including instance variable access. *Everything* is done using delegation. Also, the system implements what Eliot wrote above about two receiver slots for state and self. Stuff is available from here: http://www.hpi.uni-potsdam.de/swa/projects/delmdsoc The implementation is optimised as far as COLA goes. There is much potential for improvement, and implementing some PIC-like caching schemes failed because of some COLA shortcomings I could not figure out how to fix. Best, Michael
Hi, On Sat, Mar 27, 2010 at 6:37 PM, Michael Haupt <mhaupt@gmail.com> wrote:
Stuff is available from here: http://www.hpi.uni-potsdam.de/swa/projects/delmdsoc
sorry, the page does not list all the relevant publications. * ECOOP'07 (description of model and semantics) http://www.hpi.uni-potsdam.de/hirschfeld/publications/media/HauptSchippers_2... * ACM SAC PSC'09 (description of implementation) http://www.hpi.uni-potsdam.de/hirschfeld/publications/media/SchippersHauptHi... There is one more paper at OOPSLA'08 that is rather theoretical, describing semantics mappings for several MDSOC languages and how the model preserves them ... for the sake of completeness: http://www.hpi.uni-potsdam.de/hirschfeld/publications/media/SchippersJanssen... Best, Michael
On Mar 27, 2010, at 6:37 PM, Michael Haupt wrote:
Hi,
first post, sorry for breaking any rules. ;-)
No rules broken..
Yes, that would be interesting to look at.
I can offer a bunch of code and papers on such a system. Hans Schippers, Dirk Janssens (University Antwerpen, Belgium), Robert Hirschfeld and myself implemented a thing we called delMDSOC. It started out as a machine model for aspect-oriented programming but turned out to be much more interesting. It's been implemented using COLA. *All* requests to objects are messages in this model, including instance variable access. *Everything* is done using delegation. Also, the system implements what Eliot wrote above about two receiver slots for state and self.
Stuff is available from here: http://www.hpi.uni-potsdam.de/swa/projects/delmdsoc
I was planing to look at your work closely (again)... the ECOOP paper is already printed on my Desk :-) Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
Hi Marcus, On Sat, Mar 27, 2010 at 6:51 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
I was planing to look at your work closely (again)... the ECOOP paper is already printed on my Desk :-)
ooh, interest! :-D If you have any (!) questions, you know how to reach me. :-) Best, Michael
A lot easier is to identify two points: 1) Why would you want to do that? 2) With the answer of 1) try to identify the image subsystem to apply the instrumentation. All other techniques are forms of interventionism, you will be trying to perform surgery to a terminal patient. Cheers, Hernán 2010/3/25 Mariano Martinez Peck <marianopeck@gmail.com>:
Hi folks. I want to intercept (and do something) for all objects. I know few approaches, for example:
1) using method wrappers (implementing run: aSelector with: arguments in: aReceiver) I can wrap all compiled methods of all classes and there I have the receiver.
2) become all objects to some kind of hacky object that stores the original object and that redefines the doesNotUnderstand:
But in 1) I need to change all CompiledMehtod of all classes. in 2) I have to become all the objects to another object
Then my question is, is there an easier way to intercept all messages send to all objects from the image side (not going to vm) and with a lower cost ? And if going to vm, how ?
Thanks.
Mariano
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
participants (8)
-
Alexandre Bergel -
Eliot Miranda -
Hernán Morales Durand -
John M McIntosh -
Marcus Denker -
Mariano Martinez Peck -
Michael Haupt -
Stéphane Ducasse