Hi! Stef told me the coming VM will expose some data about the execution to the image. This will be great for our profiling tools. What is the status? Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
Hello, This is true but this is a work in progress. Right now this feature is not stable. I will keep you in touch. I think in May I may have something stable enough (I will work with Eliot on April) but it will not be on the default VM. It will be available in the default pharo VM at some point but not now (maybe in a year ?). However, you may or may not be able to use it for your profiling tools. The idea is that you can get branch and send information from the previous executions of the method. However, the method needs to be cogged to have information available and once the cogged method is discarded the information is lost. It is really nice for hot spots but I don't know how it could work for a profiling tool. 2014-03-01 17:06 GMT+01:00 Alexandre Bergel <alexandre.bergel@me.com>:
Hi!
Stef told me the coming VM will expose some data about the execution to the image. This will be great for our profiling tools. What is the status?
Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
Ok, thanks for the update. The VM is a formidable thing in which everything happen. Exposing to the image whatâs going on in it is really pushing the innovation. Go go go! Alexandre On Mar 1, 2014, at 2:19 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello,
This is true but this is a work in progress. Right now this feature is not stable. I will keep you in touch. I think in May I may have something stable enough (I will work with Eliot on April) but it will not be on the default VM. It will be available in the default pharo VM at some point but not now (maybe in a year ?).
However, you may or may not be able to use it for your profiling tools. The idea is that you can get branch and send information from the previous executions of the method. However, the method needs to be cogged to have information available and once the cogged method is discarded the information is lost. It is really nice for hot spots but I don't know how it could work for a profiling tool.
2014-03-01 17:06 GMT+01:00 Alexandre Bergel <alexandre.bergel@me.com>: Hi!
Stef told me the coming VM will expose some data about the execution to the image. This will be great for our profiling tools. What is the status?
Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
On Sat, Mar 1, 2014 at 10:44 AM, Alexandre Bergel <alexandre.bergel@me.com>wrote:
Ok, thanks for the update.
The VM is a formidable thing in which everything happen. Exposing to the image what's going on in it is really pushing the innovation.
The computer is a formidable thing in which everything happens ;-). The VM is just mechanism. The real magic is up in the image ;-). What's exciting about Clément's and my work is that it will allow adaptive optimization, so a much faster system, but that optimization will be done at the image level and will optimize from compiled methods to compiled methods with inlining, on-the-fly. So this optimization will /not/ be in the VM. No one's done this before so we really will be pushing innovation.
Go go go!
On y va!
Alexandre
On Mar 1, 2014, at 2:19 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello,
This is true but this is a work in progress. Right now this feature is not stable. I will keep you in touch. I think in May I may have something stable enough (I will work with Eliot on April) but it will not be on the default VM. It will be available in the default pharo VM at some point but not now (maybe in a year ?).
However, you may or may not be able to use it for your profiling tools. The idea is that you can get branch and send information from the previous executions of the method. However, the method needs to be cogged to have information available and once the cogged method is discarded the information is lost. It is really nice for hot spots but I don't know how it could work for a profiling tool.
2014-03-01 17:06 GMT+01:00 Alexandre Bergel <alexandre.bergel@me.com>: Hi!
Stef told me the coming VM will expose some data about the execution to the image. This will be great for our profiling tools. What is the status?
Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
-- best, Eliot
Excellent! Eager to try it!!! Alexandre
Le 01-03-2014 à 16:01, Eliot Miranda <eliot.miranda@gmail.com> a écrit :
On Sat, Mar 1, 2014 at 10:44 AM, Alexandre Bergel <alexandre.bergel@me.com> wrote: Ok, thanks for the update.
The VM is a formidable thing in which everything happen. Exposing to the image whatâs going on in it is really pushing the innovation.
The computer is a formidable thing in which everything happens ;-). The VM is just mechanism. The real magic is up in the image ;-).
What's exciting about Clément's and my work is that it will allow adaptive optimization, so a much faster system, but that optimization will be done at the image level and will optimize from compiled methods to compiled methods with inlining, on-the-fly. So this optimization will /not/ be in the VM. No one's done this before so we really will be pushing innovation.
Go go go!
On y va!
Alexandre
On Mar 1, 2014, at 2:19 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello,
This is true but this is a work in progress. Right now this feature is not stable. I will keep you in touch. I think in May I may have something stable enough (I will work with Eliot on April) but it will not be on the default VM. It will be available in the default pharo VM at some point but not now (maybe in a year ?).
However, you may or may not be able to use it for your profiling tools. The idea is that you can get branch and send information from the previous executions of the method. However, the method needs to be cogged to have information available and once the cogged method is discarded the information is lost. It is really nice for hot spots but I don't know how it could work for a profiling tool.
2014-03-01 17:06 GMT+01:00 Alexandre Bergel <alexandre.bergel@me.com>: Hi!
Stef told me the coming VM will expose some data about the execution to the image. This will be great for our profiling tools. What is the status?
Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
-- best, Eliot
Hi: On 01 Mar 2014, at 19:44, Alexandre Bergel <alexandre.bergel@me.com> wrote:
The VM is a formidable thing in which everything happen. Exposing to the image whatâs going on in it is really pushing the innovation.
Perhaps something that could be relevant in case someone decides to implement such an interface in Cog: Just one, pretty old example: http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#EventIndex You can get access to many different information in terms of âeventsâ on the JVM. For building a profiler, more than sufficient. And, at least in my personal opinion also something that would be desirable for Cog, perhaps in a slightly more modern design, but with a similar flavor. Best regards Stefan -- Stefan Marr INRIA Lille - Nord Europe http://stefan-marr.de/research/
Hi Stefan, On Sat, Mar 1, 2014 at 12:55 PM, Stefan Marr <smalltalk@stefan-marr.de>wrote:
Hi:
On 01 Mar 2014, at 19:44, Alexandre Bergel <alexandre.bergel@me.com> wrote:
The VM is a formidable thing in which everything happen. Exposing to the image what's going on in it is really pushing the innovation.
Perhaps something that could be relevant in case someone decides to implement such an interface in Cog: Just one, pretty old example: http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#EventIndex
Interesting list. Assuming the list refers to events one can handle in Smalltalk, then let me go through these and see which we need, don't need or already have: Event Index - Breakpoint<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#Breakpoint> we already have this. An illegal bytecode will send an error message to the current context. See my MethodMassage package. We use this at Cadence to implement coverage. - Class File Load Hook<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassFileLoad...> not needed. classes are loaded by Smalltalk code, not the VM. - *Class Load <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassLoad>* not needed. ditto - *Class Prepare <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassPrepare>* not needed. ditto - *Compiled Method Load <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#CompiledMethodLoad>* not needed. methods are loaded by Smalltalk code, not the VM. - *Compiled Method Unload <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#CompiledMethodUnload>* not needed. ditto - Data Dump Request<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#DataDumpReque...> to what extent is snapshot adequate? Has been used for adequate image debugging (and of course we can serialize processes so people are using things like Fuel to save the stack traces of processes that encounter errors). Not adequate for VM debugging: the heap may be invalid and not saveable; shapshot load does more than merely fill memory; it swizzles etc, and these operations can and will fail on an invalid heap. Personally I think things like -blockOnError which causes the VM to block rather than exit on error, allowing one to attach gdb to a crashed VM process is more useful. - *Dynamic Code Generated <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#DynamicCodeGenerated>* not needed. methods are loaded by Smalltalk code, not the VM. - *Exception <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#Exception>* not needed. Smalltalk's exception system has resume semantics, not retry, so there's no need to ask to see an exception at source; the source of the exception can be examined after the fact (unlike Java, which has restart semantics). In fact folks like Avi Bryant (and I believe him) think that the business opportunity with hadoop/map-reduce is indeed Smalltalk's exception semantics which allow live debugging of errors. - *Exception Catch <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ExceptionCatch>* not needed. ditto - Field Access<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FieldAccess> Quite possibly. If one is using classical Smalltalk there is no linguistic hook for field access. With Pharo's slots then presumably transforming methods to insert traps on field access is trivial (and databases like GemStone have done similar things with brute bytecode manipulation). With Newspeak, which has no direct inst var access, this can be subsumed by MethodEntry (see below). So whether this is needed or not depends on either language evolution or good bytecode manipulation tools; if these are available this is not needed. - Field Modification<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FieldModifica...> Quite possibly. Again the approaches outlined above can be done (as gemStone has done in the past). But we nearly have this. I've implemented per-instance immutability for the old Newspeak VM (a variant of the Squeak interpreter) and this is easy to fold into Cog (and indeed the Interpreter VM). GemStone engineers prefer per-instance immutability than their own bytecode modification. I think I agree. I'd rather depend on immutability (which has other uses such as immutable literals) than a special VM hook. - Frame Pop<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FramePop> Hmm. Possibly. IIRC, I think method wrappers provide a manageable way to do this. - Garbage Collection Finish<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#GarbageCollec...> Indeed. There are hacks to get this. Further there are many ways to implement this. e.g. is it a callback on finishing a form of GC, or is it merely the signalling of a semaphore which has some Smalltalk process waiting on it (as is the case in VW)? - Garbage Collection Start<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#GarbageCollec...> OK, but what are you going to do when you catch this? Arguably there's no memory with which to do anything at the point that you get this. Give me a scenario and I'll consider it. - Method Entry<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MethodEntry> Not sure we need this because all method calls are virtual and hence one can use proxies (MNU hook) or method wrappers. - Method Exit<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MethodExit> Ditto. - Monitor Contended Enter<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorConten...> Not relevant. Smalltalk doesn't have synchronized methods. - Monitor Contended Entered<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorConten...> Ditto. - *Monitor Wait <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorWait>* Ditto. - *Monitor Waited <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorWaited>* Ditto. - Native Method Bind<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#NativeMethodB...> OK. Not really relevant if we refactor dll loading and function lookup as in Alien (and VW) so that the image does the work and allows one to implement the hook without VM support. - Object Free<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ObjectFree> Hmmm. Is this for finalization or something else? In either case I think that Ephemerons fit the bill and Cog Spur supports ephemerons. So we'll have this before the end of the year. - Resource Exhausted<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ResourceExhau...> Hmmm, I think this is not relevant. Resources such as memory produce catchable primitive failures. Likewise for OS resources. if things are engineered right then exhaustion fo things like file handles should produce exceptions in the image. - Single Step<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#SingleStep> Not relevant. We already have ways of making Smalltalk single-step, heh, heh. - Thread End<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ThreadEnd> If this applies to Process, then easy. Change fork: et al to add code on thread termination. Native threads are a different issue. But as yet a non-issue. We don't have them yet, although a prototype VM is there to revive when resources allow. - Thread Start<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ThreadStart> Ditto. - VM Death Event<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMDeath> Seems like a non-sequitur to me. If the VM has died then there's nothing reliable the system above it can do. However, see below... - VM Initialization Event<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMInit> The existing startUp registration mechanism seems adequate to me... - VM Object Allocation<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMObjectAlloc> AGain lots of hooks to allow this. The link os pretty vague on what this might mean. They mention using other mechanisms to instrument this. - *VM Start Event <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMStart>* Again, the existing startUp registration mechanism seems adequate to me... However, if this list refers to sub-image level debugging (and that's something I do all the time) then again this isn't relevant. We have the simulator which is a fabulous tool for intercepting any and all of the above. Gdb is less convenient but can be used by someone knowledgeable. You can get access to many different information in terms of 'events' on
the JVM. For building a profiler, more than sufficient. And, at least in my personal opinion also something that would be desirable for Cog, perhaps in a slightly more modern design, but with a similar flavor.
So how about responding to the above breakdown with a sketch of what events remain, and then speculate on how they might be packaged as events. -- best, Eliot
Hey, It's fun because I looked at the JVM list and most point looked irrelevant as you described. It seems that the only thing we miss is the GC hook: we could have a selector in special object array to call at each GC on the 'Smalltalk' object to trigger a method in the image that would be by default an empty method. Field access is a solved problem with slots, definitely. 2014-03-01 22:35 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Stefan,
On Sat, Mar 1, 2014 at 12:55 PM, Stefan Marr <smalltalk@stefan-marr.de>wrote:
Hi:
On 01 Mar 2014, at 19:44, Alexandre Bergel <alexandre.bergel@me.com> wrote:
The VM is a formidable thing in which everything happen. Exposing to the image what's going on in it is really pushing the innovation.
Perhaps something that could be relevant in case someone decides to implement such an interface in Cog: Just one, pretty old example: http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#EventIndex
Interesting list. Assuming the list refers to events one can handle in Smalltalk, then let me go through these and see which we need, don't need or already have:
Event Index
- Breakpoint<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#Breakpoint> we already have this. An illegal bytecode will send an error message to the current context. See my MethodMassage package. We use this at Cadence to implement coverage. - Class File Load Hook<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassFileLoad...> not needed. classes are loaded by Smalltalk code, not the VM. - *Class Load <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassLoad>* not needed. ditto - *Class Prepare <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassPrepare>* not needed. ditto - *Compiled Method Load <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#CompiledMethodLoad>* not needed. methods are loaded by Smalltalk code, not the VM. - *Compiled Method Unload <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#CompiledMethodUnload>* not needed. ditto - Data Dump Request<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#DataDumpReque...> to what extent is snapshot adequate? Has been used for adequate image debugging (and of course we can serialize processes so people are using things like Fuel to save the stack traces of processes that encounter errors). Not adequate for VM debugging: the heap may be invalid and not saveable; shapshot load does more than merely fill memory; it swizzles etc, and these operations can and will fail on an invalid heap. Personally I think things like -blockOnError which causes the VM to block rather than exit on error, allowing one to attach gdb to a crashed VM process is more useful. - *Dynamic Code Generated <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#DynamicCodeGenerated>* not needed. methods are loaded by Smalltalk code, not the VM. - *Exception <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#Exception>* not needed. Smalltalk's exception system has resume semantics, not retry, so there's no need to ask to see an exception at source; the source of the exception can be examined after the fact (unlike Java, which has restart semantics). In fact folks like Avi Bryant (and I believe him) think that the business opportunity with hadoop/map-reduce is indeed Smalltalk's exception semantics which allow live debugging of errors. - *Exception Catch <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ExceptionCatch>* not needed. ditto - Field Access<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FieldAccess> Quite possibly. If one is using classical Smalltalk there is no linguistic hook for field access. With Pharo's slots then presumably transforming methods to insert traps on field access is trivial (and databases like GemStone have done similar things with brute bytecode manipulation). With Newspeak, which has no direct inst var access, this can be subsumed by MethodEntry (see below). So whether this is needed or not depends on either language evolution or good bytecode manipulation tools; if these are available this is not needed. - Field Modification<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FieldModifica...> Quite possibly. Again the approaches outlined above can be done (as gemStone has done in the past). But we nearly have this. I've implemented per-instance immutability for the old Newspeak VM (a variant of the Squeak interpreter) and this is easy to fold into Cog (and indeed the Interpreter VM). GemStone engineers prefer per-instance immutability than their own bytecode modification. I think I agree. I'd rather depend on immutability (which has other uses such as immutable literals) than a special VM hook. - Frame Pop<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FramePop> Hmm. Possibly. IIRC, I think method wrappers provide a manageable way to do this. - Garbage Collection Finish<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#GarbageCollec...> Indeed. There are hacks to get this. Further there are many ways to implement this. e.g. is it a callback on finishing a form of GC, or is it merely the signalling of a semaphore which has some Smalltalk process waiting on it (as is the case in VW)? - Garbage Collection Start<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#GarbageCollec...> OK, but what are you going to do when you catch this? Arguably there's no memory with which to do anything at the point that you get this. Give me a scenario and I'll consider it. - Method Entry<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MethodEntry> Not sure we need this because all method calls are virtual and hence one can use proxies (MNU hook) or method wrappers. - Method Exit<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MethodExit> Ditto. - Monitor Contended Enter<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorConten...> Not relevant. Smalltalk doesn't have synchronized methods. - Monitor Contended Entered<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorConten...> Ditto. - *Monitor Wait <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorWait>* Ditto. - *Monitor Waited <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorWaited>* Ditto. - Native Method Bind<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#NativeMethodB...> OK. Not really relevant if we refactor dll loading and function lookup as in Alien (and VW) so that the image does the work and allows one to implement the hook without VM support. - Object Free<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ObjectFree> Hmmm. Is this for finalization or something else? In either case I think that Ephemerons fit the bill and Cog Spur supports ephemerons. So we'll have this before the end of the year. - Resource Exhausted<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ResourceExhau...> Hmmm, I think this is not relevant. Resources such as memory produce catchable primitive failures. Likewise for OS resources. if things are engineered right then exhaustion fo things like file handles should produce exceptions in the image. - Single Step<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#SingleStep> Not relevant. We already have ways of making Smalltalk single-step, heh, heh. - Thread End<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ThreadEnd> If this applies to Process, then easy. Change fork: et al to add code on thread termination. Native threads are a different issue. But as yet a non-issue. We don't have them yet, although a prototype VM is there to revive when resources allow. - Thread Start<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ThreadStart> Ditto. - VM Death Event<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMDeath> Seems like a non-sequitur to me. If the VM has died then there's nothing reliable the system above it can do. However, see below... - VM Initialization Event<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMInit> The existing startUp registration mechanism seems adequate to me... - VM Object Allocation<http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMObjectAlloc> AGain lots of hooks to allow this. The link os pretty vague on what this might mean. They mention using other mechanisms to instrument this. - *VM Start Event <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMStart>* Again, the existing startUp registration mechanism seems adequate to me...
However, if this list refers to sub-image level debugging (and that's something I do all the time) then again this isn't relevant. We have the simulator which is a fabulous tool for intercepting any and all of the above. Gdb is less convenient but can be used by someone knowledgeable.
You can get access to many different information in terms of 'events' on
the JVM. For building a profiler, more than sufficient. And, at least in my personal opinion also something that would be desirable for Cog, perhaps in a slightly more modern design, but with a similar flavor.
So how about responding to the above breakdown with a sketch of what events remain, and then speculate on how they might be packaged as events.
-- best, Eliot
Maybe I'm missing something obvious, but from my point of view the fundamental difference here is that with JVM's JPDA triggering of event actually **does not** affect the monitored system. If these events are handled by the image, it is quite tricky to figure out which one is caused by "application" and which one is caused by the event handling machinery. For instance, if (non-empty) method is called after a GC, this handler may trigger another GC, which may... That brings me to another, if you do in-image profiling (event handling), data could be quite different than what would you get using out-of-image profiler. <personal experience:> I've used (and implemented some :-) both - Smalltalk in-image tools and out-of-image tools, namely JPDA and DTRACE/STAP. For serious profiling and monitoring, I always use out-of-image approach if possible. Jan On 01/03/14 23:34, Clément Bera wrote:
Hey,
It's fun because I looked at the JVM list and most point looked irrelevant as you described.
It seems that the only thing we miss is the GC hook: we could have a selector in special object array to call at each GC on the 'Smalltalk' object to trigger a method in the image that would be by default an empty method.
Field access is a solved problem with slots, definitely.
2014-03-01 22:35 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com <mailto:eliot.miranda@gmail.com>>:
Hi Stefan,
On Sat, Mar 1, 2014 at 12:55 PM, Stefan Marr <smalltalk@stefan-marr.de <mailto:smalltalk@stefan-marr.de>> wrote:
Hi:
On 01 Mar 2014, at 19:44, Alexandre Bergel <alexandre.bergel@me.com <mailto:alexandre.bergel@me.com>> wrote:
> The VM is a formidable thing in which everything happen. Exposing to the image whatâs going on in it is really pushing the innovation.
Perhaps something that could be relevant in case someone decides to implement such an interface in Cog: Just one, pretty old example: http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#EventIndex
Interesting list. Assuming the list refers to events one can handle in Smalltalk, then let me go through these and see which we need, don't need or already have:
Event Index
* Breakpoint <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#Breakpoint> we already have this. An illegal bytecode will send an error message to the current context. See my MethodMassage package. We use this at Cadence to implement coverage. * Class File Load Hook <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassFileLoad...> not needed. classes are loaded by Smalltalk code, not the VM. * *Class Load <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassLoad>* not needed. ditto * *Class Prepare <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassPrepare>* not needed. ditto * *Compiled Method Load <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#CompiledMethodLoad>* not needed. methods are loaded by Smalltalk code, not the VM. * *Compiled Method Unload <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#CompiledMethodUnload>* not needed. ditto * Data Dump Request <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#DataDumpReque...> to what extent is snapshot adequate? Has been used for adequate image debugging (and of course we can serialize processes so people are using things like Fuel to save the stack traces of processes that encounter errors). Not adequate for VM debugging: the heap may be invalid and not saveable; shapshot load does more than merely fill memory; it swizzles etc, and these operations can and will fail on an invalid heap. Personally I think things like -blockOnError which causes the VM to block rather than exit on error, allowing one to attach gdb to a crashed VM process is more useful. * *Dynamic Code Generated <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#DynamicCodeGenerated>* not needed. methods are loaded by Smalltalk code, not the VM. * *Exception <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#Exception>* not needed. Smalltalk's exception system has resume semantics, not retry, so there's no need to ask to see an exception at source; the source of the exception can be examined after the fact (unlike Java, which has restart semantics). In fact folks like Avi Bryant (and I believe him) think that the business opportunity with hadoop/map-reduce is indeed Smalltalk's exception semantics which allow live debugging of errors. * *Exception Catch <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ExceptionCatch>* not needed. ditto * Field Access <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FieldAccess> Quite possibly. If one is using classical Smalltalk there is no linguistic hook for field access. With Pharo's slots then presumably transforming methods to insert traps on field access is trivial (and databases like GemStone have done similar things with brute bytecode manipulation). With Newspeak, which has no direct inst var access, this can be subsumed by MethodEntry (see below). So whether this is needed or not depends on either language evolution or good bytecode manipulation tools; if these are available this is not needed. * Field Modification <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FieldModifica...> Quite possibly. Again the approaches outlined above can be done (as gemStone has done in the past). But we nearly have this. I've implemented per-instance immutability for the old Newspeak VM (a variant of the Squeak interpreter) and this is easy to fold into Cog (and indeed the Interpreter VM). GemStone engineers prefer per-instance immutability than their own bytecode modification. I think I agree. I'd rather depend on immutability (which has other uses such as immutable literals) than a special VM hook. * Frame Pop <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FramePop> Hmm. Possibly. IIRC, I think method wrappers provide a manageable way to do this. * Garbage Collection Finish <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#GarbageCollec...> Indeed. There are hacks to get this. Further there are many ways to implement this. e.g. is it a callback on finishing a form of GC, or is it merely the signalling of a semaphore which has some Smalltalk process waiting on it (as is the case in VW)? * Garbage Collection Start <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#GarbageCollec...> OK, but what are you going to do when you catch this? Arguably there's no memory with which to do anything at the point that you get this. Give me a scenario and I'll consider it. * Method Entry <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MethodEntry> Not sure we need this because all method calls are virtual and hence one can use proxies (MNU hook) or method wrappers. * Method Exit <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MethodExit> Ditto. * Monitor Contended Enter <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorConten...> Not relevant. Smalltalk doesn't have synchronized methods. * Monitor Contended Entered <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorConten...> Ditto. * *Monitor Wait <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorWait>* Ditto. * *Monitor Waited <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorWaited>* Ditto. * Native Method Bind <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#NativeMethodB...> OK. Not really relevant if we refactor dll loading and function lookup as in Alien (and VW) so that the image does the work and allows one to implement the hook without VM support. * Object Free <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ObjectFree> Hmmm. Is this for finalization or something else? In either case I think that Ephemerons fit the bill and Cog Spur supports ephemerons. So we'll have this before the end of the year. * Resource Exhausted <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ResourceExhau...> Hmmm, I think this is not relevant. Resources such as memory produce catchable primitive failures. Likewise for OS resources. if things are engineered right then exhaustion fo things like file handles should produce exceptions in the image. * Single Step <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#SingleStep> Not relevant. We already have ways of making Smalltalk single-step, heh, heh. * Thread End <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ThreadEnd> If this applies to Process, then easy. Change fork: et al to add code on thread termination. Native threads are a different issue. But as yet a non-issue. We don't have them yet, although a prototype VM is there to revive when resources allow. * Thread Start <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ThreadStart> Ditto. * VM Death Event <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMDeath> Seems like a non-sequitur to me. If the VM has died then there's nothing reliable the system above it can do. However, see below... * VM Initialization Event <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMInit> The existing startUp registration mechanism seems adequate to me... * VM Object Allocation <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMObjectAlloc> AGain lots of hooks to allow this. The link os pretty vague on what this might mean. They mention using other mechanisms to instrument this. * *VM Start Event <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMStart>* Again, the existing startUp registration mechanism seems adequate to me...
However, if this list refers to sub-image level debugging (and that's something I do all the time) then again this isn't relevant. We have the simulator which is a fabulous tool for intercepting any and all of the above. Gdb is less convenient but can be used by someone knowledgeable.
You can get access to many different information in terms of âeventsâ on the JVM. For building a profiler, more than sufficient. And, at least in my personal opinion also something that would be desirable for Cog, perhaps in a slightly more modern design, but with a similar flavor.
So how about responding to the above breakdown with a sketch of what events remain, and then speculate on how they might be packaged as events.
-- best, Eliot
On Sat, Mar 1, 2014 at 4:09 PM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:
Maybe I'm missing something obvious, but from my point of view the fundamental difference here is that with JVM's JPDA triggering of event actually **does not** affect the monitored system.
If these events are handled by the image, it is quite tricky to figure out which one is caused by "application" and which one is caused by the event handling machinery. For instance, if (non-empty) method is called after a GC, this handler may trigger another GC, which may...
That brings me to another, if you do in-image profiling (event handling), data could be quite different than what would you get using out-of-image profiler.
<personal experience:> I've used (and implemented some :-) both - Smalltalk in-image tools and out-of-image tools, namely JPDA and DTRACE/STAP. For serious profiling and monitoring, I always use out-of-image approach if possible.
I disagree. What I use for serious VM profiling is a hybrid approach that collects data in the VM but uses the image to display and interact with it. The VMProfiler, which is in CogTools in the VMMaker repository is IMNAAHO a very nice VM profiler. In the VM there is a thread that interrupts the main VM thread periodically, sampling its pc and storing this in a circular buffer. The thread and buffer are manipulated by a couple of primitives. Alongside is a platform-specific plugin to reveal the dlls loaded in the VM. The JIT reports current machine code methods and PICs through a primitive. Above the VM, the profiler uses OSProcess to invoke tools like objdump or nm to extract symbols and their addresses from the VM and loaded dlls, and uses morphic to draw graphs of the pc activity, allowing one to zoom in to look at specific code sequences. It generates reports such as the following: /Users/eliot/Cog/oscogvm/spurcogbuild/macbuild/Fast.app/Contents/MacOS/Squeak 3/4/2014 eden size: 2,603,344 stack pages: 160 code size: 1,048,576 0 tinyBenchmarks gc prior. clear prior. 4.945 seconds; sampling frequency 1451 hz 7149 samples in the VM (7175 samples in the entire program) 99.64% of total 6533 samples in generated vm code 91.38% of entire vm (91.05% of total) 616 samples in vanilla vm code 8.62% of entire vm ( 8.59% of total) % of generated vm code (% of total) (samples) (cumulative) 49.04% (44.66%) Integer>>benchFib (3204) (49.04%) 19.10% (17.39%) Integer>>benchmark (1248) (68.15%) 15.18% (13.83%) Object>>at:put: (992) (83.33%) 12.23% (11.14%) SmallInteger>>+ (799) (95.56%) 4.18% ( 3.80%) Object>>at: (273) (99.74%) 0.15% ( 0.14%) ceMethodAbort0Args (10) (99.89%) 0.05% ( 0.04%) ceBaseFram...Trampoline(3) (99.94%) 0.02% ( 0.01%) ceEnterCog...ceiverReg (1) (99.95%) 0.02% ( 0.01%) Array>>repla...startingAt:(1) (99.97%) 0.02% ( 0.01%) Sequenceabl...om:to:put: (1) (99.98%) 0.02% ( 0.01%) SmallInteger>>- (1) (100.0%) % of vanilla vm code (% of total) (samples) (cumulative) 27.27% ( 2.34%) scavengingGCTenuringIf (168) (27.27%) 22.89% ( 1.97%) moveFramesInthroughtoPage (141) (50.16%) 20.13% ( 1.73%) primitiveStringReplace (124) (70.29%) 7.63% ( 0.66%) instantiateClassindexableSize (47) (77.92%) 6.49% ( 0.56%) marryFrameSP (40) (84.42%) 4.71% ( 0.40%) handleStackOverflow (29) (89.12%) 4.06% ( 0.35%) ceBaseFrameReturn (25) (93.18%) 1.79% ( 0.15%) ceStackOverflow (11) (94.97%) 1.30% ( 0.11%) scavengeReferentsOf (8) (96.27%) 0.65% ( 0.06%) scavengeLoop (4) (96.92%) 0.65% ( 0.06%) numStrongSlots...eronInactiveIf (4) (97.56%) 0.49% ( 0.04%) ioUTCMicroseconds (3) (98.05%) 0.32% ( 0.03%) checkForEvent...ontextSwitch (2) (98.38%) 0.32% ( 0.03%) primitiveNewWithArg (2) (98.70%) 0.32% ( 0.03%) unlinkSolitaryFreeTreeNode (2) (99.03%) 0.32% ( 0.03%) minCogMethodAddress (2) (99.35%) 0.16% ( 0.01%) allocateSlotsInO...ormatclassIndex(1) (99.51%) 0.16% ( 0.01%) copyAndForward (1) (99.68%) 0.16% ( 0.01%) processWeaklings (1) (99.84%) 0.16% ( 0.01%) returnToExecuti...tContextSwitch(1) (100.0%) **Memory** old +412,400 bytes young -412,400 bytes used +0 bytes free +0 bytes **GCs** full 0 totalling 0ms (0% elapsed time) incr 425 totalling 138ms (2.8000000000000003% elapsed time), avg 0.32ms tenures 0 root table 0 overflows **Compiled Code Compactions** 0 totalling 0ms (0% elapsed time) **Events** Process switches 52 (11 per second) ioProcessEvents calls 245 (50 per second) Interrupt checks 2866 (580 per second) Event checks 3189 (645 per second) Stack overflows 1043888 (211100 per second) Stack page divorces 0 (0 per second) Here's what the interface looks like: [image: Inline image 1] In this case the region of code being displayed is where the JIT keeps the machine code for methods, and the list to the left is some of the methods in the code zone. The pink graph in the main region is the integral of the black pc histogram. Jan
On 01/03/14 23:34, Clément Bera wrote:
Hey,
It's fun because I looked at the JVM list and most point looked irrelevant as you described.
It seems that the only thing we miss is the GC hook: we could have a selector in special object array to call at each GC on the 'Smalltalk' object to trigger a method in the image that would be by default an empty method.
Field access is a solved problem with slots, definitely.
2014-03-01 22:35 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com <mailto:eliot.miranda@gmail.com>>:
Hi Stefan,
On Sat, Mar 1, 2014 at 12:55 PM, Stefan Marr <smalltalk@stefan-marr.de <mailto:smalltalk@stefan-marr.de>> wrote:
Hi:
On 01 Mar 2014, at 19:44, Alexandre Bergel <alexandre.bergel@me.com <mailto:alexandre.bergel@me.com>> wrote:
> The VM is a formidable thing in which everything happen. Exposing to the image what's going on in it is really pushing the innovation.
Perhaps something that could be relevant in case someone decides to implement such an interface in Cog: Just one, pretty old example: http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti. html#EventIndex
Interesting list. Assuming the list refers to events one can handle in Smalltalk, then let me go through these and see which we need, don't need or already have:
Event Index
* Breakpoint <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#Breakpoint>
we already have this. An illegal bytecode will send an error message to the current context. See my MethodMassage package. We use this at Cadence to implement coverage. * Class File Load Hook <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#ClassFileLoadHook>
not needed. classes are loaded by Smalltalk code, not the VM. * *Class Load <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#ClassLoad>* not needed. ditto * *Class Prepare <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#ClassPrepare>* not needed. ditto * *Compiled Method Load <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#CompiledMethodLoad>*
not needed. methods are loaded by Smalltalk code, not the VM. * *Compiled Method Unload <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html# CompiledMethodUnload>* not needed. ditto * Data Dump Request <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#DataDumpRequest>
to what extent is snapshot adequate? Has been used for adequate image debugging (and of course we can serialize processes so people are using things like Fuel to save the stack traces of processes that encounter errors). Not adequate for VM debugging: the heap may be invalid and not saveable; shapshot load does more than merely fill memory; it swizzles etc, and these operations can and will fail on an invalid heap. Personally I think things like -blockOnError which causes the VM to block rather than exit on error, allowing one to attach gdb to a crashed VM process is more useful. * *Dynamic Code Generated <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html# DynamicCodeGenerated>*
not needed. methods are loaded by Smalltalk code, not the VM. * *Exception <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#Exception>*
not needed. Smalltalk's exception system has resume semantics, not retry, so there's no need to ask to see an exception at source; the source of the exception can be examined after the fact (unlike Java, which has restart semantics). In fact folks like Avi Bryant (and I believe him) think that the business opportunity with hadoop/map-reduce is indeed Smalltalk's exception semantics which allow live debugging of errors. * *Exception Catch <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#ExceptionCatch>* not needed. ditto * Field Access <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#FieldAccess>
Quite possibly. If one is using classical Smalltalk there is no linguistic hook for field access. With Pharo's slots then presumably transforming methods to insert traps on field access is trivial (and databases like GemStone have done similar things with brute bytecode manipulation). With Newspeak, which has no direct inst var access, this can be subsumed by MethodEntry (see below). So whether this is needed or not depends on either language evolution or good bytecode manipulation tools; if these are available this is not needed. * Field Modification <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#FieldModification>
Quite possibly. Again the approaches outlined above can be done (as gemStone has done in the past). But we nearly have this. I've implemented per-instance immutability for the old Newspeak VM (a variant of the Squeak interpreter) and this is easy to fold into Cog (and indeed the Interpreter VM). GemStone engineers prefer per-instance immutability than their own bytecode modification. I think I agree. I'd rather depend on immutability (which has other uses such as immutable literals) than a special VM hook. * Frame Pop <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#FramePop>
Hmm. Possibly. IIRC, I think method wrappers provide a manageable way to do this. * Garbage Collection Finish <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html# GarbageCollectionFinish>
Indeed. There are hacks to get this. Further there are many ways to implement this. e.g. is it a callback on finishing a form of GC, or is it merely the signalling of a semaphore which has some Smalltalk process waiting on it (as is the case in VW)? * Garbage Collection Start <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html# GarbageCollectionStart>
OK, but what are you going to do when you catch this? Arguably there's no memory with which to do anything at the point that you get this. Give me a scenario and I'll consider it. * Method Entry <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#MethodEntry>
Not sure we need this because all method calls are virtual and hence one can use proxies (MNU hook) or method wrappers. * Method Exit <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#MethodExit> Ditto. * Monitor Contended Enter <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html# MonitorContendedEnter>
Not relevant. Smalltalk doesn't have synchronized methods. * Monitor Contended Entered <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html# MonitorContendedEntered> Ditto. * *Monitor Wait <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#MonitorWait>* Ditto. * *Monitor Waited <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#MonitorWaited>* Ditto. * Native Method Bind <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#NativeMethodBind>
OK. Not really relevant if we refactor dll loading and function lookup as in Alien (and VW) so that the image does the work and allows one to implement the hook without VM support. * Object Free <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#ObjectFree>
Hmmm. Is this for finalization or something else? In either case I think that Ephemerons fit the bill and Cog Spur supports ephemerons. So we'll have this before the end of the year. * Resource Exhausted <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#ResourceExhausted>
Hmmm, I think this is not relevant. Resources such as memory produce catchable primitive failures. Likewise for OS resources. if things are engineered right then exhaustion fo things like file handles should produce exceptions in the image. * Single Step <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#SingleStep>
Not relevant. We already have ways of making Smalltalk single-step, heh, heh. * Thread End <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#ThreadEnd>
If this applies to Process, then easy. Change fork: et al to add code on thread termination. Native threads are a different issue. But as yet a non-issue. We don't have them yet, although a prototype VM is there to revive when resources allow. * Thread Start <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#ThreadStart> Ditto. * VM Death Event <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#VMDeath>
Seems like a non-sequitur to me. If the VM has died then there's nothing reliable the system above it can do. However, see below... * VM Initialization Event <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#VMInit>
The existing startUp registration mechanism seems adequate to me... * VM Object Allocation <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#VMObjectAlloc>
AGain lots of hooks to allow this. The link os pretty vague on what this might mean. They mention using other mechanisms to instrument this. * *VM Start Event <http://docs.oracle.com/javase/6/docs/platform/jvmti/ jvmti.html#VMStart>*
Again, the existing startUp registration mechanism seems adequate to me...
However, if this list refers to sub-image level debugging (and that's something I do all the time) then again this isn't relevant. We have the simulator which is a fabulous tool for intercepting any and all of the above. Gdb is less convenient but can be used by someone knowledgeable.
You can get access to many different information in terms of 'events' on the JVM. For building a profiler, more than sufficient. And, at least in my personal opinion also something that would be desirable for Cog, perhaps in a slightly more modern design, but with a similar flavor.
So how about responding to the above breakdown with a sketch of what events remain, and then speculate on how they might be packaged as events.
-- best, Eliot
-- best, Eliot
On 04/03/14 17:37, Eliot Miranda wrote:
On Sat, Mar 1, 2014 at 4:09 PM, Jan Vrany <jan.vrany@fit.cvut.cz <mailto:jan.vrany@fit.cvut.cz>> wrote:
Maybe I'm missing something obvious, but from my point of view the fundamental difference here is that with JVM's JPDA triggering of event actually **does not** affect the monitored system.
If these events are handled by the image, it is quite tricky to figure out which one is caused by "application" and which one is caused by the event handling machinery. For instance, if (non-empty) method is called after a GC, this handler may trigger another GC, which may...
That brings me to another, if you do in-image profiling (event handling), data could be quite different than what would you get using out-of-image profiler.
<personal experience:> I've used (and implemented some :-) both - Smalltalk in-image tools and out-of-image tools, namely JPDA and DTRACE/STAP. For serious profiling and monitoring, I always use out-of-image approach if possible.
I disagree.
Fair enough :-) What I use for serious VM profiling is a hybrid approach
that collects data in the VM but uses the image to display and interact with it. The VMProfiler, which is in CogTools in the VMMaker repository is IMNAAHO a very nice VM profiler.
Looks good, indeed.
In the VM there is a thread that interrupts the main VM thread periodically, sampling its pc and storing this in a circular buffer. The thread and buffer are manipulated by a couple of primitives. Alongside is a platform-specific plugin to reveal the dlls loaded in the VM. The JIT reports current machine code methods and PICs through a primitive.
Looks like everybody has a slightly different techniques and its own tools :-) I collect data in VM but the VM reports them to "outside" (via log file or kind of IPC). Then I analyze data post-mortem using various tools, some of them are hand-crafted, some not. I found KCachegrind quite nice. Anyway, the interesting question to me is what are the advantages of your approach? Best, Jan
Hi Jan, On Mar 6, 2014, at 2:29 PM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:
On 04/03/14 17:37, Eliot Miranda wrote:
On Sat, Mar 1, 2014 at 4:09 PM, Jan Vrany <jan.vrany@fit.cvut.cz <mailto:jan.vrany@fit.cvut.cz>> wrote:
Maybe I'm missing something obvious, but from my point of view the fundamental difference here is that with JVM's JPDA triggering of event actually **does not** affect the monitored system.
If these events are handled by the image, it is quite tricky to figure out which one is caused by "application" and which one is caused by the event handling machinery. For instance, if (non-empty) method is called after a GC, this handler may trigger another GC, which may...
That brings me to another, if you do in-image profiling (event handling), data could be quite different than what would you get using out-of-image profiler.
<personal experience:> I've used (and implemented some :-) both - Smalltalk in-image tools and out-of-image tools, namely JPDA and DTRACE/STAP. For serious profiling and monitoring, I always use out-of-image approach if possible.
I disagree.
Fair enough :-)
What I use for serious VM profiling is a hybrid approach
that collects data in the VM but uses the image to display and interact with it. The VMProfiler, which is in CogTools in the VMMaker repository is IMNAAHO a very nice VM profiler.
Looks good, indeed.
In the VM there is a thread that interrupts the main VM thread periodically, sampling its pc and storing this in a circular buffer. The thread and buffer are manipulated by a couple of primitives. Alongside is a platform-specific plugin to reveal the dlls loaded in the VM. The JIT reports current machine code methods and PICs through a primitive.
Looks like everybody has a slightly different techniques and its own tools :-) I collect data in VM but the VM reports them to "outside" (via log file or kind of IPC). Then I analyze data post-mortem using various tools, some of them are hand-crafted, some not. I found KCachegrind quite nice.
Anyway, the interesting question to me is what are the advantages of your approach?
One is interactivity. The tool supports drilling down to instruction level, which is good for critical code sequences such as method prolog. Another is comprehensibility. Because it can distinguish between generated machine code (specific methods and runtime routines), functions in the VM and other support code (including stuff like Linux on stack system call entry) one sees all activities in the VM presented meaningfully. Another is evolvability. Because it's written in smalltalk I can develop the tool quickly and do so while I'm using it.
Best, Jan
Eliot (phone)
Hi Eliot,
Anyway, the interesting question to me is what are the advantages of your approach?
One is interactivity. The tool supports drilling down to instruction level, which is good for critical code sequences such as method prolog.
Another is comprehensibility. Because it can distinguish between generated machine code (specific methods and runtime routines), functions in the VM and other support code (including stuff like Linux on stack system call entry) one sees all activities in the VM presented meaningfully.
That's of course important, I agree. But if one uses out-of-image tools, this could be seen too, couldn't it? If one uses tools like OProfile or cachegrind, all this data are collected and available for analysis.
Another is evolvability. Because it's written in smalltalk I can develop the tool quickly and do so while I'm using it.
But this is because it's written in Smalltalk rather than because it runs within the debugee/profilee, no? Jan
On Wed, Mar 12, 2014 at 3:59 AM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:
Hi Eliot,
Anyway, the interesting question to me is what are the advantages
of your approach?
One is interactivity. The tool supports drilling down to instruction level, which is good for critical code sequences such as method prolog.
Another is comprehensibility. Because it can distinguish between generated machine code (specific methods and runtime routines), functions in the VM and other support code (including stuff like Linux on stack system call entry) one sees all activities in the VM presented meaningfully.
That's of course important, I agree. But if one uses out-of-image tools, this could be seen too, couldn't it? If one uses tools like OProfile or cachegrind, all this data are collected and available for analysis.
There's no symbol table info in the executable for generated methods, so no. In those tools the data is just somewhere in memory, not ascribable to specific functions. Of course with my profiler I had to write that support. I don't know those other tools and so don't know if they allow import of labels to analyse jitted code. Brief googling seems to indicate that neither has support for labelling jitted code. Another is evolvability. Because it's written in smalltalk I can
develop the tool quickly and do so while I'm using it.
But this is because it's written in Smalltalk rather than because it runs within the debugee/profilee, no?
That's what I said, yes ;-) -- best, Eliot
On 01.03.14 21:55, Stefan Marr wrote:
Hi:
On 01 Mar 2014, at 19:44, Alexandre Bergel <alexandre.bergel@me.com> wrote:
The VM is a formidable thing in which everything happen. Exposing to the image whatâs going on in it is really pushing the innovation.
Perhaps something that could be relevant in case someone decides to implement such an interface in Cog: Just one, pretty old example: http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#EventIndex
You can get access to many different information in terms of âeventsâ on the JVM. For building a profiler, more than sufficient. And, at least in my personal opinion also something that would be desirable for Cog, perhaps in a slightly more modern design, but with a similar flavor.
Oracle itself describes JVMTI as too intrusive for production use [1]. They built their own proprietary solution called Flight Recorder [2] [3] (Oracle marketing names) which AFAIK is based on JRockit technology. They are confident enough about it to explicitly recommend it for production use. They claim the overhead is about 2 to 3 %. And yes, it's a commercial feature for production and you have to unlock with: -XX:+UnlockCommercialFeatures It has a horrible UI but it's quite a nice tool. [1] http://www.oracle.com/technetwork/java/javaseproducts/mission-control/java-m... [2] http://download.oracle.com/technology/products/missioncontrol/updatesites/ba... [3] http://docs.oracle.com/javase/7/docs/technotes/guides/jfr/usingjfr.html Cheers Philippe
participants (6)
-
Alexandre Bergel -
Clément Bera -
Eliot Miranda -
Jan Vrany -
Philippe Marschall -
Stefan Marr