[Pharo-project] How to measure VM changes from the image side
Hi folks. I am playing with some important parts of the VM and I would like to measure the impact of this changes. As an example, I am changing the Interpreter >> normalSend which is the code that represents the send of a message (at least that's what I understood). So, you can imagine that this will have an impact ;) So...what I would like to know if is someone has ideas of how can I measure this impact. I though some benchmarks that can be run before and after this change. In Java I know that there are some libraries or suites of benchmarks already written for this kind of purpose or similar. Is there something in Smalltalk ? Do you have some ideas for this ? It would be cool not do depend on network traffic or things like that. It would be cool also if I can do it directly from code so that it is repeatable. Maybe something that requires a lot of process, time, and involve a lot of objects...ideas? I though running all Pharo tests, or test coverage, ScriptLoader cleanUpForRelease, or cleanUpForProduction, run Lints Thanks in advance. Mariano
are you going to measure the change in a newly built VM, or in the interpreter simulator,or...? If you look on the VM list (and this list I think) there are a number of benchmarks used for comparison. #tinyBenchmarks is one. cheers, Mike 2010/4/2 Mariano Martinez Peck <marianopeck@gmail.com>:
Hi folks. I am playing with some important parts of the VM and I would like to measure the impact of this changes. As an example, I am changing the Interpreter >> normalSend which is the code that represents the send of a message (at least that's what I understood). So, you can imagine that this will have an impact ;)
So...what I would like to know if is someone has ideas of how can I measure this impact. I though some benchmarks that can be run before and after this change. In Java I know that there are some libraries or suites of benchmarks already written for this kind of purpose or similar. Is there something in Smalltalk ?
Do you have some ideas for this ? It would be cool not do depend on network traffic or things like that. It would be cool also if I can do it directly from code so that it is repeatable. Maybe something that requires a lot of process, time, and involve a lot of objects...ideas?
I though running all Pharo tests, or test coverage, ScriptLoader cleanUpForRelease, or cleanUpForProduction, run Lints
Thanks in advance.
Mariano
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Fri, Apr 2, 2010 at 10:43 AM, Michael Roberts <mike@mjr104.co.uk> wrote:
are you going to measure the change in a newly built VM, or in the interpreter simulator,or...?
Into a new newly compiled VM by myself.
If you look on the VM list (and this list I think) there are a number of benchmarks used for comparison. #tinyBenchmarks is one.
I was only aware of tinyBenchmarks. But ok, I will search more. I saw also #benchmark Thanks! Mariano
cheers, Mike
2010/4/2 Mariano Martinez Peck <marianopeck@gmail.com>:
Hi folks. I am playing with some important parts of the VM and I would like to measure the impact of this changes. As an example, I am changing the Interpreter >> normalSend which is the code that represents the send of a message (at least that's what I understood). So, you can imagine that this will have an impact ;)
So...what I would like to know if is someone has ideas of how can I measure this impact. I though some benchmarks that can be run before and after this change. In Java I know that there are some libraries or suites of benchmarks already written for this kind of purpose or similar. Is there something in Smalltalk ?
Do you have some ideas for this ? It would be cool not do depend on network traffic or things like that. It would be cool also if I can do it directly from code so that it is repeatable. Maybe something that requires a lot of process, time, and involve a lot of objects...ideas?
I though running all Pharo tests, or test coverage, ScriptLoader cleanUpForRelease, or cleanUpForProduction, run Lints
Thanks in advance.
Mariano
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Apr 2, 2010, at 10:47 AM, Mariano Martinez Peck wrote:
On Fri, Apr 2, 2010 at 10:43 AM, Michael Roberts <mike@mjr104.co.uk> wrote: are you going to measure the change in a newly built VM, or in the interpreter simulator,or...?
Into a new newly compiled VM by myself.
If you look on the VM list (and this list I think) there are a number of benchmarks used for comparison. #tinyBenchmarks is one.
I was only aware of tinyBenchmarks. But ok, I will search more. I saw also #benchmark
I once collected all existing benchmarks... http://www.squeaksource.com/SystemBenchmarks.html I am sure this needs a little fixing for Pharo, some stuff for sure is completely outdated, but it would be good to have it again in a usable shape. Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
For the Object Flow VM I also used the STones80 and the Squeak macro benchmarks that are on SqueakMap. You find a pointer and description of these in my thesis [1]. HTH, Adrian [1] http://www.iam.unibe.ch/~scg/Archive/PhD/lienhard-phd.pdf On Apr 2, 2010, at 10:53 , Marcus Denker wrote:
On Apr 2, 2010, at 10:47 AM, Mariano Martinez Peck wrote:
On Fri, Apr 2, 2010 at 10:43 AM, Michael Roberts <mike@mjr104.co.uk> wrote: are you going to measure the change in a newly built VM, or in the interpreter simulator,or...?
Into a new newly compiled VM by myself.
If you look on the VM list (and this list I think) there are a number of benchmarks used for comparison. #tinyBenchmarks is one.
I was only aware of tinyBenchmarks. But ok, I will search more. I saw also #benchmark
I once collected all existing benchmarks...
http://www.squeaksource.com/SystemBenchmarks.html
I am sure this needs a little fixing for Pharo, some stuff for sure is completely outdated, but it would be good to have it again in a usable shape.
Marcus
-- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Thanks Adrian. I have just read it and seems the SqueakMap points to the repo Marcus said...I have just try to load it but has the problems with underscore as assigment, InputSensor...etc. But I will give it a try. Thank you very much to both of you. Cheers Mariano On Fri, Apr 2, 2010 at 12:31 PM, Adrian Lienhard <adi@netstyle.ch> wrote:
For the Object Flow VM I also used the STones80 and the Squeak macro benchmarks that are on SqueakMap. You find a pointer and description of these in my thesis [1].
HTH, Adrian
On Apr 2, 2010, at 10:53 , Marcus Denker wrote:
On Apr 2, 2010, at 10:47 AM, Mariano Martinez Peck wrote:
On Fri, Apr 2, 2010 at 10:43 AM, Michael Roberts <mike@mjr104.co.uk>
wrote:
are you going to measure the change in a newly built VM, or in the interpreter simulator,or...?
Into a new newly compiled VM by myself.
If you look on the VM list (and this list I think) there are a number of benchmarks used for comparison. #tinyBenchmarks is one.
I was only aware of tinyBenchmarks. But ok, I will search more. I saw also #benchmark
I once collected all existing benchmarks...
http://www.squeaksource.com/SystemBenchmarks.html
I am sure this needs a little fixing for Pharo, some stuff for sure is completely outdated, but it would be good to have it again in a usable shape.
Marcus
-- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
participants (4)
-
Adrian Lienhard -
Marcus Denker -
Mariano Martinez Peck -
Michael Roberts