Hi: On 14 Mar 2011, at 09:36, Mariano Martinez Peck wrote:
Hi Stef. I would love this. Not only to see how successful an optimization was, but also, how much overhead your solution has ;) Just in case you are talking to me, it is Stefan, not Stef...
Yes, please. Did you consider the repository PharoBenchmarks (even if the package can be called SBench)? because it would be good to collect them somewhere, and that seems a good enough place. No, PharoBenchmarks is not 'good enough'. I am not going to restrict my work to a particular Smalltalk, more than necessary. And to make that apparent I will insist on dropping the 'P' form the name. The benchmark framework should not be restricted to Pharo in any way. Benchmarks perhaps, but not the execution framework.
BTW, how do you compare the benchmarks from PBenchmark to the ones that are in PharoBenchmarks ? PharoBenchmarks contains mostly microbenchmarks of all different kind. Should be relatively easy to adapt them to be used with the new framework. PBenchmark contains a small set of microbenchmarks, too. A bit better organized thought.
The Whetstone benchmarks are already included in my RoarVM benchmarks and will definitely ported.
Finally, I would also love to have such benchmarks run in each hudson build to see wether each commit to pharo make it slower/faster.
We already do that for the RoarVM, and the whole point is to enable you to do it too, within a common framework. What I want, is to be able to compare the results of the various VMs(and/or images) with each other. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525