Hi: I have currently the following benchmarks at hand, partially taken from PharoBenchmarks and partially from the language shootout (http://shootout.alioth.debian.org/, https://alioth.debian.org/snapshots.php?group_id=30402). CompilerBench.st FloatLoopBench.st IntLoopBench.st RoarParallelTinyBench.st StartupBench.st NasParallelBenchmarksIS.st BinaryTrees.st Chameleon.st FannkuchRedux.st Fasta.st NBody.st PermGeneratorRedux.st ChameneosRedux.st RoarSlopstoneBenchmark.st RoarSmopstoneBenchmark.st This also includes a small benchmark harness that allows parallel execution, i.e., benchmark for weak scalability. It also includes the necessary details to be run in batch mode from the command-line, very important to do performance regression testing for our RoarVM. An in-image-only solution does not provide enough flexibility for us. Would be interesting to discuss these things today or tomorrow at the sprint. Best regards Stefan On 11 Mar 2011, at 15:05, Alexandre Bergel wrote:
I am not sure what you mean with vm benchmarks, but in almost all the tools I am contributing come with some benchmarks (spy, Mondrian, I wrote some benchmarks for Glamour as well). Naturally, those are macro benchmarks, which is probably what matter the most.
Cheers, Alexandre
Le 11 mars 2011 à 10:55, Mariano Martinez Peck <marianopeck@gmail.com> a écrit :
does anyone have news regarding this topic? we VERY welcome people helping with benchmarks. please feel free to improve http://www.squeaksource.com/PharoBenchmarks
On Tue, Jan 4, 2011 at 11:51 PM, Stefan Marr <pharo@stefan-marr.de> wrote: Hi:
On 04 Jan 2011, at 23:40, Mariano Martinez Peck wrote:
On Tue, Jan 4, 2011 at 11:35 PM, Stefan Marr <squeak@stefan-marr.de> wrote: Hi Igor:
On 04 Jan 2011, at 22:40, Igor Stasenko wrote:
Okay, how about creating a separate VMBenchmarks repository and putting VMBenchmarks package there?
Sure, sounds good. There are also the Systems benchmarks at http://www.squeaksource.com/PharoBenchmarks.
So, at least, I was able to run all benchmarks :)
However, it still needs cleaning, improvement, testing, blah. But it is a good start point I think.
Well, I am not to sure about the general value of those benchmarks. There are many microbenchmarks which do not tell you a lot. All those test* things. And well, their value for testing is also questionable. They only can help you to identify where it goes *boom* and crashes the VM, but they do not actually assert for anything.
Also not sure what the value of Slopstone and Smopstone (names from the top of my head might be slightly different) is nowadays.
The compiler benchmark is a reasonable application benchmark. Would be good to have a few others in that collection, too.
Best regards Stefan
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525 -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525