2011/2/10 Levente Uzonyi
<leves@elte.hu>
You made several mistakes:
- You didn't post the source code for #tinyBenchmarks. Do all those Smalltalks implement it? Are the implementations equal?
No, they don't implement it. Just Pharo and Squeak. I just added the same implementation to Integer in VA, VisualWorks and GNU Smalltalk ( Integer>>tinyBenchmarks, Integer>>benchmark and Integer>>benchFib), so the source code is the same.
- You didn't specify the version numbers for the VMs used. Or if you built them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced.
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) �However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
�
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns�'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
�
- You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake.
Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this.
�
- Does #tinyBenchmarks alone give fair performance comparison? Hardly.
Well, it was the benchmark I knew of. Do you know a better performance benchmark available? If so, I could retry the experiment.
�
Levente