see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark Maybe Eliot is able to say something here on the main differences between Cog <-> VW VM. Thx T. -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone
On Thu, 10 Feb 2011, Torsten Bergmann wrote:
see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark
Maybe Eliot is able to say something here on the main differences between Cog <-> VW VM.
You made several mistakes: - You didn't post the source code for #tinyBenchmarks. Do all those Smalltalks implement it? Are the implementations equal? - You didn't specify the version numbers for the VMs used. Or if you built them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced. - There are two different Cog VMs. You didn't specify which one did you use with Pharo. - You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake. - Does #tinyBenchmarks alone give fair performance comparison? Hardly. Levente
Thx T. -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone
On Thu, 10 Feb 2011, Levente Uzonyi wrote:
On Thu, 10 Feb 2011, Torsten Bergmann wrote:
see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark
Maybe Eliot is able to say something here on the main differences between Cog <-> VW VM.
You made several mistakes:
I mean Felipe, not you Torsten. Levente
- You didn't post the source code for #tinyBenchmarks. Do all those Smalltalks implement it? Are the implementations equal? - You didn't specify the version numbers for the VMs used. Or if you built them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced. - There are two different Cog VMs. You didn't specify which one did you use with Pharo. - You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake. - Does #tinyBenchmarks alone give fair performance comparison? Hardly.
Levente
Thx T. -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone
Hi Everybody! This is my first post on the list, even though i've been subscribed for more than 6 months. 2011/2/10 Levente Uzonyi <leves@elte.hu>
On Thu, 10 Feb 2011, Torsten Bergmann wrote:
see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark
Maybe Eliot is able to say something here on the main differences between Cog <-> VW VM.
You made several mistakes: - You didn't post the source code for #tinyBenchmarks. Do all those Smalltalks implement it? Are the implementations equal?
No, they don't implement it. Just Pharo and Squeak. I just added the same implementation to Integer in VA, VisualWorks and GNU Smalltalk ( Integer>>tinyBenchmarks, Integer>>benchmark and Integer>>benchFib), so the source code is the same. - You didn't specify the version numbers for the VMs used. Or if you built
them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced.
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
- You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake.
Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this.
- Does #tinyBenchmarks alone give fair performance comparison? Hardly.
Well, it was the benchmark I knew of. Do you know a better performance benchmark available? If so, I could retry the experiment.
Levente
Thx T. -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone
-- Felipe Bañados Schwerter Estudiante de IngenierÃa Civil en Computación http://www.dcc.uchile.cl/~fbanados Universidad de Chile
2011/2/10 Levente Uzonyi <leves@elte.hu>
On Thu, 10 Feb 2011, Torsten Bergmann wrote:
see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark
Maybe Eliot is able to say something here on the main differences between Cog <-> VW VM.
You made several mistakes: - You didn't post the source code for #tinyBenchmarks. Do all those Smalltalks implement it? Are the implementations equal?
No, they don't implement it. Just Pharo and Squeak. I just added the same implementation to Integer in VA, VisualWorks and GNU Smalltalk ( Integer>>tinyBenchmarks, Integer>>benchmark and Integer>>benchFib), so the source code is the same.
- You didn't specify the version numbers for the VMs used. Or if you built
them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced.
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
- You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake.
Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this.
Squeak 4.2 is cog ready
On 10 Feb 2011, at 17:45, Felipe Bañados Schwerter wrote:
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
+1 There does indeed not seem to be a very good way to get at the proper VM version (let alone build) from inside the Smalltalk image. It frustrates me too.
- You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake.
Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this.
Again, you are right: even though Squeak 4.2 is ready for Cog, there is (not yet) a simple one-click installer using it (as far as I can tell from a casual look). So your experience mirrors what some others might find. Benchmarking is always fuzzy, you picked one that was available and has been there for years, so if it would be no good, somebody should have replaced it. Regards, Sven
On Thu, 10 Feb 2011, Sven Van Caekenberghe wrote:
On 10 Feb 2011, at 17:45, Felipe Ba?ados Schwerter wrote:
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
+1
There does indeed not seem to be a very good way to get at the proper VM version (let alone build) from inside the Smalltalk image. It frustrates me too.
These methods should do it: Smalltalk buildDate. 'Win32 built on Feb 6 2011 18:06:24 Compiler: 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)'. Smalltalk isRunningCog. true. Smalltalk isRunningCogit. true. Smalltalk interpreterClass. 'CoInterpreter VMMaker-oscog.47 uuid: a4053f20-cb87-415d-aad3-3bb29b03724c Feb 6 2011'. Smalltalk cogitClass. 'StackToRegisterMappingCogit VMMaker-oscog.47 uuid: a4053f20-cb87-415d-aad3-3bb29b03724c Feb 6 2011'.
- You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake.
Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this.
Again, you are right: even though Squeak 4.2 is ready for Cog, there is (not yet) a simple one-click installer using it (as far as I can tell from a casual look). So your experience mirrors what some others might find.
There will be when it will be released. ;)
Benchmarking is always fuzzy, you picked one that was available and has been there for years, so if it would be no good, somebody should have replaced it.
It's fine for benchmarking the VM. As the name suggests, it's not a system wide benchmark, just a tiny one. Levente
Regards,
Sven
On Thu, Feb 10, 2011 at 07:12:03PM +0100, Sven Van Caekenberghe wrote:
On 10 Feb 2011, at 17:45, Felipe Ba?ados Schwerter wrote:
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
+1
There does indeed not seem to be a very good way to get at the proper VM version (let alone build) from inside the Smalltalk image. It frustrates me too.
Check out SystemReporter on SqueakSource. It's being used in Squeak now and AFIK should load in Pharo also. Very convenient, and tells you everything you need to know about VM version and lots of other things. Dave
so much noise for so insignificant smoke test. I'd rather prefer seeing Cog doing copy-down compiled method, at least for most simple cases - self sends :) -- Best regards, Igor Stasenko AKA sig.
On 11 February 2011 02:52, Levente Uzonyi <leves@elte.hu> wrote:
On Fri, 11 Feb 2011, Igor Stasenko wrote:
so much noise for so insignificant smoke test. I'd rather prefer seeing Cog doing copy-down compiled method, at least for most simple cases - self sends :)
Do you mean inlining?
yes
Levente
-- Best regards, Igor Stasenko AKA sig.
-- Best regards, Igor Stasenko AKA sig.
it is on the radar http://code.google.com/p/pharo/issues/detail?id=3546 Stef On Feb 11, 2011, at 2:03 AM, David T. Lewis wrote:
On Thu, Feb 10, 2011 at 07:12:03PM +0100, Sven Van Caekenberghe wrote:
On 10 Feb 2011, at 17:45, Felipe Ba?ados Schwerter wrote:
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
+1
There does indeed not seem to be a very good way to get at the proper VM version (let alone build) from inside the Smalltalk image. It frustrates me too.
Check out SystemReporter on SqueakSource. It's being used in Squeak now and AFIK should load in Pharo also. Very convenient, and tells you everything you need to know about VM version and lots of other things.
Dave
2011/2/10 Felipe Bañados Schwerter <fbanados@dcc.uchile.cl>
Hi Everybody! This is my first post on the list, even though i've been subscribed for more than 6 months.
2011/2/10 Levente Uzonyi <leves@elte.hu>
On Thu, 10 Feb 2011, Torsten Bergmann wrote:
see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark
Maybe Eliot is able to say something here on the main differences between Cog <-> VW VM.
You made several mistakes: - You didn't post the source code for #tinyBenchmarks. Do all those Smalltalks implement it? Are the implementations equal?
No, they don't implement it. Just Pharo and Squeak. I just added the same implementation to Integer in VA, VisualWorks and GNU Smalltalk ( Integer>>tinyBenchmarks, Integer>>benchmark and Integer>>benchFib), so the source code is the same.
- You didn't specify the version numbers for the VMs used. Or if you built
them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced.
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
But Smalltalk getSystemAttribute: 1008 returns the JIT version, which is much more important for this performance measurement.
- You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake.
Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this.
- Does #tinyBenchmarks alone give fair performance comparison? Hardly.
Well, it was the benchmark I knew of. Do you know a better performance benchmark available? If so, I could retry the experiment.
As large a benchmark that is as close to your working case as possible. tinyBenchmarks is little more than a smoke test (which is what we use it for at work; if tinyBenchmarks hasn't plummeted ten the build is OK performance-wise). In the Cog-Benchmarks (in Cog package) that I uploaded to squeaksource recently you'll find ShootoutTests which has 4 tests from the computer language shootouts that are pure Smalltalk.
Levente
Thx T. -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone
-- Felipe Bañados Schwerter Estudiante de IngenierÃa Civil en Computación http://www.dcc.uchile.cl/~fbanados Universidad de Chile
On Thu, 10 Feb 2011, Felipe Ba?ados Schwerter wrote:
Hi Everybody! This is my first post on the list, even though i've been subscribed for more than 6 months.
2011/2/10 Levente Uzonyi <leves@elte.hu>
On Thu, 10 Feb 2011, Torsten Bergmann wrote:
see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark
Maybe Eliot is able to say something here on the main differences between Cog <-> VW VM.
You made several mistakes: - You didn't post the source code for #tinyBenchmarks. Do all those Smalltalks implement it? Are the implementations equal?
No, they don't implement it. Just Pharo and Squeak. I just added the same implementation to Integer in VA, VisualWorks and GNU Smalltalk ( Integer>>tinyBenchmarks, Integer>>benchmark and Integer>>benchFib), so the source code is the same.
Cool. I've ported #tinyBenchmarks to Clamato and also tried StrongTalk. I had to rewrite the code to make it run on both of them.
- You didn't specify the version numbers for the VMs used. Or if you built
them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced.
Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
It makes sense, just note it.
- There are two different Cog VMs. You didn't specify which one did you use with Pharo.
Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
So it's SimpleStackBasedCogit. If you use StackToRegisterMappingCogit you'll get much better results.
- You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake.
Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this.
It's a question of how much time you spend on it. You used Pharo 1.2, though you have to dig deep to download it from the Pharo website. Finding Pharo 1.1.1 is obvious from there :). Anyway it doesn't really matter which image you use for #tinyBenchmarks, because it's a VM benchmark. The image can hardly affect it's results.
- Does #tinyBenchmarks alone give fair performance comparison? Hardly.
Well, it was the benchmark I knew of. Do you know a better performance benchmark available? If so, I could retry the experiment.
See what Eliot suggested. Levente
Levente
Thx T. -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone
-- Felipe Ba?ados Schwerter Estudiante de IngenierÃa Civil en Computación http://www.dcc.uchile.cl/~fbanados Universidad de Chile
participants (9)
-
David T. Lewis -
Eliot Miranda -
Felipe Bañados Schwerter -
Igor Stasenko -
Lemuus -
Levente Uzonyi -
Stéphane Ducasse -
Sven Van Caekenberghe -
Torsten Bergmann