Re: [Pharo-project] Remove #class as a Compiler optimization
On Mar 25, 2011, at 5:46 PM, Camillo Bruni wrote:
Arguing with absolute values is a bit dangerous:
But it makes 13ms / 17ms * 100% = 76% => 25% speed improvement! in COG 50ms / 60ms * 100% = 83% => 17% faster
So the impact is still quite big. But indeed I would be in favor of a normal message send...
And are you sure that your Doit is actually JITed in Cog? 1) always take the overhead of the loop into acount. Especially if it is a large percentage, it can make you "slowdown" seem very small even though it is huge. e.g. the overhead of the loop in your case should be *very* high. 90% of the execution time? 2) always make sure that you actually JIT. -> put it in a normal method in a class -> execute it at least 2 times Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
On Mar 25, 2011, at 5:51 PM, Marcus Denker wrote:
On Mar 25, 2011, at 5:46 PM, Camillo Bruni wrote:
Arguing with absolute values is a bit dangerous:
But it makes 13ms / 17ms * 100% = 76% => 25% speed improvement! in COG 50ms / 60ms * 100% = 83% => 17% faster
So for Cog, you get this counter-intuitive result is because Cog executes the overhead loop faster in combination with not jitting what you wanted to test. As the doit is *not* Jited, it executes as fast as in the intepreter case. You see a higher percentage just because the base overhead loop is executed faster and the thing you test gets a larger share of the runtime. Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
On 2011-03-25, at 18:12, Marcus Denker wrote:
On Mar 25, 2011, at 5:51 PM, Marcus Denker wrote:
On Mar 25, 2011, at 5:46 PM, Camillo Bruni wrote:
Arguing with absolute values is a bit dangerous:
But it makes 13ms / 17ms * 100% = 76% => 25% speed improvement! in COG 50ms / 60ms * 100% = 83% => 17% faster
So for Cog, you get this counter-intuitive result is because Cog executes the overhead loop faster in combination with not jitting what you wanted to test.
I didn't run the benchmarks, I just listed the results mariano provided. Just to show that they do not provide a valid argument for removal of the method. So we have a nice Benchmarking framework on sqeaksource which we should use instead of relying on some pseudo valid results. I do not have to time right now to do so... but I expect the overhead to be neglectable in COG, since it should be fairly easy to JIT this in...
As the doit is *not* Jited, it executes as fast as in the intepreter case. You see a higher percentage just because the base overhead loop is executed faster and the thing you test gets a larger share of the runtime.
Marcus
-- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
Hi: On 25 Mar 2011, at 18:15, Camillo Bruni wrote:
13ms / 17ms * 100% = 76% => 25% speed improvement! in COG 50ms / 60ms * 100% = 83% => 17% faster
So we have a nice Benchmarking framework on sqeaksource which we should use instead of relying on some pseudo valid results.
It is not "there" yet, in terms of usability. And it is not trimmed for such kind of microbenchmarks. But it would be great to establish it as a standard to avoid those common traps, like the code did not get JITed... Mariano, if you do these kind of measurements often, please have a look at [1] and [2]. And perhaps, we can extend [3] to your liking. Best regards Stefan [1] A. Georges, D. Buytaert, and L. Eeckhout, âStatistically Rigorous Java Performance Evaluation,â SIGPLAN Not., vol. 42, no. 10, p. 57-76, 2007. [2] http://code.google.com/p/caliper/ [3] http://www.squeaksource.com/SMark.html -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
participants (3)
-
Camillo Bruni -
Marcus Denker -
Stefan Marr