Hi,

i'm having trouble doing benchmarks in Pharo, at least i don't understand the results (maybe i'm missing something). I want to compare anonymous subclasses lookup speed with standard pharo classes lookup speed.

I have a method m and a method m2 that execute the following code:     ^ 100 printString. I have implemented these 2 methods in one class then in one anonymous subclass. I thought that measuring the execution speed of the same code in these two different cases would give me different times if the lookup speed was not the same.

So I did the following:

|o results repetitions |

o := theClass new.

results := OrderedCollection new.

repetitions := 100.

repetitions timesRepeat:[ results add: [1000000 timesRepeat:[o m]] timeToRun].

I do the same for m2, and then i compute an average of all the values measured in results.

What i don't understand is that, for example, for an average on 100 measurements, m is 1% slower and m2 is 2% faster in the Pharo class case than with anonymous subclasses. But for 1 000 measurements, m is 11% faster but m2 is 3% slower. Results continue to vary as i change the number of measurements, but they do not increase with it (seems not to be linear).

I don't have enough knowledge about how to benchmark code, or what will make a difference in Pharo. For now the only explanations i have is that maybe the results are too slow to be significant and then they can vary, or i have done something wrong in how i measure it.

How would you measure the lookup speed of a method ?

Steven.