Pharo-dev

By thread

pharo-dev@lists.pharo.org

January 2022

52 messages

Re: Array sum. is very slow

by Guillermo Polito

Hi Jummie, Is it possible that your program is computing a lot of **very** large integers? Iâm just trying the following with small numbers, and I donât see the issue. #sum executes on a 28k large collection around 20 million times per second on my old 2015 i5. a := (1 to: 28000). [a sum] bench "'20256552.490 per secondâ" If you could share with us more data, we could take a look. Now iâm curious. Thanks, G > El 6 ene 2022, a las 21:37, Jimmie Houchin <jlhouchin(a)gmail.com> escribiÃ³: > > I have written a micro benchmark which stresses a language in areas which are crucial to my application. > > I have written this micro benchmark in Pharo, Crystal, Nim, Python, PicoLisp, C, C++, Java and Julia. > > On my i7 laptop Julia completes it in about 1 minute and 15 seconds, amazing magic they have done. > > Crystal and Nim do it in about 5 minutes. Python in about 25 minutes. Pharo takes over 2 hours. :( > > In my benchmarks if I comment out the sum and average of the array. It completes in 3.5 seconds. > And when I sum the array it gives the correct results. So I can verify its validity. > > To illustrate below is some sample code of what I am doing. I iterate over the array and do calculations on each value of the array and update the array and sum and average at each value simple to stress array access and sum and average. > > 28800 is simply derived from time series one minute values for 5 days, 4 weeks. > > randarray := Array new: 28800. > > 1 to: randarray size do: [ :i | randarray at: i put: Number random ]. > > randarrayttr := [ 1 to: randarray size do: [ :i | "other calculations here." randarray sum. randarray average ]] timeToRun. > > randarrayttr. "0:00:00:36.135" > > > I do 2 loops with 100 iterations each. > > randarrayttr * 200. "0:02:00:27" > > > I learned early on in this adventure when dealing with compiled languages that if you donât do a lot, the test may not last long enough to give any times. > > Pharo is my preference. But this is an awful big gap in performance. When doing backtesting this is huge. Does my backtest take minutes, hours or days? > > I am not a computer scientist nor expert in Pharo or Smalltalk. So I do not know if there is anything which can improve this. > > > However I have played around with several experiments of my #sum: method. > > This implementation reduces the time on the above randarray in half. > > sum: col > | sum | > sum := 0. > 1 to: col size do: [ :i | > sum := sum + (col at: i) ]. > ^ sum > > randarrayttr2 := [ 1 to: randarray size do: [ :i | "other calculations here." > ltsa sum: randarray. ltsa sum: randarray ]] timeToRun. > randarrayttr2. "0:00:00:18.563" > > And this one reduces it a little more. > > sum10: col > | sum | > sum := 0. > 1 to: ((col size quo: 10) * 10) by: 10 do: [ :i | > sum := sum + (col at: i) + (col at: (i + 1)) + (col at: (i + 2)) + (col at: (i + 3)) + (col at: (i + 4)) > + (col at: (i + 5)) + (col at: (i + 6)) + (col at: (i + 7)) + (col at: (i + 8)) + (col at: (i + 9))]. > ((col size quo: 10) * 10 + 1) to: col size do: [ :i | > sum := sum + (col at: i)]. > ^ sum > > randarrayttr3 := [ 1 to: randarray size do: [ :i | "other calculations here." > ltsa sum10: randarray. ltsa sum10: randarray ]] timeToRun. > randarrayttr3. "0:00:00:14.592" > > It closes the gap with plain Python3 no numpy. But that is a pretty low standard. > > Any ideas, thoughts, wisdom, directions to pursue. > > Thanks > > Jimmie >

Jan. 6, 2022

Array sum. is very slow

by Jimmie Houchin

I have written a micro benchmark which stresses a language in areas which are crucial to my application. I have written this micro benchmark in Pharo, Crystal, Nim, Python, PicoLisp, C, C++, Java and Julia. On my i7 laptop Julia completes it in about 1 minute and 15 seconds, amazing magic they have done. Crystal and Nim do it in about 5 minutes. Python in about 25 minutes. Pharo takes over 2 hours. :( In my benchmarks if I comment out the sum and average of the array. It completes in 3.5 seconds. And when I sum the array it gives the correct results. So I can verify its validity. To illustrate below is some sample code of what I am doing. I iterate over the array and do calculations on each value of the array and update the array and sum and average at each value simple to stress array access and sum and average. 28800 is simply derived from time series one minute values for 5 days, 4 weeks. randarray := Array new: 28800. 1 to: randarray size do: [ :i | randarray at: i put: Number random ]. randarrayttr := [ 1 to: randarray size do: [ :i | "other calculations here." randarray sum. randarray average ]] timeToRun. randarrayttr. "0:00:00:36.135" I do 2 loops with 100 iterations each. randarrayttr * 200. "0:02:00:27" I learned early on in this adventure when dealing with compiled languages that if you donât do a lot, the test may not last long enough to give any times. Pharo is my preference. But this is an awful big gap in performance. When doing backtesting this is huge. Does my backtest take minutes, hours or days? I am not a computer scientist nor expert in Pharo or Smalltalk. So I do not know if there is anything which can improve this. However I have played around with several experiments of my #sum: method. This implementation reduces the time on the above randarray in half. sum: col | sum | sum := 0. 1 to: col size do: [ :i | Â Â Â Â sum := sum + (col at: i) ]. ^ sum randarrayttr2 := [ 1 to: randarray size do: [ :i | "other calculations here." Â Â Â ltsa sum: randarray. ltsa sum: randarray ]] timeToRun. randarrayttr2. "0:00:00:18.563" And this one reduces it a little more. sum10: col | sum | sum := 0. 1 to: ((col size quo: 10) * 10) by: 10 do: [ :i | Â Â Â Â sum := sum + (col at: i) + (col at: (i + 1)) + (col at: (i + 2)) + (col at: (i + 3)) + (col at: (i + 4)) Â Â Â Â Â Â Â Â + (col at: (i + 5)) + (col at: (i + 6)) + (col at: (i + 7)) + (col at: (i + 8)) + (col at: (i + 9))]. ((col size quo: 10) * 10 + 1) to: col size do: [ :i | Â Â Â Â sum := sum + (col at: i)]. ^ sum randarrayttr3 := [ 1 to: randarray size do: [ :i | "other calculations here." Â Â Â ltsa sum10: randarray. ltsa sum10: randarray ]] timeToRun. randarrayttr3. "0:00:00:14.592" It closes the gap with plain Python3 no numpy. But that is a pretty low standard. Any ideas, thoughts, wisdom, directions to pursue. Thanks Jimmie

Jan. 6, 2022