- Pharo-dev - lists.pharo.org

Re: Array sum. is very slow

by Nicolas Anquetil

Hi, don'tforget to weight in your time too. The ease to develop AND evolve a program is an important aspect that the benchmarks don't show. Nowdays, developer time count often more than processing time because you may easily spent days on a nasty bug or an unplanned evolution. have a nice day nicolas On Mon, 2022-01-10 at 14:05 -0600, Jimmie Houchin wrote: > Some experiments and discoveries. > I am running my full language test every time. It is the only way I > can compare results. It is also what fully stresses the language. > The reason I wrote the test as I did is because I wanted to know a > couple of things. Is the language sufficiently performant on basic > maths. I am not doing any high PolyMath level math. Simple things > like moving averages over portions of arrays. > The other is efficiency of array iteration and access. This why #sum > is the best test of this attribute. #sum iterates and accesses every > element of the array. It will reveal if there are any problems. > The default testÂ Julia 1m15s, Python 24.5 minutes, Pharo 2hour > 4minutes. > When I comment out the #sum and #average calls, Pharo completes the > test in 3.5 seconds. So almost all the time is spent in those two > calls. > So most of this conversation has focused on why #sum is as slow as it > is or how to improve the performance of #sum with other > implementations. > > Â > So I decided to breakdown the #sum and try some things. > Starting with the initial implementation and SequenceableCollection's > default #sumÂ time of 02:04:03 > > "This implementation does no work. Only iterates through the array. > It completed in 00:10:08" > Â sum > Â Â Â Â | sum | > Â Â Â Â sum := 1. > Â Â Â Â 1 to: self size do: [ :each | ]. > Â Â Â Â ^ sum > Â > Â > Â "This implementation does no work, but adds to iteration, accessing > the value of the array. > It completed in 00:32:32. > Quite a bit of time for simply iterating and accessing." > Â sum > Â Â Â Â | sum | > Â Â Â Â sum := 1. > Â Â Â Â 1 to: self size do: [ :each | self at: each ]. > Â Â Â Â ^ sum > Â > Â > Â "This implementation I had in my initial email as an experiment and > also several other did the same in theirs. > A naive simple implementation. > It completed in 01:00:53.Â Half the time of the original." > Â sum > Â Â Â | sum | > Â Â Â Â sum := 0. > Â Â Â Â 1 to: self size do: [ :each | > Â Â Â Â Â Â Â sum := sum + (self at: each) ]. > Â Â Â Â ^ sum > Â > Â > Â > Â "This implementation I also had in my initial email as an experiment > I had done. > It completed in 00:50:18. > It reduces the iterations and increases the accesses per iteration. > It is the fastest implementation so far." > Â sum > Â Â Â Â | sum | > Â Â Â Â sum := 0. > Â Â Â Â 1 to: ((self size quo: 10) * 10) by: 10 do: [ :i | > Â Â Â Â Â Â Â sum := sum + (self at: i) + (self at: (i + 1)) + (self at: > (i + 2)) + (self at: (i + 3)) + (self at: (i + 4)) Â Â Â Â Â Â Â Â Â Â + > (self at: (i + 5)) + (self at: (i + 6)) + (self at: (i + 7)) + (self > at: (i + 8)) + (self at: (i + 9))]. > Â > Â Â Â Â ((self size quo: 10) * 10 + 1) to: self size do: [ :i | > Â Â Â Â Â Â Â sum := sum + (self at: i)]. > Â Â Â Â Â ^ sum > Â > Summary > For whatever reason iterating and accessing on an Array is expensive. > That alone took longer than Python to complete the entire test. > Â > Â I had allowed this knowledge of how much slower Pharo was to stop me > from using Pharo. Encouraged me to explore other options. > Â > Â I have the option to use any language I want. I like Pharo. I do not > like Python at all. Julia is unexciting to me. I don't like their > anti-OO approach. > Â > Â At one point I had a fairly complete Pharo implementation, which is > where I got frustrated with backtesting taking days. > Â > Â That implementation is gone. I had not switched to Iceberg. I had a > problem with my hard drive. So I am starting over. > I am not a computer scientist, language expert, vm expert or anyone > with the skills to discover and optimize arrays. So I will end my > tilting at windmills here. > I value all the other things that Pharo brings, that I miss when I am > using Julia or Python or Crystal, etc. Those languages do not have > the vision to do what Pharo (or any Smalltalk) does. > Pharo may not optimize my app as much as x,y or z. But Pharo > optimized me. > That said, I have made the decision to go all in with Pharo. Set > aside all else. > Â In that regard I went ahead and put my money in with my decision and > joined the Pharo Association last week. > Thanks for all of your help in exploring the problem. > > Jimmie Houchin > Â

Jan. 11, 2022

Re: Array sum. is very slow

by Sven Van Caekenberghe

> On 11 Jan 2022, at 11:17, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > which would seem to be 3 times faster ! And with my changes (faster #sum, message spy removed): [ (LanguageTest newSize: 60*24*5*4 iterations: 1) run ] timeToRun. "0:00:00:26.612" 6 times faster.

Jan. 11, 2022

Re: Array sum. is very slow

by Sven Van Caekenberghe

Hi Andrei, That is a good catch, indeed, that makes all the difference and is an unfair comparison. If I take Jimmie's code and add def sum2(l): sum = 0 for i in range(0,len(l)): sum = sum + l[i] return sum def average(l): return sum2(l)/len(l) and replace the other calls of sum to sum2 in loop1 and loop2, I get the following for 1 iteration: >>> doit(1) Tue Jan 11 10:34:24 2022 Creating list createList(n), na[-1]: 0.28800000000000003 reps: 1 inside at top loop1: start: Tue Jan 11 10:34:24 2022 Loop1 time: 1.5645889163017273 nsum: 11242.949400371168 navg: 0.3903801875128878 loop2: start: Tue Jan 11 10:35:58 2022 Loop2 time: -27364895.977849767 nsum: 10816.16871440453 navg: 0.3755614136946017 finished: Tue Jan 11 10:37:33 2022 start time: 1641893664.795651 end time: 1641893853.597397 total time: 1614528959.1841362 nsum: 10816.16871440453 navg: 0.3755614136946017 The total time is calculated wrongly, but doing the calculation in Pharo: (1641893853.597397 - 1641893664.795651) seconds. "0:00:03:08.80174613" so 3 minutes. Jimmie's unmodified Pharo code give for 1 iteration: [ (LanguageTest newSize: 60*24*5*4 iterations: 1) run ] timeToRun. "0:00:01:00.438" Starting test for array size: 28800 iterations: 1 Creating array of size: 28800 timeToRun: 0:00:00:00.035 Starting loop 1 at: 2022-01-11T10:53:53.423313+01:00 1: 2022-01-11T10:53:53 innerttr: 0:00:00:30.073 averageTime: 0:00:00:30.073 Loop 1 time: nil nsum: 11242.949400371168 navg: 0.3903801875128878 Starting loop 2 at: 2022-01-11T10:54:23.497281+01:00 1: 2022-01-11T10:54:23 innerttr: 0:00:00:30.306 averageTime: 0:00:00:30.306 Loop 2 time: 0:00:00:30.306 nsum: 10816.168714404532 navg: 0.3755614136946018 End of test. TotalTime: 0:00:01:00.416 which would seem to be 3 times faster ! Benchmarking is a black art. Sven > On 11 Jan 2022, at 10:07, Andrei Chis <chisvasileandrei(a)gmail.com> wrote: > > Hi Jimmie, > > I was scanning through this thread and saw that the Python call uses > the sum function. If I remember correctly, in Python the built-in sum > function is directly implemented in C [1] (unless Python is compiled > with SLOW_SUM set to true). In that case on large arrays the function > can easily be several times faster than just iterating over the > individual objects as the Pharo code does. The benchmark seems to > compare summing numbers in C with summing numbers in Pharo. Would be > interesting to modify the Python code to use a similar loop as in > Pharo for doing the sum. > > Cheers, > Andrei > > [1] https://github.com/python/cpython/blob/135cabd328504e1648d17242b42b675cdbd0… > > On Mon, Jan 10, 2022 at 9:06 PM Jimmie Houchin <jlhouchin(a)gmail.com> wrote: >> >> Some experiments and discoveries. >> >> I am running my full language test every time. It is the only way I can compare results. It is also what fully stresses the language. >> >> The reason I wrote the test as I did is because I wanted to know a couple of things. Is the language sufficiently performant on basic maths. I am not doing any high PolyMath level math. Simple things like moving averages over portions of arrays. >> >> The other is efficiency of array iteration and access. This why #sum is the best test of this attribute. #sum iterates and accesses every element of the array. It will reveal if there are any problems. >> >> The default test Julia 1m15s, Python 24.5 minutes, Pharo 2hour 4minutes. >> >> When I comment out the #sum and #average calls, Pharo completes the test in 3.5 seconds. So almost all the time is spent in those two calls. >> >> So most of this conversation has focused on why #sum is as slow as it is or how to improve the performance of #sum with other implementations. >> >> >> >> So I decided to breakdown the #sum and try some things. >> >> Starting with the initial implementation and SequenceableCollection's default #sum time of 02:04:03 >> >> >> "This implementation does no work. Only iterates through the array. >> It completed in 00:10:08" >> sum >> | sum | >> sum := 1. >> 1 to: self size do: [ :each | ]. >> ^ sum >> >> >> "This implementation does no work, but adds to iteration, accessing the value of the array. >> It completed in 00:32:32. >> Quite a bit of time for simply iterating and accessing." >> sum >> | sum | >> sum := 1. >> 1 to: self size do: [ :each | self at: each ]. >> ^ sum >> >> >> "This implementation I had in my initial email as an experiment and also several other did the same in theirs. >> A naive simple implementation. >> It completed in 01:00:53. Half the time of the original." >> sum >> | sum | >> sum := 0. >> 1 to: self size do: [ :each | >> sum := sum + (self at: each) ]. >> ^ sum >> >> >> >> "This implementation I also had in my initial email as an experiment I had done. >> It completed in 00:50:18. >> It reduces the iterations and increases the accesses per iteration. >> It is the fastest implementation so far." >> sum >> | sum | >> sum := 0. >> 1 to: ((self size quo: 10) * 10) by: 10 do: [ :i | >> sum := sum + (self at: i) + (self at: (i + 1)) + (self at: (i + 2)) + (self at: (i + 3)) + (self at: (i + 4)) + (self at: (i + 5)) + (self at: (i + 6)) + (self at: (i + 7)) + (self at: (i + 8)) + (self at: (i + 9))]. >> >> ((self size quo: 10) * 10 + 1) to: self size do: [ :i | >> sum := sum + (self at: i)]. >> ^ sum >> >> Summary >> >> For whatever reason iterating and accessing on an Array is expensive. That alone took longer than Python to complete the entire test. >> >> I had allowed this knowledge of how much slower Pharo was to stop me from using Pharo. Encouraged me to explore other options. >> >> I have the option to use any language I want. I like Pharo. I do not like Python at all. Julia is unexciting to me. I don't like their anti-OO approach. >> >> At one point I had a fairly complete Pharo implementation, which is where I got frustrated with backtesting taking days. >> >> That implementation is gone. I had not switched to Iceberg. I had a problem with my hard drive. So I am starting over. >> >> I am not a computer scientist, language expert, vm expert or anyone with the skills to discover and optimize arrays. So I will end my tilting at windmills here. >> >> I value all the other things that Pharo brings, that I miss when I am using Julia or Python or Crystal, etc. Those languages do not have the vision to do what Pharo (or any Smalltalk) does. >> >> Pharo may not optimize my app as much as x,y or z. But Pharo optimized me. >> >> That said, I have made the decision to go all in with Pharo. Set aside all else. >> In that regard I went ahead and put my money in with my decision and joined the Pharo Association last week. >> >> Thanks for all of your help in exploring the problem. >> >> >> Jimmie Houchin

Jan. 11, 2022

Re: Array sum. is very slow

by Andrei Chis

Hi Jimmie, I was scanning through this thread and saw that the Python call uses the sum function. If I remember correctly, in Python the built-in sum function is directly implemented in C [1] (unless Python is compiled with SLOW_SUM set to true). In that case on large arrays the function can easily be several times faster than just iterating over the individual objects as the Pharo code does. The benchmark seems to compare summing numbers in C with summing numbers in Pharo. Would be interesting to modify the Python code to use a similar loop as in Pharo for doing the sum. Cheers, Andrei [1] https://github.com/python/cpython/blob/135cabd328504e1648d17242b42b675cdbd0… On Mon, Jan 10, 2022 at 9:06 PM Jimmie Houchin <jlhouchin(a)gmail.com> wrote: > > Some experiments and discoveries. > > I am running my full language test every time. It is the only way I can compare results. It is also what fully stresses the language. > > The reason I wrote the test as I did is because I wanted to know a couple of things. Is the language sufficiently performant on basic maths. I am not doing any high PolyMath level math. Simple things like moving averages over portions of arrays. > > The other is efficiency of array iteration and access. This why #sum is the best test of this attribute. #sum iterates and accesses every element of the array. It will reveal if there are any problems. > > The default test Julia 1m15s, Python 24.5 minutes, Pharo 2hour 4minutes. > > When I comment out the #sum and #average calls, Pharo completes the test in 3.5 seconds. So almost all the time is spent in those two calls. > > So most of this conversation has focused on why #sum is as slow as it is or how to improve the performance of #sum with other implementations. > > > > So I decided to breakdown the #sum and try some things. > > Starting with the initial implementation and SequenceableCollection's default #sum time of 02:04:03 > > > "This implementation does no work. Only iterates through the array. > It completed in 00:10:08" > sum > | sum | > sum := 1. > 1 to: self size do: [ :each | ]. > ^ sum > > > "This implementation does no work, but adds to iteration, accessing the value of the array. > It completed in 00:32:32. > Quite a bit of time for simply iterating and accessing." > sum > | sum | > sum := 1. > 1 to: self size do: [ :each | self at: each ]. > ^ sum > > > "This implementation I had in my initial email as an experiment and also several other did the same in theirs. > A naive simple implementation. > It completed in 01:00:53. Half the time of the original." > sum > | sum | > sum := 0. > 1 to: self size do: [ :each | > sum := sum + (self at: each) ]. > ^ sum > > > > "This implementation I also had in my initial email as an experiment I had done. > It completed in 00:50:18. > It reduces the iterations and increases the accesses per iteration. > It is the fastest implementation so far." > sum > | sum | > sum := 0. > 1 to: ((self size quo: 10) * 10) by: 10 do: [ :i | > sum := sum + (self at: i) + (self at: (i + 1)) + (self at: (i + 2)) + (self at: (i + 3)) + (self at: (i + 4)) + (self at: (i + 5)) + (self at: (i + 6)) + (self at: (i + 7)) + (self at: (i + 8)) + (self at: (i + 9))]. > > ((self size quo: 10) * 10 + 1) to: self size do: [ :i | > sum := sum + (self at: i)]. > ^ sum > > Summary > > For whatever reason iterating and accessing on an Array is expensive. That alone took longer than Python to complete the entire test. > > I had allowed this knowledge of how much slower Pharo was to stop me from using Pharo. Encouraged me to explore other options. > > I have the option to use any language I want. I like Pharo. I do not like Python at all. Julia is unexciting to me. I don't like their anti-OO approach. > > At one point I had a fairly complete Pharo implementation, which is where I got frustrated with backtesting taking days. > > That implementation is gone. I had not switched to Iceberg. I had a problem with my hard drive. So I am starting over. > > I am not a computer scientist, language expert, vm expert or anyone with the skills to discover and optimize arrays. So I will end my tilting at windmills here. > > I value all the other things that Pharo brings, that I miss when I am using Julia or Python or Crystal, etc. Those languages do not have the vision to do what Pharo (or any Smalltalk) does. > > Pharo may not optimize my app as much as x,y or z. But Pharo optimized me. > > That said, I have made the decision to go all in with Pharo. Set aside all else. > In that regard I went ahead and put my money in with my decision and joined the Pharo Association last week. > > Thanks for all of your help in exploring the problem. > > > Jimmie Houchin

Jan. 11, 2022

Re: Array sum. is very slow

by Jimmie Houchin

Some experiments and discoveries. I am running my full language test every time. It is the only way I can compare results. It is also what fully stresses the language. The reason I wrote the test as I did is because I wanted to know a couple of things. Is the language sufficiently performant on basic maths. I am not doing any high PolyMath level math. Simple things like moving averages over portions of arrays. The other is efficiency of array iteration and access. This why #sum is the best test of this attribute. #sum iterates and accesses every element of the array. It will reveal if there are any problems. The default testÂ Julia 1m15s, Python 24.5 minutes, Pharo 2hour 4minutes. When I comment out the #sum and #average calls, Pharo completes the test in 3.5 seconds. So almost all the time is spent in those two calls. So most of this conversation has focused on why #sum is as slow as it is or how to improve the performance of #sum with other implementations. So I decided to breakdown the #sum and try some things. Starting with the initial implementation and SequenceableCollection's default #sumÂ time of 02:04:03 /"This implementation does no work. Only iterates through the array.// //It completed in 00:10:08"/ sum Â Â Â | sum | Â Â sum := 1. Â Â Â 1 to: self size do: [ :each | ]. Â Â Â ^ sum /"This implementation does no work, but adds to iteration, accessing the value of the array.// //It completed in 00:32:32.// //Quite a bit of time for simply iterating and accessing."/ sum Â Â Â | sum | Â Â Â sum := 1. Â Â Â 1 to: self size do: [ :each | self at: each ]. Â Â Â ^ sum /"This implementation I had in my initial email as an experiment and also several other did the same in theirs.// //A naive simple implementation.// //It completed in 01:00:53.Â Half the time of the original."/ sum Â Â | sum | Â Â Â sum := 0. Â Â Â 1 to: self size do: [ :each | Â Â Â Â Â Â sum := sum + (self at: each) ]. Â Â Â ^ sum /"This implementation I also had in my initial email as an experiment I had done.// //It completed in 00:50:18.// //It reduces the iterations and increases the accesses per iteration.// //It is the fastest implementation so far."/ sum Â Â Â | sum | Â Â Â sum := 0. Â Â Â 1 to: ((self size quo: 10) * 10) by: 10 do: [ :i | Â Â Â Â Â Â sum := sum + (self at: i) + (self at: (i + 1)) + (self at: (i + 2)) + (self at: (i + 3)) + (self at: (i + 4)) Â + (self at: (i + 5)) + (self at: (i + 6)) + (self at: (i + 7)) + (self at: (i + 8)) + (self at: (i + 9))]. Â Â Â ((self size quo: 10) * 10 + 1) to: self size do: [ :i | Â Â Â Â Â Â sum := sum + (self at: i)]. Â Â Â ^ sum *Summary * For whatever reason iterating and accessing on an Array is expensive. That alone took longer than Python to complete the entire test. I had allowed this knowledge of how much slower Pharo was to stop me from using Pharo. Encouraged me to explore other options. I have the option to use any language I want. I like Pharo. I do not like Python at all. Julia is unexciting to me. I don't like their anti-OO approach. At one point I had a fairly complete Pharo implementation, which is where I got frustrated with backtesting taking days. That implementation is gone. I had not switched to Iceberg. I had a problem with my hard drive. So I am starting over. I am not a computer scientist, language expert, vm expert or anyone with the skills to discover and optimize arrays. So I will end my tilting at windmills here. I value all the other things that Pharo brings, that I miss when I am using Julia or Python or Crystal, etc. Those languages do not have the vision to do what Pharo (or any Smalltalk) does. Pharo may not optimize my app as much as x,y or z. But Pharo optimized me. That said, I have made the decision to go all in with Pharo. Set aside all else. In that regard I went ahead and put my money in with my decision and joined the Pharo Association last week. Thanks for all of your help in exploring the problem. Jimmie Houchin

Jan. 10, 2022

[Pharo Days'22] Call for participation

by Guillermo Polito

-------------------------------------------------------------------------------- Pharo Days March 3rd and 4th, Lille, France https://days.pharo.org/ -------------------------------------------------------------------------------- Do not miss the opportunity to meet Pharo and its community! PharoDays is a unique event where you can meet other Pharoers, discover latest advances, and share your experience. Hand-ons sessions are there to help you getting started on advance topics. Follow the "Show us your project" sessions to get immersed in Pharo lively community. This year, PharoDays is a mix of technical talks and hand-ons. Come to join business chat and enjoy friendly atmosphere and discuss with experts. Do you have a project to show? We are looking for presentations to fill up our schedule! Presentations will have 30' slots. Contact the Pharo board at <board(a)pharo.org> before January 21st with your ideas! Are you a student wanting to show your research project? We are going to organize poster sessions so you can show your work to the community! Prepare your posters, and contact us for a slot at <board(a)pharo.org>!

Jan. 10, 2022

Re: Array sum. is very slow

by Nicolas Anquetil

On Sun, 2022-01-09 at 12:05 +0100, Nicolas Anquetil wrote: > > Definitly not easy to do benchmarking > I got these strange results: > > n := 10000000. > floatArray := Array new: n. > > Time millisecondsToRun: [ floatArray doWithIndex: [:each :idx | > floatArray at: idx put: Random new ] ]. > "-> 2871" > > Time millisecondsToRun: [ floatArray doWithIndex: [:each :idx | > floatArray at: idx put: i ] ]. ooops, that was 'floatArray at: idx put: idx' (-> similar time) > "-> 86" > > Time millisecondsToRun: [1 to: n do: [:i | Random new ]]. > "-> 829" > > so > - assigning 'Random new' to 1M array elements takes 2.8 seconds. > - assigning a value to 1M array elements takes 0.08 seconds. > - computing 'Random new' 1M times takes 0.8 seconds > > > I wonder where the extra 2 seconds come from? > some optimization in the background? > > I did the 3 of them several times in different order and the results > are similar. > > nicolas > > On Fri, 2022-01-07 at 15:36 +0000, Benoit St-Jean via Pharo-dev > wrote: > > Can you come up with a simple "base case" so we can find the > > bottleneck/problem? > > > > I'm not sure about what you're trying to do. > > > > What do you get if you try this in a workspace (adjust the value of > > n > > to what you want, I tested it with 10 million items). > > > > Let's get this one step at a time! > > > > > > > > > Â floatArrayÂ nÂ rng t1 t2 t3 r1 r2 r3 | > > > > n := 10000000. > > > > rng := Random new. > > > > floatArray := Array new: n.Â > > floatArray doWithIndex: [:each :idx | floatArray at: idx put: rng > > next]. > > > > t1 := Time millisecondsToRun: [r1 := floatArray sum]. > > t2 := Time millisecondsToRun: [| total | > > Â Â Â Â Â Â Â Â > > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â tot > > al > > := 0. > > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â flo > > atA > > rray do: [:each | total := total + each ]. > > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â r2 > > := > > total]. > > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â > > t3 := Time millisecondsToRun: [r3 := floatArray inject: 0 into:Â [: > > total :each | total + each ]]. > > > > Transcript cr. > > Transcript cr; show: 'Test with ', n printString, ' elements'. > > Transcript cr;show: 'Original #sum -> Time: ', t1 printString, ' > > milliseconds, Total: ', r1 printString. > > Transcript cr;show: 'Naive #sum -> Time: ', t2 printString, ' > > milliseconds, Total: ', r2 printString.Â Â > > Transcript cr;show: 'Inject #sum -> Time: ', t3 printString, ' > > milliseconds, Total: ', r3 printString.Â Â > > > > -------------------------- > > > > Here are the results I get on Squeak 5.3 > > > > Test with 10000000 elements > > Original #sum -> Time: 143 milliseconds, Total: 4.999271889099622e6 > > Naive #sum -> Time: 115 milliseconds, Total: 4.999271889099622e6 > > Inject #sum -> Time: 102 milliseconds, Total: 4.999271889099622e6 > > > > > > > > ----------------- > > BenoÃ®t St-Jean > > Yahoo! Messenger: bstjean > > Twitter: @BenLeChialeux > > Pinterest: benoitstjean > > Instagram: Chef_Benito > > IRC: lamneth > > GitHub: bstjean > > Blogue: endormitoire.wordpress.com > > "A standpoint is an intellectual horizon of radius zero".Â (A. > > Einstein) > > > > > > Â On Thursday, January 6, 2022, 03:38:22 p.m. EST, Jimmie Houchin > > <jlhouchin(a)gmail.com> wrote: > > > > > > I have written a micro benchmark which stresses a language in areas > > which are crucial to my application. > > > > I have written this micro benchmark in Pharo, Crystal, Nim, Python, > > PicoLisp, C, C++, Java and Julia. > > > > On my i7 laptop Julia completes it in about 1 minute and 15 > > seconds, > > amazing magic they have done. > > > > Pharo takes over 2 hours. :( > > > > In my benchmarks if I comment out the sum and average of the array. > > It > > completes in 3.5 seconds. > > And when I sum the array it gives the correct results. So I can > > verify > > its validity. > > > > over the array and do calculations on each value of the array and > > update > > the array and sum and average at each value simple to stress array > > access and sum and average. > > > > 28800 is simply derived from time series one minute values for 5 > > days, > > 4 > > weeks. > > > > randarray := Array new: 28800. > > > > 1 to: randarray size do: [ :i | randarray at: i put: Number random > > ]. > > > > here." randarray sum. randarray average ]] timeToRun. > > > > randarrayttr. "0:00:00:36.135" > > > > > > I do 2 loops with 100 iterations each. > > > > randarrayttr * 200. "0:02:00:27" > > > > > > I learned early on in this adventure when dealing with compiled > > languages that if you donât do a lot, the test may not last long > > enough > > to give any times. > > > > Pharo is my preference. But this is an awful big gap in > > performance. > > When doing backtesting this is huge. Does my backtest take minutes, > > hours or days? > > > > I am not a computer scientist nor expert in Pharo or Smalltalk. So > > I do > > not know if there is anything which can improve this. > > > > > > However I have played around with several experiments of my #sum: > > method. > > > > This implementation reduces the time on the above randarray in > > half. > > > > sum: col > > > sum | > > sum := 0. > > 1 to: col size do: [ :i | > > Â Â Â Â Â sum := sum + (col at: i) ]. > > ^ sum > > > > randarrayttr2 := [ 1 to: randarray size do: [ :i | "other > > calculations > > here." > > Â Â Â Â ltsa sum: randarray. ltsa sum: randarray ]] timeToRun. > > randarrayttr2. "0:00:00:18.563" > > > > And this one reduces it a little more. > > > > sum10: col > > > sum | > > sum := 0. > > 1 to: ((col size quo: 10) * 10) by: 10 do: [ :i | > > Â Â Â Â Â sum := sum + (col at: i) + (col at: (i + 1)) + (col at: (i + > > 2)) > > + > > (col at: (i + 3)) + (col at: (i + 4)) > > Â Â Â Â Â Â Â Â Â + (col at: (i + 5)) + (col at: (i + 6)) + (col at: (i + > > 7)) + > > (col at: (i + 8)) + (col at: (i + 9))]. > > ((col size quo: 10) * 10 + 1) to: col size do: [ :i | > > Â Â Â Â Â sum := sum + (col at: i)]. > > ^ sum > > > > randarrayttr3 := [ 1 to: randarray size do: [ :i | "other > > calculations > > here." > > Â Â Â Â ltsa sum10: randarray. ltsa sum10: randarray ]] timeToRun. > > randarrayttr3. "0:00:00:14.592" > > > > It closes the gap with plain Python3 no numpy. But that is a pretty > > low > > standard. > > > > Any ideas, thoughts, wisdom, directions to pursue. > > > > Thanks > > > > Jimmie > > >

Jan. 9, 2022

Re: Array sum. is very slow

by Nicolas Anquetil

Definitly not easy to do benchmarking I got these strange results: n := 10000000. floatArray := Array new: n. Time millisecondsToRun: [ floatArray doWithIndex: [:each :idx | floatArray at: idx put: Random new ] ]. "-> 2871" Time millisecondsToRun: [ floatArray doWithIndex: [:each :idx | floatArray at: idx put: i ] ]. "-> 86" Time millisecondsToRun: [1 to: n do: [:i | Random new ]]. "-> 829" so - assigning 'Random new' to 1M array elements takes 2.8 seconds. - assigning a value to 1M array elements takes 0.08 seconds. - computing 'Random new' 1M times takes 0.8 seconds I wonder where the extra 2 seconds come from? some optimization in the background? I did the 3 of them several times in different order and the results are similar. nicolas On Fri, 2022-01-07 at 15:36 +0000, Benoit St-Jean via Pharo-dev wrote: > Can you come up with a simple "base case" so we can find the > bottleneck/problem? > > I'm not sure about what you're trying to do. > > What do you get if you try this in a workspace (adjust the value of n > to what you want, I tested it with 10 million items). > > Let's get this one step at a time! > > > > |Â floatArrayÂ nÂ rng t1 t2 t3 r1 r2 r3 | > > n := 10000000. > > rng := Random new. > > floatArray := Array new: n.Â > floatArray doWithIndex: [:each :idx | floatArray at: idx put: rng > next]. > > t1 := Time millisecondsToRun: [r1 := floatArray sum]. > t2 := Time millisecondsToRun: [| total | > > total > := 0. > floatA > rray do: [:each | total := total + each ]. > r2 := > total]. > > t3 := Time millisecondsToRun: [r3 := floatArray inject: 0 into:Â [: > total :each | total + each ]]. > > Transcript cr. > Transcript cr; show: 'Test with ', n printString, ' elements'. > Transcript cr;show: 'Original #sum -> Time: ', t1 printString, ' > milliseconds, Total: ', r1 printString. > Transcript cr;show: 'Naive #sum -> Time: ', t2 printString, ' > milliseconds, Total: ', r2 printString.Â Â > Transcript cr;show: 'Inject #sum -> Time: ', t3 printString, ' > milliseconds, Total: ', r3 printString.Â Â > > -------------------------- > > Here are the results I get on Squeak 5.3 > > Test with 10000000 elements > Original #sum -> Time: 143 milliseconds, Total: 4.999271889099622e6 > Naive #sum -> Time: 115 milliseconds, Total: 4.999271889099622e6 > Inject #sum -> Time: 102 milliseconds, Total: 4.999271889099622e6 > > > > ----------------- > BenoÃ®t St-Jean > Yahoo! Messenger: bstjean > Twitter: @BenLeChialeux > Pinterest: benoitstjean > Instagram: Chef_Benito > IRC: lamneth > GitHub: bstjean > Blogue: endormitoire.wordpress.com > "A standpoint is an intellectual horizon of radius zero".Â (A. > Einstein) > > > Â On Thursday, January 6, 2022, 03:38:22 p.m. EST, Jimmie Houchin > <jlhouchin(a)gmail.com> wrote: > > > I have written a micro benchmark which stresses a language in areas > which are crucial to my application. > > I have written this micro benchmark in Pharo, Crystal, Nim, Python, > PicoLisp, C, C++, Java and Julia. > > On my i7 laptop Julia completes it in about 1 minute and 15 seconds, > amazing magic they have done. > > Pharo takes over 2 hours. :( > > In my benchmarks if I comment out the sum and average of the array. It > completes in 3.5 seconds. > And when I sum the array it gives the correct results. So I can verify > its validity. > > over the array and do calculations on each value of the array and > update > the array and sum and average at each value simple to stress array > access and sum and average. > > 28800 is simply derived from time series one minute values for 5 days, > 4 > weeks. > > randarray := Array new: 28800. > > 1 to: randarray size do: [ :i | randarray at: i put: Number random ]. > > here." randarray sum. randarray average ]] timeToRun. > > randarrayttr. "0:00:00:36.135" > > > I do 2 loops with 100 iterations each. > > randarrayttr * 200. "0:02:00:27" > > > I learned early on in this adventure when dealing with compiled > languages that if you donât do a lot, the test may not last long enough > to give any times. > > Pharo is my preference. But this is an awful big gap in performance. > When doing backtesting this is huge. Does my backtest take minutes, > hours or days? > > I am not a computer scientist nor expert in Pharo or Smalltalk. So I do > not know if there is anything which can improve this. > > > However I have played around with several experiments of my #sum: > method. > > This implementation reduces the time on the above randarray in half. > > sum: col > | sum | > sum := 0. > 1 to: col size do: [ :i | > Â Â Â Â Â sum := sum + (col at: i) ]. > ^ sum > > randarrayttr2 := [ 1 to: randarray size do: [ :i | "other calculations > here." > Â Â Â Â ltsa sum: randarray. ltsa sum: randarray ]] timeToRun. > randarrayttr2. "0:00:00:18.563" > > And this one reduces it a little more. > > sum10: col > | sum | > sum := 0. > 1 to: ((col size quo: 10) * 10) by: 10 do: [ :i | > Â Â Â Â Â sum := sum + (col at: i) + (col at: (i + 1)) + (col at: (i + 2)) > + > (col at: (i + 3)) + (col at: (i + 4)) > Â Â Â Â Â Â Â Â Â + (col at: (i + 5)) + (col at: (i + 6)) + (col at: (i + 7)) + > (col at: (i + 8)) + (col at: (i + 9))]. > ((col size quo: 10) * 10 + 1) to: col size do: [ :i | > Â Â Â Â Â sum := sum + (col at: i)]. > ^ sum > > randarrayttr3 := [ 1 to: randarray size do: [ :i | "other calculations > here." > Â Â Â Â ltsa sum10: randarray. ltsa sum10: randarray ]] timeToRun. > randarrayttr3. "0:00:00:14.592" > > It closes the gap with plain Python3 no numpy. But that is a pretty low > standard. > > Any ideas, thoughts, wisdom, directions to pursue. > > Thanks > > Jimmie >

Jan. 9, 2022

Re: [lse-consortium-eng] Array sum. is very slow

by Guillermo Polito

Yet, be careful, that way of benchmarking will have a lot of variation and noise. Remember there is an OS, other apps open, even the CPU getting hot/cold can introduce performance differences... At least, that snippet should be run so many times (I do 100 iterations in general), and the averages should be compared taking into account standard deviation. El dom., 9 ene. 2022 10:14, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> escribiÃ³: > On my machine so this is the same. > > SQ5.3 > > Test with 10000000 elements > Original #sum -> Time: 196 milliseconds, Total: 5.001448710680429e6 > Naive #sum -> Time: 152 milliseconds, Total: 5.001448710680429e6 > Inject #sum -> Time: 143 milliseconds, Total: 5.001448710680429e6 > > > > On 8 Jan 2022, at 21:47, stephane ducasse <stephane.ducasse(a)inria.fr> > wrote: > > Thanks benoit for the snippet > I run it in Pharo 10 and I got > > Test with 10000000 elements > Original #sum -> Time: 195 milliseconds, Total: 4.999452880735064e6 > Naive #sum -> Time: 153 milliseconds, Total: 4.999452880735063e6 > Inject #sum -> Time: 198 milliseconds, Total: 4.999452880735063e6 > > > in Pharo 9 > Test with 10000000 elements > Original #sum -> Time: 182 milliseconds, Total: 4.999339450212771e6 > Naive #sum -> Time: 148 milliseconds, Total: 4.999339450212771e6 > Inject #sum -> Time: 203 milliseconds, Total: 4.999339450212771e6 > > Iâm interested to understand why Pharo is slower. May be this is the > impact > of the new full blocks. > We started to play with the idea of regression benchmarks. > > S > > > On 7 Jan 2022, at 16:36, Benoit St-Jean via Pharo-dev < > pharo-dev(a)lists.pharo.org> wrote: > > Can you come up with a simple "base case" so we can find the > bottleneck/problem? > > I'm not sure about what you're trying to do. > > What do you get if you try this in a workspace (adjust the value of n to > what you want, I tested it with 10 million items). > > Let's get this one step at a time! > > > > | floatArray n rng t1 t2 t3 r1 r2 r3 | > > n := 10000000. > > rng := Random new. > > floatArray := Array new: n. > floatArray doWithIndex: [:each :idx | floatArray at: idx put: rng next]. > > t1 := Time millisecondsToRun: [r1 := floatArray sum]. > t2 := Time millisecondsToRun: [| total | > total := 0. > floatArray do: [:each | total := total + each ]. > r2 := total]. > t3 := Time millisecondsToRun: [r3 := floatArray inject: 0 into: [: total > :each | total + each ]]. > > Transcript cr. > Transcript cr; show: 'Test with ', n printString, ' elements'. > Transcript cr;show: 'Original #sum -> Time: ', t1 printString, ' > milliseconds, Total: ', r1 printString. > Transcript cr;show: 'Naive #sum -> Time: ', t2 printString, ' > milliseconds, Total: ', r2 printString. > Transcript cr;show: 'Inject #sum -> Time: ', t3 printString, ' > milliseconds, Total: ', r3 printString. > > -------------------------- > > Here are the results I get on Squeak 5.3 > > Test with 10000000 elements > Original #sum -> Time: 143 milliseconds, Total: 4.999271889099622e6 > Naive #sum -> Time: 115 milliseconds, Total: 4.999271889099622e6 > Inject #sum -> Time: 102 milliseconds, Total: 4.999271889099622e6 > > > > ----------------- > BenoÃ®t St-Jean > Yahoo! Messenger: bstjean > Twitter: @BenLeChialeux > Pinterest: benoitstjean > Instagram: Chef_Benito > IRC: lamneth > GitHub: bstjean > Blogue: endormitoire.wordpress.com > "A standpoint is an intellectual horizon of radius zero". (A. Einstein) > > > On Thursday, January 6, 2022, 03:38:22 p.m. EST, Jimmie Houchin < > jlhouchin(a)gmail.com> wrote: > > > I have written a micro benchmark which stresses a language in areas > which are crucial to my application. > > I have written this micro benchmark in Pharo, Crystal, Nim, Python, > PicoLisp, C, C++, Java and Julia. > > On my i7 laptop Julia completes it in about 1 minute and 15 seconds, > amazing magic they have done. > > Crystal and Nim do it in about 5 minutes. Python in about 25 minutes. > Pharo takes over 2 hours. :( > > In my benchmarks if I comment out the sum and average of the array. It > completes in 3.5 seconds. > And when I sum the array it gives the correct results. So I can verify > its validity. > > To illustrate below is some sample code of what I am doing. I iterate > over the array and do calculations on each value of the array and update > the array and sum and average at each value simple to stress array > access and sum and average. > > 28800 is simply derived from time series one minute values for 5 days, 4 > weeks. > > randarray := Array new: 28800. > > 1 to: randarray size do: [ :i | randarray at: i put: Number random ]. > > randarrayttr := [ 1 to: randarray size do: [ :i | "other calculations > here." randarray sum. randarray average ]] timeToRun. > > randarrayttr. "0:00:00:36.135" > > > I do 2 loops with 100 iterations each. > > randarrayttr * 200. "0:02:00:27" > > > I learned early on in this adventure when dealing with compiled > languages that if you donât do a lot, the test may not last long enough > to give any times. > > Pharo is my preference. But this is an awful big gap in performance. > When doing backtesting this is huge. Does my backtest take minutes, > hours or days? > > I am not a computer scientist nor expert in Pharo or Smalltalk. So I do > not know if there is anything which can improve this. > > > However I have played around with several experiments of my #sum: method. > > This implementation reduces the time on the above randarray in half. > > sum: col > | sum | > sum := 0. > 1 to: col size do: [ :i | > sum := sum + (col at: i) ]. > ^ sum > > randarrayttr2 := [ 1 to: randarray size do: [ :i | "other calculations > here." > ltsa sum: randarray. ltsa sum: randarray ]] timeToRun. > randarrayttr2. "0:00:00:18.563" > > And this one reduces it a little more. > > sum10: col > | sum | > sum := 0. > 1 to: ((col size quo: 10) * 10) by: 10 do: [ :i | > sum := sum + (col at: i) + (col at: (i + 1)) + (col at: (i + 2)) + > (col at: (i + 3)) + (col at: (i + 4)) > + (col at: (i + 5)) + (col at: (i + 6)) + (col at: (i + 7)) + > (col at: (i + 8)) + (col at: (i + 9))]. > ((col size quo: 10) * 10 + 1) to: col size do: [ :i | > sum := sum + (col at: i)]. > ^ sum > > randarrayttr3 := [ 1 to: randarray size do: [ :i | "other calculations > here." > ltsa sum10: randarray. ltsa sum10: randarray ]] timeToRun. > randarrayttr3. "0:00:00:14.592" > > It closes the gap with plain Python3 no numpy. But that is a pretty low > standard. > > Any ideas, thoughts, wisdom, directions to pursue. > > Thanks > > Jimmie > > > >

Jan. 9, 2022

Re: Array sum. is very slow

by StÃ©phane Ducasse

On my machine so this is the same. SQ5.3 Test with 10000000 elements Original #sum -> Time: 196 milliseconds, Total: 5.001448710680429e6 Naive #sum -> Time: 152 milliseconds, Total: 5.001448710680429e6 Inject #sum -> Time: 143 milliseconds, Total: 5.001448710680429e6 > On 8 Jan 2022, at 21:47, stephane ducasse <stephane.ducasse(a)inria.fr> wrote: > > Thanks benoit for the snippet > I run it in Pharo 10 and I got > > Test with 10000000 elements > Original #sum -> Time: 195 milliseconds, Total: 4.999452880735064e6 > Naive #sum -> Time: 153 milliseconds, Total: 4.999452880735063e6 > Inject #sum -> Time: 198 milliseconds, Total: 4.999452880735063e6 > > > in Pharo 9 > Test with 10000000 elements > Original #sum -> Time: 182 milliseconds, Total: 4.999339450212771e6 > Naive #sum -> Time: 148 milliseconds, Total: 4.999339450212771e6 > Inject #sum -> Time: 203 milliseconds, Total: 4.999339450212771e6 > > Iâm interested to understand why Pharo is slower. May be this is the impact > of the new full blocks. > We started to play with the idea of regression benchmarks. > > S > > >> On 7 Jan 2022, at 16:36, Benoit St-Jean via Pharo-dev <pharo-dev(a)lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> wrote: >> >> Can you come up with a simple "base case" so we can find the bottleneck/problem? >> >> I'm not sure about what you're trying to do. >> >> What do you get if you try this in a workspace (adjust the value of n to what you want, I tested it with 10 million items). >> >> Let's get this one step at a time! >> >> >> >> | floatArray n rng t1 t2 t3 r1 r2 r3 | >> >> n := 10000000. >> >> rng := Random new. >> >> floatArray := Array new: n. >> floatArray doWithIndex: [:each :idx | floatArray at: idx put: rng next]. >> >> t1 := Time millisecondsToRun: [r1 := floatArray sum]. >> t2 := Time millisecondsToRun: [| total | >> >> total := 0. >> floatArray do: [:each | total := total + each ]. >> r2 := total]. >> >> t3 := Time millisecondsToRun: [r3 := floatArray inject: 0 into: [: total :each | total + each ]]. >> >> Transcript cr. >> Transcript cr; show: 'Test with ', n printString, ' elements'. >> Transcript cr;show: 'Original #sum -> Time: ', t1 printString, ' milliseconds, Total: ', r1 printString. >> Transcript cr;show: 'Naive #sum -> Time: ', t2 printString, ' milliseconds, Total: ', r2 printString. >> Transcript cr;show: 'Inject #sum -> Time: ', t3 printString, ' milliseconds, Total: ', r3 printString. >> >> -------------------------- >> >> Here are the results I get on Squeak 5.3 >> >> Test with 10000000 elements >> Original #sum -> Time: 143 milliseconds, Total: 4.999271889099622e6 >> Naive #sum -> Time: 115 milliseconds, Total: 4.999271889099622e6 >> Inject #sum -> Time: 102 milliseconds, Total: 4.999271889099622e6 >> >> >> >> ----------------- >> BenoÃ®t St-Jean >> Yahoo! Messenger: bstjean >> Twitter: @BenLeChialeux >> Pinterest: benoitstjean >> Instagram: Chef_Benito >> IRC: lamneth >> GitHub: bstjean >> Blogue: endormitoire.wordpress.com <http://endormitoire.wordpress.com/> >> "A standpoint is an intellectual horizon of radius zero". (A. Einstein) >> >> >> On Thursday, January 6, 2022, 03:38:22 p.m. EST, Jimmie Houchin <jlhouchin(a)gmail.com <mailto:jlhouchin@gmail.com>> wrote: >> >> >> I have written a micro benchmark which stresses a language in areas >> which are crucial to my application. >> >> I have written this micro benchmark in Pharo, Crystal, Nim, Python, >> PicoLisp, C, C++, Java and Julia. >> >> On my i7 laptop Julia completes it in about 1 minute and 15 seconds, >> amazing magic they have done. >> >> Crystal and Nim do it in about 5 minutes. Python in about 25 minutes. >> Pharo takes over 2 hours. :( >> >> In my benchmarks if I comment out the sum and average of the array. It >> completes in 3.5 seconds. >> And when I sum the array it gives the correct results. So I can verify >> its validity. >> >> To illustrate below is some sample code of what I am doing. I iterate >> over the array and do calculations on each value of the array and update >> the array and sum and average at each value simple to stress array >> access and sum and average. >> >> 28800 is simply derived from time series one minute values for 5 days, 4 >> weeks. >> >> randarray := Array new: 28800. >> >> 1 to: randarray size do: [ :i | randarray at: i put: Number random ]. >> >> randarrayttr := [ 1 to: randarray size do: [ :i | "other calculations >> here." randarray sum. randarray average ]] timeToRun. >> >> randarrayttr. "0:00:00:36.135" >> >> >> I do 2 loops with 100 iterations each. >> >> randarrayttr * 200. "0:02:00:27" >> >> >> I learned early on in this adventure when dealing with compiled >> languages that if you donât do a lot, the test may not last long enough >> to give any times. >> >> Pharo is my preference. But this is an awful big gap in performance. >> When doing backtesting this is huge. Does my backtest take minutes, >> hours or days? >> >> I am not a computer scientist nor expert in Pharo or Smalltalk. So I do >> not know if there is anything which can improve this. >> >> >> However I have played around with several experiments of my #sum: method. >> >> This implementation reduces the time on the above randarray in half. >> >> sum: col >> | sum | >> sum := 0. >> 1 to: col size do: [ :i | >> sum := sum + (col at: i) ]. >> ^ sum >> >> randarrayttr2 := [ 1 to: randarray size do: [ :i | "other calculations >> here." >> ltsa sum: randarray. ltsa sum: randarray ]] timeToRun. >> randarrayttr2. "0:00:00:18.563" >> >> And this one reduces it a little more. >> >> sum10: col >> | sum | >> sum := 0. >> 1 to: ((col size quo: 10) * 10) by: 10 do: [ :i | >> sum := sum + (col at: i) + (col at: (i + 1)) + (col at: (i + 2)) + >> (col at: (i + 3)) + (col at: (i + 4)) >> + (col at: (i + 5)) + (col at: (i + 6)) + (col at: (i + 7)) + >> (col at: (i + 8)) + (col at: (i + 9))]. >> ((col size quo: 10) * 10 + 1) to: col size do: [ :i | >> sum := sum + (col at: i)]. >> ^ sum >> >> randarrayttr3 := [ 1 to: randarray size do: [ :i | "other calculations >> here." >> ltsa sum10: randarray. ltsa sum10: randarray ]] timeToRun. >> randarrayttr3. "0:00:00:14.592" >> >> It closes the gap with plain Python3 no numpy. But that is a pretty low >> standard. >> >> Any ideas, thoughts, wisdom, directions to pursue. >> >> Thanks >> >> Jimmie >> >

Jan. 9, 2022