Pharo vs. Squeak performance
I have some code that creates a several hundred MB model. When I run the code under Pharo it takes ~2.5 minutes to run. However, if I run the same code in Squeak, it takes ~2 minutes. Since my code just uses base collections and streams, I thought the times should be very similar between the two. After a little investigation, I noticed that even simple things like âObject newâ can take much more time in Pharo. Hereâs an example that I executed in Squeak and Pharo: Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]] Squeak times: 1255 1257 1261 1265 1280 1294 1314 1337 1350 1360 Pharo times: 1815 1818 1870 1879 1900 1922 1944 1952 1958 2170 The results are the first 10 executions sorted by time after opening an image. Pharo doesnât always give these poor results. Occasionally I can get times as good a Squeak. For example, I was able to get these times in Pharo: 1253, 1284, 1297, 1314, 1317. However, it generally takes ~1.8 seconds in Pharo vs. the ~1.3 seconds for Squeak. The worst time I got for Squeak was in the 1.6 second range. The worst for Pharo was in the 4.3 second range. Does anyone know why Pharo is slower? Is there some memory setting that I need to change? John Brant
Hello John. I'm just guessing here. Lacking information. It could be: Guess 1) the vm versions are different. What version of Pharo do you use ? (world menu>System>About) Something like: Pharo6.0 Latest update: #60030 What version of Squeak do you use ? (world menu>Help>About this system) Something like: Squeak5.0 latest update: #15792 What vm do you use in both cases ? What does answer ? In Pharo: Smalltalk vm version In Squeak: Smalltalk vmVersion Guess 2) the machine code zone size are different Can you try in Squeak: Smalltalk vmParameterAt: 46 and in Pharo: Smalltalk vm parameterAt: 46. Is it the same ? It's the number of bytes of the machine code zone. If you need performance and you can afford wasting an extra Mb, you can speed it up by doing: Smalltalk vm parameterAt: 47 put: ((Smalltalk vm parameterAt: 46) * 2) And then restarting the image. Don't grow over 2Mb. Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System. On Tue, Jun 21, 2016 at 1:28 AM, John Brant <brant@refactoryworkers.com> wrote:
I have some code that creates a several hundred MB model. When I run the code under Pharo it takes ~2.5 minutes to run. However, if I run the same code in Squeak, it takes ~2 minutes. Since my code just uses base collections and streams, I thought the times should be very similar between the two. After a little investigation, I noticed that even simple things like âObject newâ can take much more time in Pharo. Hereâs an example that I executed in Squeak and Pharo:
Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]]
Squeak times: 1255 1257 1261 1265 1280 1294 1314 1337 1350 1360
Pharo times: 1815 1818 1870 1879 1900 1922 1944 1952 1958 2170
The results are the first 10 executions sorted by time after opening an image. Pharo doesnât always give these poor results. Occasionally I can get times as good a Squeak. For example, I was able to get these times in Pharo: 1253, 1284, 1297, 1314, 1317. However, it generally takes ~1.8 seconds in Pharo vs. the ~1.3 seconds for Squeak. The worst time I got for Squeak was in the 1.6 second range. The worst for Pharo was in the 4.3 second range.
Does anyone know why Pharo is slower? Is there some memory setting that I need to change?
John Brant
On Tue, Jun 21, 2016 at 3:29 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System.
Or try something like... [ Transcript cr; show: (Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]] ) ] forkAt: 75. cheers -ben
On Tue, Jun 21, 2016 at 1:28 AM, John Brant <brant@refactoryworkers.com> wrote:
I have some code that creates a several hundred MB model. When I run the code under Pharo it takes ~2.5 minutes to run. However, if I run the same code in Squeak, it takes ~2 minutes. Since my code just uses base collections and streams, I thought the times should be very similar between the two. After a little investigation, I noticed that even simple things like âObject newâ can take much more time in Pharo. Hereâs an example that I executed in Squeak and Pharo:
Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]]
Squeak times: 1255 1257 1261 1265 1280 1294 1314 1337 1350 1360
Pharo times: 1815 1818 1870 1879 1900 1922 1944 1952 1958 2170
The results are the first 10 executions sorted by time after opening an image. Pharo doesnât always give these poor results. Occasionally I can get times as good a Squeak. For example, I was able to get these times in Pharo: 1253, 1284, 1297, 1314, 1317. However, it generally takes ~1.8 seconds in Pharo vs. the ~1.3 seconds for Squeak. The worst time I got for Squeak was in the 1.6 second range. The worst for Pharo was in the 4.3 second range.
Does anyone know why Pharo is slower? Is there some memory setting that I need to change?
John Brant
On 21 Jun 2016, at 11:58, Ben Coman <btc@openinworld.com> wrote:
On Tue, Jun 21, 2016 at 3:29 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System.
Or try something like... [ Transcript cr; show: (Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]] ) ] forkAt: 75.
BTW, this is essentially a garbage collection benchmark: you create 100,000,000 empty objects. It stresses the GC, especially the ephemeral phase of it. Since the GC is part of the VM, you are testing the VM more than any image code.
cheers -ben
On Tue, Jun 21, 2016 at 1:28 AM, John Brant <brant@refactoryworkers.com> wrote:
I have some code that creates a several hundred MB model. When I run the code under Pharo it takes ~2.5 minutes to run. However, if I run the same code in Squeak, it takes ~2 minutes. Since my code just uses base collections and streams, I thought the times should be very similar between the two. After a little investigation, I noticed that even simple things like âObject newâ can take much more time in Pharo. Hereâs an example that I executed in Squeak and Pharo:
Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]]
Squeak times: 1255 1257 1261 1265 1280 1294 1314 1337 1350 1360
Pharo times: 1815 1818 1870 1879 1900 1922 1944 1952 1958 2170
The results are the first 10 executions sorted by time after opening an image. Pharo doesnât always give these poor results. Occasionally I can get times as good a Squeak. For example, I was able to get these times in Pharo: 1253, 1284, 1297, 1314, 1317. However, it generally takes ~1.8 seconds in Pharo vs. the ~1.3 seconds for Squeak. The worst time I got for Squeak was in the 1.6 second range. The worst for Pharo was in the 4.3 second range.
Does anyone know why Pharo is slower? Is there some memory setting that I need to change?
John Brant
On Tue, Jun 21, 2016 at 3:13 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 21 Jun 2016, at 11:58, Ben Coman <btc@openinworld.com> wrote:
On Tue, Jun 21, 2016 at 3:29 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System.
Or try something like... [ Transcript cr; show: (Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]] ) ] forkAt: 75.
BTW, this is essentially a garbage collection benchmark: you create 100,000,000 empty objects. It stresses the GC, especially the ephemeral phase of it. Since the GC is part of the VM, you are testing the VM more than any image code.
Right, it could be related to the new finalization ... We can't guess properly we have no VM versions.
cheers -ben
On Tue, Jun 21, 2016 at 1:28 AM, John Brant <brant@refactoryworkers.com
wrote:
I have some code that creates a several hundred MB model. When I run
the
code under Pharo it takes ~2.5 minutes to run. However, if I run the same code in Squeak, it takes ~2 minutes. Since my code just uses base collections and streams, I thought the times should be very similar between the two. After a little investigation, I noticed that even simple things like âObject newâ can take much more time in Pharo. Hereâs an example that I executed in Squeak and Pharo:
Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]]
Squeak times: 1255 1257 1261 1265 1280 1294 1314 1337 1350 1360
Pharo times: 1815 1818 1870 1879 1900 1922 1944 1952 1958 2170
The results are the first 10 executions sorted by time after opening an image. Pharo doesnât always give these poor results. Occasionally I can get times as good a Squeak. For example, I was able to get these times in Pharo: 1253, 1284, 1297, 1314, 1317. However, it generally takes ~1.8 seconds in Pharo vs. the ~1.3 seconds for Squeak. The worst time I got for Squeak was in the 1.6 second range. The worst for Pharo was in the 4.3 second range.
Does anyone know why Pharo is slower? Is there some memory setting that I need to change?
John Brant
On 06/21/2016 08:13 AM, Sven Van Caekenberghe wrote:
On 21 Jun 2016, at 11:58, Ben Coman <btc@openinworld.com> wrote:
On Tue, Jun 21, 2016 at 3:29 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System.
Or try something like... [ Transcript cr; show: (Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]] ) ] forkAt: 75.
BTW, this is essentially a garbage collection benchmark: you create 100,000,000 empty objects. It stresses the GC, especially the ephemeral phase of it. Since the GC is part of the VM, you are testing the VM more than any image code.
That's what I thought too, and that's why I was surprised that there was much difference between Squeak and Pharo for this. Furthermore, when I ran the Pharo image using a Squeak vm, I still saw the difference so I thought it must be some image related thing instead of a vm. John Brant
Hi John, On Tue, Jun 21, 2016 at 1:30 PM, John Brant <brant@refactoryworkers.com> wrote:
On 06/21/2016 08:13 AM, Sven Van Caekenberghe wrote:
On 21 Jun 2016, at 11:58, Ben Coman <btc@openinworld.com> wrote:
On Tue, Jun 21, 2016 at 3:29 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System.
Or try something like... [ Transcript cr; show: (Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]] ) ] forkAt: 75.
BTW, this is essentially a garbage collection benchmark: you create 100,000,000 empty objects. It stresses the GC, especially the ephemeral phase of it. Since the GC is part of the VM, you are testing the VM more than any image code.
That's what I thought too, and that's why I was surprised that there was much difference between Squeak and Pharo for this. Furthermore, when I ran the Pharo image using a Squeak vm, I still saw the difference so I thought it must be some image related thing instead of a vm.
FYI, the differences between the VMs are minimal. The base VMs differ only in the FilePlugin API, such that the Pharo API supports accessing POSIX file permissions whereas the Squeak one does not. There are differences in the set of external plugins, Pharo providing Freetype and Cairo support, whereas buy default the Squeak VM does not. But in the execution machinery they are identical. So it's a relief to me that you've pin-pointed a UI issue. That makes sense to me, whereas, say, differences in GC performance due to finalisation load would confuse me as I see no likely cause of the size of effect you're seeing. HTH John Brant
_,,,^..^,,,_ best, Eliot
On 21 Jun 2016, at 23:32, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi John,
On Tue, Jun 21, 2016 at 1:30 PM, John Brant <brant@refactoryworkers.com <mailto:brant@refactoryworkers.com>> wrote: On 06/21/2016 08:13 AM, Sven Van Caekenberghe wrote:
On 21 Jun 2016, at 11:58, Ben Coman <btc@openinworld.com <mailto:btc@openinworld.com>> wrote:
On Tue, Jun 21, 2016 at 3:29 PM, Clément Bera <bera.clement@gmail.com <mailto:bera.clement@gmail.com>> wrote: Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System.
Or try something like... [ Transcript cr; show: (Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]] ) ] forkAt: 75.
BTW, this is essentially a garbage collection benchmark: you create 100,000,000 empty objects. It stresses the GC, especially the ephemeral phase of it. Since the GC is part of the VM, you are testing the VM more than any image code.
That's what I thought too, and that's why I was surprised that there was much difference between Squeak and Pharo for this. Furthermore, when I ran the Pharo image using a Squeak vm, I still saw the difference so I thought it must be some image related thing instead of a vm.
FYI, the differences between the VMs are minimal. The base VMs differ only in the FilePlugin API, such that the Pharo API supports accessing POSIX file permissions whereas the Squeak one does not. There are differences in the set of external plugins, Pharo providing Freetype and Cairo support, whereas buy default the Squeak VM does not. But in the execution machinery they are identical. So it's a relief to me that you've pin-pointed a UI issue. That makes sense to me, whereas, say, differences in GC performance due to finalisation load would confuse me as I see no likely cause of the size of effect you're seeing.
yes, I was going to point there⦠also maybe some compilations flag could change, but I do not think so. Anyway⦠can you test headless? pharo Pharo.image eval âetcâ and if not, also testing with old Workspace (instead Playground and then Rubric) would be nice: Workspace open. (and of course, all other windows closed) cheers, Esteban
HTH
John Brant
_,,,^..^,,,_ best, Eliot
Also, to remove noise from finalization process, you can try to execute your snippet in a higher process priority. 80 meaning it cannot be interrupted. On Tue, Jun 21, 2016 at 11:38 PM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 21 Jun 2016, at 23:32, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi John,
On Tue, Jun 21, 2016 at 1:30 PM, John Brant <brant@refactoryworkers.com> wrote:
On 06/21/2016 08:13 AM, Sven Van Caekenberghe wrote:
On 21 Jun 2016, at 11:58, Ben Coman <btc@openinworld.com> wrote:
On Tue, Jun 21, 2016 at 3:29 PM, Clément Bera <bera.clement@gmail.com> wrote:
Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System.
Or try something like... [ Transcript cr; show: (Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]] ) ] forkAt: 75.
BTW, this is essentially a garbage collection benchmark: you create 100,000,000 empty objects. It stresses the GC, especially the ephemeral phase of it. Since the GC is part of the VM, you are testing the VM more than any image code.
That's what I thought too, and that's why I was surprised that there was much difference between Squeak and Pharo for this. Furthermore, when I ran the Pharo image using a Squeak vm, I still saw the difference so I thought it must be some image related thing instead of a vm.
FYI, the differences between the VMs are minimal. The base VMs differ only in the FilePlugin API, such that the Pharo API supports accessing POSIX file permissions whereas the Squeak one does not. There are differences in the set of external plugins, Pharo providing Freetype and Cairo support, whereas buy default the Squeak VM does not. But in the execution machinery they are identical. So it's a relief to me that you've pin-pointed a UI issue. That makes sense to me, whereas, say, differences in GC performance due to finalisation load would confuse me as I see no likely cause of the size of effect you're seeing.
yes, I was going to point there⦠also maybe some compilations flag could change, but I do not think so. Anyway⦠can you test headless?
pharo Pharo.image eval âetcâ
and if not, also testing with old Workspace (instead Playground and then Rubric) would be nice:
Workspace open.
(and of course, all other windows closed)
cheers, Esteban
HTH
John Brant
_,,,^..^,,,_ best, Eliot
On 06/21/2016 02:29 AM, Clément Bera wrote:
Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 1) the vm versions are different. What version of Pharo do you use ? (world menu>System>About) Something like: Pharo6.0 Latest update: #60030 What version of Squeak do you use ? (world menu>Help>About this system) Something like: Squeak5.0 latest update: #15792 What vm do you use in both cases ? What does answer ? In Pharo: Smalltalk vm version In Squeak: Smalltalk vmVersion
I just downloaded the latest to test on my Linux machine: Pharo6.0 Latest update: #60101 Pharo VM: 'CoInterpreter VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Jun 21 2016 StackToRegisterMappingCogit VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Jun 21 2016 https://github.com/pharo-project/pharo-vm.git Commit: 9638b0190a9fc01479bfb752becd96edfd253c8c Date: 2016-06-21 12:29:26 +0200 By: GitHub <noreply@github.com> Jenkins build #594 ' Squeak: Image ----- /home/brant/Smalltalk/Squeak/SpurTrunkImage.image Squeak5.1 latest update: #16066 Current Change Set: Unnamed1 Image format 6521 (32 bit) Virtual Machine --------------- /home/brant/Smalltalk/Squeak/lib/squeak/5.0-3732/squeak Croquet Closure Cog[Spur] VM [CoInterpreterPrimitives VMMaker.oscog-eem.1872] Unix built on May 26 2016 15:18:08 Compiler: 4.4.7 20120313 (Red Hat 4.4.7-4) platform sources revision VM: r3732 http://www.squeakvm.org/svn/squeak/branches/Cog Date: 2016-05-26 13:01:37 -0700 Plugins: r3730 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins CoInterpreter VMMaker.oscog-eem.1872 uuid: 6db6d610-b1a5-4f4d-978d-22c917bdb3e4 May 26 2016 StackToRegisterMappingCogit VMMaker.oscog-eem.1860 uuid: 4d58d995-ee0c-4330-9190-adfa038e8f24 May 26 2016 Starting the image, opening a workspace/playground, and executing "Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]]" 10 times, I got these times in Pharo: "2147" "2303" "1177" "1127" "1153" "1179" "1162" "1136" "1262" "1148" Closing that image, and doing it again, I got these: "2128" "2210" "2239" "2051" "2040" "2070" "2087" "2166" "2060" "2078" Doing it once in Squeak, I got: 1183 1189 1193 1189 1216 1238 1193 1273 1218 1214 I done it a few more times in both, and Squeak always had times from the upper 1100's to the upper 1200's, but Pharo had times from the low 1100's to the 2300's.
Guess 2) the machine code zone size are different Can you try in Squeak: Smalltalk vmParameterAt: 46 and in Pharo: Smalltalk vm parameterAt: 46.
Is it the same ? It's the number of bytes of the machine code zone. If you need performance and you can afford wasting an extra Mb, you can speed it up by doing: Smalltalk vm parameterAt: 47 put: ((Smalltalk vm parameterAt: 46) * 2) And then restarting the image. Don't grow over 2Mb.
They are both 1MB.
Guess 3) the UI is known to be much slower in Pharo. Can you try headless or after ticking "Server mode" In the Pharo settings in System.
This appears to be what is causing the problem. Even though I'm not using any UI when I run my code, it must be doing something in the background that is causing the slowdown. When I ran my model code headlessly in Pharo, it actually finished the 2 minute job a couple seconds faster than the headless Squeak image (instead of 30 seconds slower). John Brant
On Tue, Jun 21, 2016 at 10:18 PM, John Brant <brant@refactoryworkers.com> wrote:
On 06/21/2016 02:29 AM, Clément Bera wrote:
Hello John.
I'm just guessing here. Lacking information. It could be:
Guess 1) the vm versions are different. What version of Pharo do you use ? (world menu>System>About) Something like: Pharo6.0 Latest update: #60030 What version of Squeak do you use ? (world menu>Help>About this system) Something like: Squeak5.0 latest update: #15792 What vm do you use in both cases ? What does answer ? In Pharo: Smalltalk vm version In Squeak: Smalltalk vmVersion
I just downloaded the latest to test on my Linux machine:
Pharo6.0 Latest update: #60101
Pharo VM: 'CoInterpreter VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Jun 21 2016 StackToRegisterMappingCogit VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Jun 21 2016 https://github.com/pharo-project/pharo-vm.git Commit: 9638b0190a9fc01479bfb752becd96edfd253c8c Date: 2016-06-21 12:29:26 +0200 By: GitHub <noreply@github.com> Jenkins build #594 '
Squeak: Image ----- /home/brant/Smalltalk/Squeak/SpurTrunkImage.image Squeak5.1 latest update: #16066 Current Change Set: Unnamed1 Image format 6521 (32 bit)
Virtual Machine --------------- /home/brant/Smalltalk/Squeak/lib/squeak/5.0-3732/squeak Croquet Closure Cog[Spur] VM [CoInterpreterPrimitives VMMaker.oscog-eem.1872] Unix built on May 26 2016 15:18:08 Compiler: 4.4.7 20120313 (Red Hat 4.4.7-4) platform sources revision VM: r3732 http://www.squeakvm.org/svn/squeak/branches/Cog Date: 2016-05-26 13:01:37 -0700 Plugins: r3730 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins CoInterpreter VMMaker.oscog-eem.1872 uuid: 6db6d610-b1a5-4f4d-978d-22c917bdb3e4 May 26 2016 StackToRegisterMappingCogit VMMaker.oscog-eem.1860 uuid: 4d58d995-ee0c-4330-9190-adfa038e8f24 May 26 2016
Starting the image, opening a workspace/playground, and executing "Time millisecondsToRun: [1 to: 100000000 do: [:i | Object new]]" 10 times, I got these times in Pharo:
"2147" "2303" "1177" "1127" "1153" "1179" "1162" "1136" "1262" "1148"
Closing that image, and doing it again, I got these:
"2128" "2210" "2239" "2051" "2040" "2070" "2087" "2166" "2060" "2078"
Doing it once in Squeak, I got:
1183 1189 1193 1189 1216 1238 1193 1273 1218 1214
I done it a few more times in both, and Squeak always had times from the upper 1100's to the upper 1200's, but Pharo had times from the low 1100's to the 2300's.
Guess 2) the machine code zone size are different
Can you try in Squeak: Smalltalk vmParameterAt: 46 and in Pharo: Smalltalk vm parameterAt: 46.
Is it the same ? It's the number of bytes of the machine code zone. If you need performance and you can afford wasting an extra Mb, you can speed it up by doing: Smalltalk vm parameterAt: 47 put: ((Smalltalk vm parameterAt: 46) * 2) And then restarting the image. Don't grow over 2Mb.
They are both 1MB.
Guess 3) the UI is known to be much slower in Pharo. Can you try
headless or after ticking "Server mode" In the Pharo settings in System.
This appears to be what is causing the problem. Even though I'm not using any UI when I run my code, it must be doing something in the background that is causing the slowdown. When I ran my model code headlessly in Pharo, it actually finished the 2 minute job a couple seconds faster than the headless Squeak image (instead of 30 seconds slower).
Very interesting. So it's UI-related. If you close all windows but the workspace where you run your code, does it solve the problem ? I believe it tries to update something somehow.
John Brant
participants (7)
-
Ben Coman -
Clément Bera -
Eliot Miranda -
Esteban Lorenzano -
Guillermo Polito -
John Brant -
Sven Van Caekenberghe