Slow compilation on one of my Windows PCs
Hello I have a long-term problem with loading of packages. Compilation takes very long time on my Windows PCs. On desktop it takes "only" double amount of time in comparison with linux, but on notebook, it is more than 10 times slower. Desktop has Windows 7 Ultimate 64-bit and notebook PC with Windows 7 Home Premium 64-bit, both have somehow similar specs (4-6 years old dual-core processors on 2,9 and 2,4 GHz), similar software, both have magnetic HDDs. I believe it used to take very long time on my notebook too, but I tried opening Pharo there again and it was somehow OK, but I'm not aware of any major change which could have caused it ... I mean... a lot changes on my PCs, but I can't remember anything which could directly affect it. Following "benchmarks" are both on same (copy paste) of VM - Windows VM 453 from 2015-06-18. Both on clean image 50141. All tested in as much as similar conditions as possible with everything else (possible) turned off. However, it used to be slow like that for a long time on multiple previous VMs and images since I can even remember (which is about a year). [ Gofer new configurationOf: 'Roassal2'; url: 'http://smalltalkhub.com/mc/ObjectProfile/Roassal2/main/'; loadDevelopment. ] timeToRun inspect. Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s. |results| results := OrderedCollection new. 20 timesRepeat: [results add: (Float readFrom: [ Object compile: 'a'. ] bench)]. ('compilations per second - avg: ', results average asString, ', min: ', results min asString, ', max: ', results max asString) inspect. Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97 I tried turning off antivirus, turning off file indexing, praying to multiple gods, but with no success. I will appreciate any help. Jan -- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668.ht... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
I've had also bad results on windows... Windows XP 32bit (VirtualBox) - 178.200 (compilations per second) compared to Ubuntu 32bit (VirtualBox) - 694.932 Debian 64bit (host) - 669.932 I had also had experience with Git handling small files on windows was incredibly slow... so maybe it has something to do with disk writes? What exactly is happening when a method is compiled? A write to changes browser, to the image, parsing, ...? Peter On Mon, Jun 29, 2015 at 7:52 PM, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
Hello
I have a long-term problem with loading of packages. Compilation takes very long time on my Windows PCs. On desktop it takes "only" double amount of time in comparison with linux, but on notebook, it is more than 10 times slower. Desktop has Windows 7 Ultimate 64-bit and notebook PC with Windows 7 Home Premium 64-bit, both have somehow similar specs (4-6 years old dual-core processors on 2,9 and 2,4 GHz), similar software, both have magnetic HDDs. I believe it used to take very long time on my notebook too, but I tried opening Pharo there again and it was somehow OK, but I'm not aware of any major change which could have caused it ... I mean... a lot changes on my PCs, but I can't remember anything which could directly affect it.
Following "benchmarks" are both on same (copy paste) of VM - Windows VM 453 from 2015-06-18. Both on clean image 50141. All tested in as much as similar conditions as possible with everything else (possible) turned off. However, it used to be slow like that for a long time on multiple previous VMs and images since I can even remember (which is about a year).
[ Gofer new configurationOf: 'Roassal2'; url: ' http://smalltalkhub.com/mc/ObjectProfile/Roassal2/main/'; loadDevelopment. ] timeToRun inspect.
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
|results| results := OrderedCollection new. 20 timesRepeat: [results add: (Float readFrom: [ Object compile: 'a'. ] bench)]. ('compilations per second - avg: ', results average asString, ', min: ', results min asString, ', max: ', results max asString) inspect.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
I tried turning off antivirus, turning off file indexing, praying to multiple gods, but with no success.
I will appreciate any help.
Jan
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668.ht... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of... I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time. Jan Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
What about "1 tinyBenchmarks" ? Just to know if the VM is slower as a whole or only compilation / source access ? 2015-06-30 0:36 GMT+02:00 Jan BlizniÄenko <bliznjan@fit.cvut.cz>:
And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
2015-06-30 3:17 GMT+02:00 Clément Bera <bera.clement@gmail.com>:
What about "1 tinyBenchmarks" ?
Just to know if the VM is slower as a whole or only compilation / source access ?
most time is spend on primitives FilePrimitives are slow on windows.
2015-06-30 0:36 GMT+02:00 Jan BlizniÄenko <bliznjan@fit.cvut.cz>:
And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Windows XP 1011857707 bytecodes/sec; 132044070 sends/sec Debian 1172295363 bytecodes/sec; 144744207 sends/sec But I think taking the compile: method apart tells a better story ~~~~~~~~~~~~~~~~~~~ code := 'a'. method := Object compiler source: code; requestor: nil; failBlock: [ ^nil ]; compile. all := [ Object compile: code. ]. compile := [ Object compiler source: code; requestor: nil; failBlock: [ ^nil ]; compile. ]. store := [ method putSource: code inFile: 2 withPreamble: [:f | f cr; nextPut: $!; nextChunkPut: 'Behavior method'; cr]. ]. add := [ Object addSelector: method selector withMethod: method notifying: nil. ]. ~~~~~~~~~~~~~~~~~~~ Windows XP: ~~~~~~~~~~~~~~~~~~~~~~~~~~~ all bench. "'183.763 per second'" compile bench. "'10,729 per second'" *store bench. "'286.170 per second'"* add bench. "'697.883 per second'" ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Debian ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ all bench. "'622.551 per second'" compile bench. "'11,129 per second'" *store bench. "'41,604 per second'"* add bench. "'787.885 per second'" ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 150x slower disk operations? -_- I mean for normal programming this doesn't really matter, but when we are downloading even medium-sized package it takes ages to process... Peter
I ran it in Pharo 4 (40616) because method putSource:inFile:withPreamble: no longer exists in Pharo 5 and results are: Desktop... all: 13.062 per second compile: 6,603 per second (why the hell is there comma as separator and not dot?) store: 13.431 per second add: 406.400 per second Notebook... all: 210.180 per second compile: 12,007 per second store: 454.818 per second add: 806.239 per second Jan Peter Uhnák wrote
Windows XP 1011857707 bytecodes/sec; 132044070 sends/sec
Debian 1172295363 bytecodes/sec; 144744207 sends/sec
But I think taking the compile: method apart tells a better story
~~~~~~~~~~~~~~~~~~~ code := 'a'.
method := Object compiler source: code; requestor: nil; failBlock: [ ^nil ]; compile.
all := [ Object compile: code. ].
compile := [ Object compiler source: code; requestor: nil; failBlock: [ ^nil ]; compile. ].
store := [ method putSource: code inFile: 2 withPreamble: [:f | f cr; nextPut: $!; nextChunkPut: 'Behavior method'; cr]. ].
add := [ Object addSelector: method selector withMethod: method notifying: nil. ]. ~~~~~~~~~~~~~~~~~~~
Windows XP: ~~~~~~~~~~~~~~~~~~~~~~~~~~~ all bench. "'183.763 per second'" compile bench. "'10,729 per second'" *store bench. "'286.170 per second'"* add bench. "'697.883 per second'" ~~~~~~~~~~~~~~~~~~~~~~~~~~~
Debian ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ all bench. "'622.551 per second'" compile bench. "'11,129 per second'" *store bench. "'41,604 per second'"* add bench. "'787.885 per second'" ~~~~~~~~~~~~~~~~~~~~~~~~~~~
150x slower disk operations? -_-
I mean for normal programming this doesn't really matter, but when we are downloading even medium-sized package it takes ages to process...
Peter
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Do you have antivirus running in windows that may be bothering with .changes? Please try temporary disable antivirus and try again. On Tue, Jun 30, 2015 at 4:25 AM, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
I ran it in Pharo 4 (40616) because method putSource:inFile:withPreamble: no longer exists in Pharo 5 and results are:
Desktop... all: 13.062 per second compile: 6,603 per second (why the hell is there comma as separator and not dot?) store: 13.431 per second add: 406.400 per second
Notebook... all: 210.180 per second compile: 12,007 per second store: 454.818 per second add: 806.239 per second
Jan
Peter Uhnák wrote
Windows XP 1011857707 bytecodes/sec; 132044070 sends/sec
Debian 1172295363 bytecodes/sec; 144744207 sends/sec
But I think taking the compile: method apart tells a better story
~~~~~~~~~~~~~~~~~~~ code := 'a'.
method := Object compiler source: code; requestor: nil; failBlock: [ ^nil ]; compile.
all := [ Object compile: code. ].
compile := [ Object compiler source: code; requestor: nil; failBlock: [ ^nil ]; compile. ].
store := [ method putSource: code inFile: 2 withPreamble: [:f | f cr; nextPut: $!; nextChunkPut: 'Behavior method'; cr]. ].
add := [ Object addSelector: method selector withMethod: method notifying: nil. ]. ~~~~~~~~~~~~~~~~~~~
Windows XP: ~~~~~~~~~~~~~~~~~~~~~~~~~~~ all bench. "'183.763 per second'" compile bench. "'10,729 per second'" *store bench. "'286.170 per second'"* add bench. "'697.883 per second'" ~~~~~~~~~~~~~~~~~~~~~~~~~~~
Debian ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ all bench. "'622.551 per second'" compile bench. "'11,129 per second'" *store bench. "'41,604 per second'"* add bench. "'787.885 per second'" ~~~~~~~~~~~~~~~~~~~~~~~~~~~
150x slower disk operations? -_-
I mean for normal programming this doesn't really matter, but when we are downloading even medium-sized package it takes ages to process...
Peter
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
Unfortunately no - all benchmarks I made with antivirus disabled. Mariano Martinez Peck wrote
Do you have antivirus running in windows that may be bothering with .changes? Please try temporary disable antivirus and try again.
-- Mariano http://marianopeck.wordpress.com
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Can you think of any other window process that could be detecting changes all the time in .changes and therefore have an impact in the performance? Do you have such pharo image in dropbox or similar service? On Tue, Jun 30, 2015 at 9:37 AM, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
Unfortunately no - all benchmarks I made with antivirus disabled.
Mariano Martinez Peck wrote
Do you have antivirus running in windows that may be bothering with .changes? Please try temporary disable antivirus and try again.
-- Mariano http://marianopeck.wordpress.com
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
we really should get rid of the .changes file⦠it would simplify things a lot.
On 30 Jun 2015, at 15:02, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
Can you think of any other window process that could be detecting changes all the time in .changes and therefore have an impact in the performance? Do you have such pharo image in dropbox or similar service?
On Tue, Jun 30, 2015 at 9:37 AM, Jan BlizniÄenko <bliznjan@fit.cvut.cz <mailto:bliznjan@fit.cvut.cz>> wrote: Unfortunately no - all benchmarks I made with antivirus disabled.
Mariano Martinez Peck wrote
Do you have antivirus running in windows that may be bothering with .changes? Please try temporary disable antivirus and try again.
-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... <http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48...> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>
I'm gonna try to reply to all of you and try your ideas Stephan Eggermont wrote
On 30-06-15 14:37, Jan BlizniÄenko wrote:
Unfortunately no - all benchmarks I made with antivirus disabled.
Including the Microsoft stuff itself? (Security Essentials/Windows Defender) Does your desktop have a drive > 2TB?
Yes, all such things I have disabled / removed. Mariano Martinez Peck wrote
Can you think of any other window process that could be detecting changes all the time in .changes and therefore have an impact in the performance? Do you have such pharo image in dropbox or similar service?
I don't think so. I tried even running Windows in diagnostic startup (safe mode / running with only core processes and services) and it had no effect. Mariano Martinez Peck wrote
I would check the processes running and CPU of Windows while loading the code...
I was looking at it some time ago and it didn't seem like processor is too much busy, but as I think about it, it is suspiciously almost not busy at all ... ~1 % of processor usage. On laptop it fluctuates between 5 and 10 %. I'm talking about usage when benchmarking the "store" code. philippeback wrote
Silly question: do you have a couple of Nautilus windows open?
Loading stuff generates annoucements and Nautilus updates are killing performance.
No, only open thing is Playground from which I run the benchmarking codes. Sven Van Caekenberghe-2 wrote
On 30 Jun 2015, at 18:36, Peter Uhnák <
i.uhnak@
> wrote:
I think we've safely established that the bottleneck is disk operations.
Let's take it one level down then,
[ 'foo.txt' asFileReference in: [ :file | file writeStreamDo: [ :out | 3 timesRepeat: [ out << String loremIpsum ] ]. file ensureDelete ] ] bench
=> "'512.595 per second'"
We could experiment with variants, calling #flush, writing by character, adding buffering, etc, but this is a start. At least this takes the source code stuff out of the equations.
"654.738 per second" on desktop "1,271 per second" on laptop Thank you all for ideas. Jan -- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Then the only thing I can think of is a vm primitive that is implemented differently in linux/mac than windows vm... On Tue, Jun 30, 2015 at 7:07 PM, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
I'm gonna try to reply to all of you and try your ideas
Stephan Eggermont wrote
On 30-06-15 14:37, Jan BlizniÄenko wrote:
Unfortunately no - all benchmarks I made with antivirus disabled.
Including the Microsoft stuff itself? (Security Essentials/Windows Defender) Does your desktop have a drive > 2TB?
Yes, all such things I have disabled / removed.
Mariano Martinez Peck wrote
Can you think of any other window process that could be detecting changes all the time in .changes and therefore have an impact in the performance? Do you have such pharo image in dropbox or similar service?
I don't think so. I tried even running Windows in diagnostic startup (safe mode / running with only core processes and services) and it had no effect.
Mariano Martinez Peck wrote
I would check the processes running and CPU of Windows while loading the code...
I was looking at it some time ago and it didn't seem like processor is too much busy, but as I think about it, it is suspiciously almost not busy at all ... ~1 % of processor usage. On laptop it fluctuates between 5 and 10 %. I'm talking about usage when benchmarking the "store" code.
philippeback wrote
Silly question: do you have a couple of Nautilus windows open?
Loading stuff generates annoucements and Nautilus updates are killing performance.
No, only open thing is Playground from which I run the benchmarking codes.
Sven Van Caekenberghe-2 wrote
On 30 Jun 2015, at 18:36, Peter Uhnák <
i.uhnak@
> wrote:
I think we've safely established that the bottleneck is disk operations.
Let's take it one level down then,
[ 'foo.txt' asFileReference in: [ :file | file writeStreamDo: [ :out | 3 timesRepeat: [ out << String loremIpsum ] ]. file ensureDelete ] ] bench
=> "'512.595 per second'"
We could experiment with variants, calling #flush, writing by character, adding buffering, etc, but this is a start. At least this takes the source code stuff out of the equations.
"654.738 per second" on desktop "1,271 per second" on laptop
Thank you all for ideas. Jan
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
2015-07-01 0:39 GMT+02:00 Mariano Martinez Peck <marianopeck@gmail.com>:
Then the only thing I can think of is a vm primitive that is implemented differently in linux/mac than windows vm...
Yes, the vm primitives, like I already told some messages above. FilePrimitives ARE slow on windows. We may get better performance, if we disable windows file cache/buffering and use slightly different ways to do the CreateFile and WriteFile calls, and do the buffering on our own, but this is not easy. Luckily (?), this may be alread work if we use the std file api (?) A small dirty test: store bench latest vm -> '22.098 per second' modified vm -> '31,027 per second' (in the modified vm I replaced all file operation from sqWin32FilePrims.c with the code from sqFilePluginBasicPrims.c) The result may seem strange, because both implementation will actually use win32s CreateFile/WriteFile methods, but maybe the second one uses better caching/buffering. nicolai
On Tue, Jun 30, 2015 at 7:07 PM, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
I'm gonna try to reply to all of you and try your ideas
Stephan Eggermont wrote
On 30-06-15 14:37, Jan BlizniÄenko wrote:
Unfortunately no - all benchmarks I made with antivirus disabled.
Including the Microsoft stuff itself? (Security Essentials/Windows Defender) Does your desktop have a drive > 2TB?
Yes, all such things I have disabled / removed.
Mariano Martinez Peck wrote
Can you think of any other window process that could be detecting changes all the time in .changes and therefore have an impact in the performance? Do you have such pharo image in dropbox or similar service?
I don't think so. I tried even running Windows in diagnostic startup (safe mode / running with only core processes and services) and it had no effect.
Mariano Martinez Peck wrote
I would check the processes running and CPU of Windows while loading the code...
I was looking at it some time ago and it didn't seem like processor is too much busy, but as I think about it, it is suspiciously almost not busy at all ... ~1 % of processor usage. On laptop it fluctuates between 5 and 10 %. I'm talking about usage when benchmarking the "store" code.
philippeback wrote
Silly question: do you have a couple of Nautilus windows open?
Loading stuff generates annoucements and Nautilus updates are killing performance.
No, only open thing is Playground from which I run the benchmarking codes.
Sven Van Caekenberghe-2 wrote
On 30 Jun 2015, at 18:36, Peter Uhnák <
i.uhnak@
> wrote:
I think we've safely established that the bottleneck is disk
operations.
Let's take it one level down then,
[ 'foo.txt' asFileReference in: [ :file | file writeStreamDo: [ :out | 3 timesRepeat: [ out << String loremIpsum ] ]. file ensureDelete ] ] bench
=> "'512.595 per second'"
We could experiment with variants, calling #flush, writing by character, adding buffering, etc, but this is a start. At least this takes the source code stuff out of the equations.
"654.738 per second" on desktop "1,271 per second" on laptop
Thank you all for ideas. Jan
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
On Wed, Jul 01, 2015 at 01:39:25AM +0200, Nicolai Hess wrote:
2015-07-01 0:39 GMT+02:00 Mariano Martinez Peck <marianopeck@gmail.com>:
Then the only thing I can think of is a vm primitive that is implemented differently in linux/mac than windows vm...
Yes, the vm primitives, like I already told some messages above. FilePrimitives ARE slow on windows.
Is that actually true? I have not done any work on Windows in a while, but I recall that file primitives on Windows were quite efficient. They work differently because they operate at the level of direct file handles rather than stdio streams, but this may be faster in many cases. So unless you have measurable evidence that the Windows IO is slower, don't assume it to be true. Dave
Sounds good, could I try it and run all those benchmarks etc. on it on my PCs? Jan Nicolai Hess wrote
Yes, the vm primitives, like I already told some messages above. FilePrimitives ARE slow on windows.
We may get better performance, if we disable windows file cache/buffering and use slightly different ways to do the CreateFile and WriteFile calls, and do the buffering on our own, but this is not easy.
Luckily (?), this may be alread work if we use the std file api (?)
A small dirty test:
store bench latest vm -> '22.098 per second' modified vm -> '31,027 per second'
(in the modified vm I replaced all file operation from sqWin32FilePrims.c with the code from sqFilePluginBasicPrims.c)
The result may seem strange, because both implementation will actually use win32s CreateFile/WriteFile methods, but maybe the second one uses better caching/buffering.
nicolai
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
2015-07-01 7:46 GMT+02:00 Jan BlizniÄenko <bliznjan@fit.cvut.cz>:
Sounds good, could I try it and run all those benchmarks etc. on it on my PCs?
Jan
With this: http://stackoverflow.com/questions/14290337/is-fwrite-faster-than-writefile-... It seems this all comes down to calling FlushFileBuffers. You can check what would happen if we just disabling the explicit flushing (comment out self primFlush: fileID in StandardFileStream>>#flush) and run the benchmarks. (Of course, I don't know what side effects this have, *I* think WriteFile calls flush on its own, and of course *if* this solves the problem, we can not just disable call to flush in the image, but have to change the windows file plugin).
Nicolai Hess wrote
Yes, the vm primitives, like I already told some messages above. FilePrimitives ARE slow on windows.
We may get better performance, if we disable windows file cache/buffering and use slightly different ways to do the CreateFile and WriteFile calls, and do the buffering on our own, but this is not easy.
Luckily (?), this may be alread work if we use the std file api (?)
A small dirty test:
store bench latest vm -> '22.098 per second' modified vm -> '31,027 per second'
(in the modified vm I replaced all file operation from sqWin32FilePrims.c with the code from sqFilePluginBasicPrims.c)
The result may seem strange, because both implementation will actually use win32s CreateFile/WriteFile methods, but maybe the second one uses better caching/buffering.
nicolai
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
I tried commenting primFlush: fileID in StandardFileStream>>#flush on my desktop PC and the "store" benchmark's speed improved significantly. Original result on Windows 7: 11 per sec Result without flushing on Windows 7: 9 430 per sec Original result on Linux Mint 17: 26 590 per sec Result without flushing on Linux Mint 17: 34 879 per sec Mentioned Linux Mint is in VirtualBox on the same PC. Also loading of Roassal2 now takes 58 seconds insted of 386. Jan Nicolai Hess wrote
2015-07-01 7:46 GMT+02:00 Jan BlizniÄenko <
bliznjan@.cvut
>:
Sounds good, could I try it and run all those benchmarks etc. on it on my PCs?
Jan
With this: http://stackoverflow.com/questions/14290337/is-fwrite-faster-than-writefile-... It seems this all comes down to calling FlushFileBuffers.
You can check what would happen if we just disabling the explicit flushing (comment out self primFlush: fileID in StandardFileStream>>#flush) and run the benchmarks.
(Of course, I don't know what side effects this have, *I* think WriteFile calls flush on its own, and of course *if* this solves the problem, we can not just disable call to flush in the image, but have to change the windows file plugin).
Nicolai Hess wrote
Yes, the vm primitives, like I already told some messages above. FilePrimitives ARE slow on windows.
We may get better performance, if we disable windows file cache/buffering and use slightly different ways to do the CreateFile and WriteFile calls, and do the buffering on our own, but this is not easy.
Luckily (?), this may be alread work if we use the std file api (?)
A small dirty test:
store bench latest vm -> '22.098 per second' modified vm -> '31,027 per second'
(in the modified vm I replaced all file operation from sqWin32FilePrims.c with the code from sqFilePluginBasicPrims.c)
The result may seem strange, because both implementation will actually use win32s CreateFile/WriteFile methods, but maybe the second one uses better caching/buffering.
nicolai
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
I'm experimenting with commenting the flush automatically by startup script and loading now takes reasonable amount of time ( StandardFileStream compile: 'flush'. ). I haven't found any drawbacks so far, but it doesn't mean anything and that "manual" flushing probably is there for a reason, what is the reason? Jan Jan BlizniÄenko wrote
I tried commenting primFlush: fileID in StandardFileStream>>#flush on my desktop PC and the "store" benchmark's speed improved significantly.
Original result on Windows 7: 11 per sec Result without flushing on Windows 7: 9 430 per sec Original result on Linux Mint 17: 26 590 per sec Result without flushing on Linux Mint 17: 34 879 per sec
Mentioned Linux Mint is in VirtualBox on the same PC.
Also loading of Roassal2 now takes 58 seconds insted of 386.
Jan
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
#flush on a stream means pushing all data to the final destination, clearing buffers, doing actual network transfers. What can happen when you disable that ? That some data does not arrive where it should I guess. Mind that #close most of the time does an automatic/implicit #flush. Anyway, I don't think disabling #flush is a real solution.
On 02 Jul 2015, at 17:46, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
I'm experimenting with commenting the flush automatically by startup script and loading now takes reasonable amount of time ( StandardFileStream compile: 'flush'. ). I haven't found any drawbacks so far, but it doesn't mean anything and that "manual" flushing probably is there for a reason, what is the reason?
Jan
Jan BlizniÄenko wrote
I tried commenting primFlush: fileID in StandardFileStream>>#flush on my desktop PC and the "store" benchmark's speed improved significantly.
Original result on Windows 7: 11 per sec Result without flushing on Windows 7: 9 430 per sec Original result on Linux Mint 17: 26 590 per sec Result without flushing on Linux Mint 17: 34 879 per sec
Mentioned Linux Mint is in VirtualBox on the same PC.
Also loading of Roassal2 now takes 58 seconds insted of 386.
Jan
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
2015-07-02 19:01 GMT+02:00 Sven Van Caekenberghe <sven@stfx.eu>:
#flush on a stream means pushing all data to the final destination, clearing buffers, doing actual network transfers.
What can happen when you disable that ?
That some data does not arrive where it should I guess.
Mind that #close most of the time does an automatic/implicit #flush.
Anyway, I don't think disabling #flush is a real solution.
+1 Maybe it is enough, if we remove the call to "self flush" in WriteStream>>#nextChunkPut: ? I see that squeak does not call flush, and :) in Squeak WriteStream>>#flush is empty (!) (But I think in squeak and pharo are many differences for stream and source/changes handling)
On 02 Jul 2015, at 17:46, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
I'm experimenting with commenting the flush automatically by startup script and loading now takes reasonable amount of time ( StandardFileStream compile: 'flush'. ). I haven't found any drawbacks so far, but it doesn't mean anything and that "manual" flushing probably is there for a reason, what is the reason?
Jan
Jan BlizniÄenko wrote
I tried commenting primFlush: fileID in StandardFileStream>>#flush on my desktop PC and the "store" benchmark's speed improved significantly.
Original result on Windows 7: 11 per sec Result without flushing on Windows 7: 9 430 per sec Original result on Linux Mint 17: 26 590 per sec Result without flushing on Linux Mint 17: 34 879 per sec
Mentioned Linux Mint is in VirtualBox on the same PC.
Also loading of Roassal2 now takes 58 seconds insted of 386.
Jan
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Flush is, in the store benchmark I used, called from from MultiByteFileStream(WriteStream) >>#nextChunkPut: and from CompiledMethod>>#putSource:inFile:withPreamble: ... when I comment out calling flush in these two methods, it is fast like when whole content of flush method is commented out. However, If I keep these two flush calls commented, but I once call flush manually in the end of the "store" code, it is slow again (well, instead of 8-18 runs per sec it runs 40-50 times per sec, but it is far from 9000 when flush does not happen at all). It seems to me like the problem is not in how many times flush is called, but more in what actually happens when flush is called (how is the primitive primFlush: achieved on Windows). Jan Nicolai Hess wrote
2015-07-02 19:01 GMT+02:00 Sven Van Caekenberghe <
sven@
>:
#flush on a stream means pushing all data to the final destination, clearing buffers, doing actual network transfers.
What can happen when you disable that ?
That some data does not arrive where it should I guess.
Mind that #close most of the time does an automatic/implicit #flush.
Anyway, I don't think disabling #flush is a real solution.
+1
Maybe it is enough, if we remove the call to "self flush" in WriteStream>>#nextChunkPut: ?
I see that squeak does not call flush, and :) in Squeak WriteStream>>#flush is empty (!) (But I think in squeak and pharo are many differences for stream and source/changes handling)
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Jan, I find it rather hard to believe you are the only Windows users for who #flush is slow. This is such a fundamental operation that it should have been caught earlier. Remember that the Pharo code itself is cross platform identical. Anyway, that is how I feel it. Just an idea: does your source code contain non-Latin-1 characters (i.e. WideStrings), even a single one ? Maybe your name itself (the Ä). Sven
On 02 Jul 2015, at 23:39, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
Flush is, in the store benchmark I used, called from from MultiByteFileStream(WriteStream) >>#nextChunkPut: and from CompiledMethod>>#putSource:inFile:withPreamble: ... when I comment out calling flush in these two methods, it is fast like when whole content of flush method is commented out. However, If I keep these two flush calls commented, but I once call flush manually in the end of the "store" code, it is slow again (well, instead of 8-18 runs per sec it runs 40-50 times per sec, but it is far from 9000 when flush does not happen at all). It seems to me like the problem is not in how many times flush is called, but more in what actually happens when flush is called (how is the primitive primFlush: achieved on Windows).
Jan
Nicolai Hess wrote
2015-07-02 19:01 GMT+02:00 Sven Van Caekenberghe <
sven@
>:
#flush on a stream means pushing all data to the final destination, clearing buffers, doing actual network transfers.
What can happen when you disable that ?
That some data does not arrive where it should I guess.
Mind that #close most of the time does an automatic/implicit #flush.
Anyway, I don't think disabling #flush is a real solution.
+1
Maybe it is enough, if we remove the call to "self flush" in WriteStream>>#nextChunkPut: ?
I see that squeak does not call flush, and :) in Squeak WriteStream>>#flush is empty (!) (But I think in squeak and pharo are many differences for stream and source/changes handling)
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Well, Peter posted here his results on his Windows XP and it is slow on his PC, too. About those non-latin characters... lots of these benchmarks are made on compilation of ASCII characters as far as I know, without our project ever loaded. I tested whole loading also on Roassal2 package where I don't expect non-ASCII characters (any way to find out for sure?). Sven Van Caekenberghe-2 wrote
Jan,
I find it rather hard to believe you are the only Windows users for who #flush is slow. This is such a fundamental operation that it should have been caught earlier. Remember that the Pharo code itself is cross platform identical. Anyway, that is how I feel it.
Just an idea: does your source code contain non-Latin-1 characters (i.e. WideStrings), even a single one ? Maybe your name itself (the Ä).
Sven
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
On Thu, Jul 2, 2015 at 11:39 PM, Jan BlizniÄenko <bliznjan@fit.cvut.cz> wrote:
Flush is, in the store benchmark I used, called from from MultiByteFileStream(WriteStream) >>#nextChunkPut: and from CompiledMethod>>#putSource:inFile:withPreamble: ... when I comment out calling flush in these two methods, it is fast like when whole content of flush method is commented out. However, If I keep these two flush calls commented, but I once call flush manually in the end of the "store" code, it is slow again (well, instead of 8-18 runs per sec it runs 40-50 times per sec, but it is far from 9000 when flush does not happen at all). It seems to me like the problem is not in how many times flush is called, but more in what actually happens when flush is called (how is the primitive primFlush: achieved on Windows).
In pharo-vm\platforms\win32\plugins\FilePlugin\sqWin32FilePrims.c sqInt sqFileFlush(SQFile *f) { if (!sqFileValid(f)) FAIL(); /* note: ignores the return value in case of read-only access */ FlushFileBuffers(FILE_HANDLE(f)); return 1; } http://blogs.msdn.com/b/oldnewthing/archive/2010/09/09/10059575.aspx "If you perform one of these explicit flush operations, you aren't letting the disk cache do its job." I guess we do just that. More on that issue: http://stackoverflow.com/questions/18276554/windows-fsync-flushfilebuffers-p... http://winntfs.com/2012/11/29/windows-write-caching-part-2-an-overview-for-a... Side question: is write caching enabled on that drive? Phil
Jan
Nicolai Hess wrote
2015-07-02 19:01 GMT+02:00 Sven Van Caekenberghe <
sven@
>:
#flush on a stream means pushing all data to the final destination, clearing buffers, doing actual network transfers.
What can happen when you disable that ?
That some data does not arrive where it should I guess.
Mind that #close most of the time does an automatic/implicit #flush.
Anyway, I don't think disabling #flush is a real solution.
+1
Maybe it is enough, if we remove the call to "self flush" in WriteStream>>#nextChunkPut: ?
I see that squeak does not call flush, and :) in Squeak WriteStream>>#flush is empty (!) (But I think in squeak and pharo are many differences for stream and source/changes handling)
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
I tried making store benchmark based on caching settings on Windows which look like this: http://www.windows7library.com/blog/wp-content/uploads/2011/07/Write-Caching... both options checked: desktop 690 per sec, laptop 490 per sec caching checked (enabled), second one not checked (which is default setting): desktop 18 per sec, laptop 480 per sec neither checked: desktop 17 per sec, laptop 9 per sec All is still very far from more than 20 000 on linux in virtual box on the same PC. Also pretty curious how it has different influence on them. I should note that desktop HDD is much older - laptop is 3 years old, but desktop HDD is WD Blue WD6400AAKS made almost 8 years ago. However, disc monitoring utilities report that it is in perfect health (but still old, technologies improved since then I believe). Jan philippeback wrote
Side question: is write caching enabled on that drive?
Phil
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
I would check the processes running and CPU of Windows while loading the code... On Tue, Jun 30, 2015 at 11:35 AM, Stephan Eggermont <stephan@stack.nl> wrote:
On 30-06-15 14:37, Jan BlizniÄenko wrote:
Unfortunately no - all benchmarks I made with antivirus disabled.
Including the Microsoft stuff itself? (Security Essentials/Windows Defender) Does your desktop have a drive > 2TB?
Stephan
-- Mariano http://marianopeck.wordpress.com
Good idea. Results are... desktop: 664 503 569 bytecodes/sec; 837 14 077 sends/sec. Linux in VM on desktop: 716 083 916 bytecodes/sec; 74 030 622 sends/sec notebook: 1 114 861 186 bytecodes/sec; 148 958 012 sends/sec ...results fluctuate by about 10 % on all systems when tried multiple times with some pauses between. It is right that notebook is almost two times faster (but not 10 times like with compilation), because it has more recent processor (Core i5 mobile vs Athlon 64 x2). Jan Clément Bera-4 wrote
What about "1 tinyBenchmarks" ?
Just to know if the VM is slower as a whole or only compilation / source access ?
2015-06-30 0:36 GMT+02:00 Jan BlizniÄenko <
bliznjan@.cvut
>:
And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Silly question: do you have a couple of Nautilus windows open? Loading stuff generates annoucements and Nautilus updates are killing performance. Phil Le 30 juin 2015 00:36, "Jan BlizniÄenko" <bliznjan@fit.cvut.cz> a écrit :
And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
On Tue, Jun 30, 2015 at 12:03 PM, phil@highoctane.be <phil@highoctane.be> wrote:
Silly question: do you have a couple of Nautilus windows open?
Loading stuff generates annoucements and Nautilus updates are killing performance.
And previously TestRunner too (don;t know now)
Phil Le 30 juin 2015 00:36, "Jan BlizniÄenko" <bliznjan@fit.cvut.cz> a écrit :
And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529. 0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
I think we've safely established that the bottleneck is disk operations. Quoting from previous emails... store := [ method putSource: code inFile: 2 withPreamble: [:f | f cr; nextPut: $!; nextChunkPut: 'Behavior method'; cr]. ]. machine 1: Debian 64bit (laptop) store bench. 41604 per second Windows XP 32bit (virtualbox; host Debian) store bench. 286 per second Ubuntu 32bit (virtualbox; host Debian) store bench. 36604 per second machine 2: Windows 7 Ultimate 64-bit (desktop pc) store bench. 13 per second machine 3: Windows 7 Home Premium 64-bit (laptop) store bench. 454 per second all disks are HDD. On Tue, Jun 30, 2015 at 6:26 PM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
On Tue, Jun 30, 2015 at 12:03 PM, phil@highoctane.be <phil@highoctane.be> wrote:
Silly question: do you have a couple of Nautilus windows open?
Loading stuff generates annoucements and Nautilus updates are killing performance.
And previously TestRunner too (don;t know now)
Phil Le 30 juin 2015 00:36, "Jan BlizniÄenko" <bliznjan@fit.cvut.cz> a écrit :
And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529. 0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
On 30 Jun 2015, at 18:36, Peter Uhnák <i.uhnak@gmail.com> wrote:
I think we've safely established that the bottleneck is disk operations.
Let's take it one level down then, [ 'foo.txt' asFileReference in: [ :file | file writeStreamDo: [ :out | 3 timesRepeat: [ out << String loremIpsum ] ]. file ensureDelete ] ] bench => "'512.595 per second'" We could experiment with variants, calling #flush, writing by character, adding buffering, etc, but this is a start. At least this takes the source code stuff out of the equations.
Quoting from previous emails...
store := [ method putSource: code inFile: 2 withPreamble: [:f | f cr; nextPut: $!; nextChunkPut: 'Behavior method'; cr]. ].
machine 1: Debian 64bit (laptop) store bench. 41604 per second
Windows XP 32bit (virtualbox; host Debian) store bench. 286 per second
Ubuntu 32bit (virtualbox; host Debian) store bench. 36604 per second
machine 2: Windows 7 Ultimate 64-bit (desktop pc) store bench. 13 per second
machine 3: Windows 7 Home Premium 64-bit (laptop) store bench. 454 per second
all disks are HDD.
On Tue, Jun 30, 2015 at 6:26 PM, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
On Tue, Jun 30, 2015 at 12:03 PM, phil@highoctane.be <phil@highoctane.be> wrote: Silly question: do you have a couple of Nautilus windows open?
Loading stuff generates annoucements and Nautilus updates are killing performance.
And previously TestRunner too (don;t know now)
Phil
Le 30 juin 2015 00:36, "Jan BlizniÄenko" <bliznjan@fit.cvut.cz> a écrit : And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
Debian (host) "'4,855 per second'" Ubuntu (vbox) "'4,709 per second'" Win XP (vbox) "'504.099 per second'" The difference here is just one order of magnitude and not two... so maybe I should take apart CompileMethod>>putSource:inFile:withPreambule statement by statement. Is there some more convenient way to do it than wrap each statement in a block? Some way to profile a method or something. Peter On Tue, Jun 30, 2015 at 7:00 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 30 Jun 2015, at 18:36, Peter Uhnák <i.uhnak@gmail.com> wrote:
I think we've safely established that the bottleneck is disk operations.
Let's take it one level down then,
[ 'foo.txt' asFileReference in: [ :file | file writeStreamDo: [ :out | 3 timesRepeat: [ out << String loremIpsum ] ]. file ensureDelete ] ] bench
=> "'512.595 per second'"
We could experiment with variants, calling #flush, writing by character, adding buffering, etc, but this is a start. At least this takes the source code stuff out of the equations.
Quoting from previous emails...
store := [ method putSource: code inFile: 2 withPreamble: [:f | f cr; nextPut: $!; nextChunkPut: 'Behavior method'; cr]. ].
machine 1: Debian 64bit (laptop) store bench. 41604 per second
Windows XP 32bit (virtualbox; host Debian) store bench. 286 per second
Ubuntu 32bit (virtualbox; host Debian) store bench. 36604 per second
machine 2: Windows 7 Ultimate 64-bit (desktop pc) store bench. 13 per second
machine 3: Windows 7 Home Premium 64-bit (laptop) store bench. 454 per second
all disks are HDD.
On Tue, Jun 30, 2015 at 6:26 PM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
On Tue, Jun 30, 2015 at 12:03 PM, phil@highoctane.be <phil@highoctane.be> wrote: Silly question: do you have a couple of Nautilus windows open?
Loading stuff generates annoucements and Nautilus updates are killing performance.
And previously TestRunner too (don;t know now)
Phil
Le 30 juin 2015 00:36, "Jan BlizniÄenko" <bliznjan@fit.cvut.cz> a écrit : And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
On 30 Jun 2015, at 19:42, Peter Uhnák <i.uhnak@gmail.com> wrote:
Debian (host) "'4,855 per second'"
Ubuntu (vbox) "'4,709 per second'"
Win XP (vbox) "'504.099 per second'"
The difference here is just one order of magnitude and not two... so maybe I should take apart CompileMethod>>putSource:inFile:withPreambule statement by statement.
Is there some more convenient way to do it than wrap each statement in a block? Some way to profile a method or something.
Open the Time Profiler and write your expression there is how you do analysis on the whole tree. It is not perfect but should help you find the bottleneck.
Peter
On Tue, Jun 30, 2015 at 7:00 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 30 Jun 2015, at 18:36, Peter Uhnák <i.uhnak@gmail.com> wrote:
I think we've safely established that the bottleneck is disk operations.
Let's take it one level down then,
[ 'foo.txt' asFileReference in: [ :file | file writeStreamDo: [ :out | 3 timesRepeat: [ out << String loremIpsum ] ]. file ensureDelete ] ] bench
=> "'512.595 per second'"
We could experiment with variants, calling #flush, writing by character, adding buffering, etc, but this is a start. At least this takes the source code stuff out of the equations.
Quoting from previous emails...
store := [ method putSource: code inFile: 2 withPreamble: [:f | f cr; nextPut: $!; nextChunkPut: 'Behavior method'; cr]. ].
machine 1: Debian 64bit (laptop) store bench. 41604 per second
Windows XP 32bit (virtualbox; host Debian) store bench. 286 per second
Ubuntu 32bit (virtualbox; host Debian) store bench. 36604 per second
machine 2: Windows 7 Ultimate 64-bit (desktop pc) store bench. 13 per second
machine 3: Windows 7 Home Premium 64-bit (laptop) store bench. 454 per second
all disks are HDD.
On Tue, Jun 30, 2015 at 6:26 PM, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
On Tue, Jun 30, 2015 at 12:03 PM, phil@highoctane.be <phil@highoctane.be> wrote: Silly question: do you have a couple of Nautilus windows open?
Loading stuff generates annoucements and Nautilus updates are killing performance.
And previously TestRunner too (don;t know now)
Phil
Le 30 juin 2015 00:36, "Jan BlizniÄenko" <bliznjan@fit.cvut.cz> a écrit : And one another benchmark of linux in VM on that desktop PC: Roassal loading - 58 s compilations per second - avg: 262.4682, min: 257.194, max: 289.684 ...so the desktop PC is capable of better result and problem is somewhere in Windows, which I was afraid of...
I'd also like to add to previous Windows tests that with antivirus turned on everything takes more time approximately by half of original time.
Jan
Jan BlizniÄenko wrote
Desktop: 386 s. Notebook: 48 s. Linux in VM on notebook: 27 s.
Notebook: compilations per second - avg: 217.7153, min: 5.0, max: 247.258 Desktop: compilations per second - avg: 23.1337, min: 19.448, max: 28.155 Linux in VM on notebook: compilations per second - avg: 529.0066600000001, min: 5.0, max: 573.97
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
I tried Time profiler on |code method| code := 'a'. method := Object compiler source: code; requestor: nil; failBlock: [ ^nil ]; compile. method putSource: code inFile: 2 withPreamble: [:f | f cr; nextPut: $!; nextChunkPut: 'Behavior method'; cr]. desktop: 37.0% {38ms} MultiByteFileStream(WriteStream)>>nextChunkPut: 34.0% {35ms} MultiByteFileStream(WriteStream)>>cr 29.0% {30ms} InMidstOfFileinNotification class(Exception class)>>signal laptop: 60.0% {4ms} MultiByteFileStream(WriteStream)>>cr 40.0% {2ms} InMidstOfFileinNotification class(Exception class)>>signal Jan Sven Van Caekenberghe-2 wrote
On 30 Jun 2015, at 19:42, Peter Uhnák <
i.uhnak@
> wrote:
Debian (host) "'4,855 per second'"
Ubuntu (vbox) "'4,709 per second'"
Win XP (vbox) "'504.099 per second'"
The difference here is just one order of magnitude and not two... so maybe I should take apart CompileMethod>>putSource:inFile:withPreambule statement by statement.
Is there some more convenient way to do it than wrap each statement in a block? Some way to profile a method or something.
Open the Time Profiler and write your expression there is how you do analysis on the whole tree. It is not perfect but should help you find the bottleneck.
Peter
-- View this message in context: http://forum.world.st/Slow-compilation-on-one-of-my-Windows-PCs-tp4834668p48... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
On 2015/06/30 07:42 PM, Peter Uhnák wrote:
Debian (host) "'4,855 per second'"
Ubuntu (vbox) "'4,709 per second'"
Win XP (vbox) "'504.099 per second'" My my Win 7 (host) ( i7 CPU @ 2.10GHz) => 862.000 per second
Still a far cry from what the Linux boxes do. I'm watching with interest. Craig
participants (11)
-
Clément Bera -
Craig Johnson -
David T. Lewis -
Jan BlizniÄenko -
Marcus Denker -
Mariano Martinez Peck -
Nicolai Hess -
Peter Uhnák -
phil＠highoctane.be -
Stephan Eggermont -
Sven Van Caekenberghe