- Pharo-dev - lists.pharo.org

Really amazing bug (or feature) with compiling some ifTrue expressions

by Denis Kudriashov

Hi. try these expressions in playground: 10 ifTrue: [ #falseResult ]. "=>DNU #ifFalse". 10 ifTrue: [ #falseResult. ]. #unexpectedResult. "=> #unexpectedResult" 10 ifTrue: [ #falseResult asString]. #unexpectedResult. "=> DNU #ifTrue:" Same happen with ifFalse: and ifTrue:ifFalse couples. If you look at bytecode you will not see any jumps in not failing cases. Probably such code is redundant because it not produce any behaviour. It start to work properly as soon as there is any assignment or message send around. But it is really strange from first look. Also it means that special boolean objects with own ifTrue:ifFalse implementation will not receive messages in such cases. I am not sure if it is a bug

March 3, 2017

Call for action: Describe your catalog entries

by Torsten Bergmann

The following snippet allows you to use the HelpBrowser to go through the descriptions of all currently listed catalog projects: |topic sub st| topic := HelpTopic named: 'CATALOG CONTENTS'. st := STON fromString: (ZnEasy get: 'http://catalog.pharo.org/catalog/json') contents. st collect: [:each | sub := HelpTopic title: (each at: #name) contents: (each at: #description) asString. topic addSubtopic: sub. ]. HelpBrowser openOn: topic Many of them have a simple descriptions missing. So if you are the author of a project please add a good description to your ConfigurationOf... Also update your configurations for Pharo 6.0 and put it into http://smalltalkhub.com/#!/~Pharo/MetaRepoForPharo60 If you have an interesting project that is not yet in the catalog then take a few minutes to build a ConfigurationOf... package and class. Using Tools -> Versionner for that is really easy. Also not that Versionner has a context menu allowing you to generate the methods necessary for the catalog. Thx T.

March 3, 2017

Re: [Pharo-dev] Growing large images: the case of Moose models

by Cyril Ferlicot D.

Le 03/03/2017 Ã 17:22, Eliot Miranda a Ã©crit : > It is important to realize that scavenge time depends on the amount of > objects that survive, not on the size of new space. So increasing the > size of new space will only cause longer pause times when an application > is growing the heap, which is the case when Moose reads models. But if > an application is following the standard pattern of creating many > objects, most of which are collected, then a large eden should not cause > noticeably longer pause times. This is because a scavenge copies the > surviving objects from eden and past space into future space, > overflowing into old space, tracing reachable objects only. So only if > lots of objects survive does scavenging touch lots of data. If only a > few objects survive the scavenger touches (copies) only those objects. > > The VM collects times so you could do some experiments and measure the > average scavenge time for the Moose application during its growth phase > and then during its normal phase. I think you'll find that the large > new space is not an issue for normal usage. > > There is another variable that can affect pause times and that is the > number of stack pages. The roots of a scavenging collection are the > remembered table and the stack zone. So the larger the stack zone, the > more time is spent scanning the stack looking for references to objects > in new space. This is a difficult trade off. If one has lots of > Smalltalk processes in one's application with lots of context switchers > between them (this is the threading benchmark) then one wants lots of > stack pages, because otherwise a process switch may involve evicting > some contexts from a stack page in order to make room for the top > context of a newly runnable process. But the more stack pages one uses > the slower scavenging becomes. > > Cog's default used to be very high (160 pages IIRC) which was determined > at Qwaq, whose Teatime application uses lots of futures. I've reduced > the default to 50 but it is an important variable to play with. > > > However for UI applications (typically the IDE), the scavenge pauses > may become significant enough to be noticed by the programmer. Maybe > not at 12Mb, but certainly at 64Mb. > > > I doubt this very much (because of the argument above). Remember that > the scavenger is a garbage collector specifically designed to work well > with systems like Smalltalk where lots of intermediate objects are > created when computing results. Scavenging doesn't touch objects that > are reclaimed, only objects that survive. So this works well. I think > you'll find that GUI applications fit this pattern very much and so a > large new sad should not present a problem. > > > -- > _,,,^..^,,,_ > best, Eliot Awesome! I look forward finding the time to test all this :) -- Cyril Ferlicot http://www.synectique.eu 2 rue Jacques PrÃ©vert 01, 59650 Villeneuve d'ascq France

March 3, 2017

Re: [Pharo-dev] Growing large images: the case of Moose models

by Eliot Miranda

oops :-) On Fri, Mar 3, 2017 at 8:19 AM, phil(a)highoctane.be <phil(a)highoctane.be> wrote: > Eliot, > > Script? > > Best, > Phil > > On Fri, Mar 3, 2017 at 5:05 PM, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > >> Hi Phil, >> >> On Fri, Mar 3, 2017 at 5:04 AM, philippe.back(a)highoctane.be < >> philippe.back(a)gmail.com> wrote: >> >>> >>> >>> Le 3 mars 2017 13:56, "ClÃ©ment Bera" <bera.clement(a)gmail.com> a Ã©crit : >>> >>> >>> >>> On Fri, Mar 3, 2017 at 12:12 PM, Cyril Ferlicot D. < >>> cyril.ferlicot(a)gmail.com> wrote: >>> >>>> On 03/03/2017 11:56, ClÃ©ment Bera wrote: >>>> > Hello everyone, >>>> > >>>> > This morning I investigated with Vincent Blondeau a problem reported >>>> by >>>> > the Moose community a while ago: loading Moose model is slower in Spur >>>> > (Pharo 5+) than in pre-Spur (Pharo 4 and older). In general, this >>>> > problem was present for anyone growing images to a significant size. >>>> > >>>> > To investigate the problem, we loaded a 200Mb[3] Moose model on a >>>> 250Mb >>>> > image, growing the image to 450Mb. Loading such a model takes 2 >>>> minutes >>>> > in Spur and 1m30s in pre-Spur VMs. >>>> > >>>> > Using the stable Pharo VM, the analysis results were the following: >>>> > - total time spent to load the Model: 2 minutes >>>> > - time spent in full GC: 1 minute (4 fullGCs) >>>> > - time spent in scavenges[1]: 15 seconds >>>> > On the 2 minutes spent, we have 50% of the time spent in full GCs, >>>> 12.5% >>>> > in scavenges, 37.5% executing code. >>>> > >>>> > We then used the latest VM that features the new compactor (VM from >>>> > beginning of March 2017 and over). The full GC execution time went >>>> down >>>> > from 1 minute to 2 seconds. >>>> > >>>> > In addition, we increased the size of Eden[2] from 4Mb to 12Mb. Time >>>> > spent in scavenges decreased from 15 seconds to 5 seconds. >>>> > >>>> > Overall, loading the model is now taking ~50 seconds instead of 2 >>>> minutes. >>>> > >>>> > To increase Eden size, one needs to run a script similar to: >>>> > >>>> > | currentEdenSize desiredEdenSize | >>>> > currentEdenSize := Smalltalk vm parameterAt: 44. >>>> > desiredEdenSize := currentEdenSize * 4. >>>> > Smalltalk vm parameterAt: 45 put: desiredEdenSize. >>>> > >>>> > _*And then restart the image.*_ >>>> > >>>> > I hope this report can be useful for some of you. I will try to make a >>>> > blog post out of it, detailing other GC settings one can change from >>>> the >>>> > image to improve performance. >>>> > _* >>>> > *_ >>>> > Best, >>>> > >>>> > Clement >>>> > >>>> > [1] A scavenge is basically the garbage collection of only young >>>> objects >>>> > [2] Eden is basically the space where objects are initially allocated. >>>> > [3] All numbers in the report are order of magnitudes and not precise >>>> > numbers >>>> > >>>> > >>>> > >>>> >>>> Hi, >>>> >>>> This is great! We will probably try it soon on our models. >>>> >>>> Guillaume had a question also, what is the counterparty if we let the >>>> EdenSize at this size when we are not loading/exporting a MSE? >>>> >>> >>> There are 2 main counterparts: >>> - You waste a bit of memory. If you increase from 4Mb to 12Mb, you waste >>> 8Mb. >>> - The user-pauses for scavenges may be more significant. >>> >>> There are customers using 64Mb Eden in production. It improves their >>> performance, they do not care about wasting 60Mb on machine with 16Gb RAM >>> and their application does heavy computation and does not need to be that >>> responsive. >>> >>> However for UI applications (typically the IDE), the scavenge pauses may >>> become significant enough to be noticed by the programmer. Maybe not at >>> 12Mb, but certainly at 64Mb. >>> >>> For your case: >>> - Do you care that your application use some more Mb of RAM ? >>> - Do you care if your application takes a couple extra ms to answer a >>> request ? (In any case, the full GC takes much more time and also delays >>> the answer right now) >>> If you don't care, you can use larger Eden. In any case, an Eden of 12 >>> or 16 Mb should be safe. >>> >>> There are other settings that can be useful in your case. I will try to >>> write a post about it. >>> >>> >>>> Because in our case we deploy a server that might need to read some MSE. >>>> We cannot restart it with our current solution. In that case it would be >>>> good to have more info to select the best EdenSize for the server. >>>> >>>> Thank you! >>>> >>> >>> I have images that do not shrink with the new compactor and as there are >>> no leaks I am wondering if there is a way for me to diagnose why this >>> happens. I am routinely loading 100MB XML in the image but these are only >>> transient. >>> >> >> First of all run the attached shell script on your image and report back >> what the "freespace" field value us. In the old compactor this would >> usually be tens to hundreds of kilobytes. With the new compactor this >> should be zero or perhaps a few hundred bytes (and I suspect the few >> hundred bytes should only show up in Newspeak images). >> >> >>> Shrinking works with a Pharo6 image but not with a 5.0 based. Strange. >>> >> >> Look at the vm parameters for growth and shrinkage. >> >> >>> Anyone willing to pair over the internet to have a look? >>> >> >> Yes, but sometime next week. >> >> >>> It is annoying because I am providing those images to customers and an >>> ever growing thing is not really great especially with a non standard tech >>> I am trying to push forward there. >>> >>> Phil >>> >>> >>>> -- >>>> Cyril Ferlicot >>>> >>>> http://www.synectique.eu >>>> >>>> 2 rue Jacques PrÃ©vert 01, >>>> 59650 Villeneuve d'ascq France >>>> >>>> >>> >>> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot >> > > -- _,,,^..^,,,_ best, Eliot

March 3, 2017

Re: [Pharo-dev] Growing large images: the case of Moose models

by Eliot Miranda

Hi ClÃ©ment, On Fri, Mar 3, 2017 at 4:54 AM, ClÃ©ment Bera <bera.clement(a)gmail.com> wrote: > > > On Fri, Mar 3, 2017 at 12:12 PM, Cyril Ferlicot D. < > cyril.ferlicot(a)gmail.com> wrote: > >> On 03/03/2017 11:56, ClÃ©ment Bera wrote: >> > Hello everyone, >> > >> > This morning I investigated with Vincent Blondeau a problem reported by >> > the Moose community a while ago: loading Moose model is slower in Spur >> > (Pharo 5+) than in pre-Spur (Pharo 4 and older). In general, this >> > problem was present for anyone growing images to a significant size. >> > >> > To investigate the problem, we loaded a 200Mb[3] Moose model on a 250Mb >> > image, growing the image to 450Mb. Loading such a model takes 2 minutes >> > in Spur and 1m30s in pre-Spur VMs. >> > >> > Using the stable Pharo VM, the analysis results were the following: >> > - total time spent to load the Model: 2 minutes >> > - time spent in full GC: 1 minute (4 fullGCs) >> > - time spent in scavenges[1]: 15 seconds >> > On the 2 minutes spent, we have 50% of the time spent in full GCs, 12.5% >> > in scavenges, 37.5% executing code. >> > >> > We then used the latest VM that features the new compactor (VM from >> > beginning of March 2017 and over). The full GC execution time went down >> > from 1 minute to 2 seconds. >> > >> > In addition, we increased the size of Eden[2] from 4Mb to 12Mb. Time >> > spent in scavenges decreased from 15 seconds to 5 seconds. >> > >> > Overall, loading the model is now taking ~50 seconds instead of 2 >> minutes. >> > >> > To increase Eden size, one needs to run a script similar to: >> > >> > | currentEdenSize desiredEdenSize | >> > currentEdenSize := Smalltalk vm parameterAt: 44. >> > desiredEdenSize := currentEdenSize * 4. >> > Smalltalk vm parameterAt: 45 put: desiredEdenSize. >> > >> > _*And then restart the image.*_ >> > >> > I hope this report can be useful for some of you. I will try to make a >> > blog post out of it, detailing other GC settings one can change from the >> > image to improve performance. >> > _* >> > *_ >> > Best, >> > >> > Clement >> > >> > [1] A scavenge is basically the garbage collection of only young objects >> > [2] Eden is basically the space where objects are initially allocated. >> > [3] All numbers in the report are order of magnitudes and not precise >> > numbers >> > >> > >> > >> >> Hi, >> >> This is great! We will probably try it soon on our models. >> >> Guillaume had a question also, what is the counterparty if we let the >> EdenSize at this size when we are not loading/exporting a MSE? >> > > There are 2 main counterparts: > - You waste a bit of memory. If you increase from 4Mb to 12Mb, you waste > 8Mb. > - The user-pauses for scavenges may be more significant. > > There are customers using 64Mb Eden in production. It improves their > performance, they do not care about wasting 60Mb on machine with 16Gb RAM > and their application does heavy computation and does not need to be that > responsive. > It is important to realize that scavenge time depends on the amount of objects that survive, not on the size of new space. So increasing the size of new space will only cause longer pause times when an application is growing the heap, which is the case when Moose reads models. But if an application is following the standard pattern of creating many objects, most of which are collected, then a large eden should not cause noticeably longer pause times. This is because a scavenge copies the surviving objects from eden and past space into future space, overflowing into old space, tracing reachable objects only. So only if lots of objects survive does scavenging touch lots of data. If only a few objects survive the scavenger touches (copies) only those objects. The VM collects times so you could do some experiments and measure the average scavenge time for the Moose application during its growth phase and then during its normal phase. I think you'll find that the large new space is not an issue for normal usage. There is another variable that can affect pause times and that is the number of stack pages. The roots of a scavenging collection are the remembered table and the stack zone. So the larger the stack zone, the more time is spent scanning the stack looking for references to objects in new space. This is a difficult trade off. If one has lots of Smalltalk processes in one's application with lots of context switchers between them (this is the threading benchmark) then one wants lots of stack pages, because otherwise a process switch may involve evicting some contexts from a stack page in order to make room for the top context of a newly runnable process. But the more stack pages one uses the slower scavenging becomes. Cog's default used to be very high (160 pages IIRC) which was determined at Qwaq, whose Teatime application uses lots of futures. I've reduced the default to 50 but it is an important variable to play with. > However for UI applications (typically the IDE), the scavenge pauses may > become significant enough to be noticed by the programmer. Maybe not at > 12Mb, but certainly at 64Mb. > I doubt this very much (because of the argument above). Remember that the scavenger is a garbage collector specifically designed to work well with systems like Smalltalk where lots of intermediate objects are created when computing results. Scavenging doesn't touch objects that are reclaimed, only objects that survive. So this works well. I think you'll find that GUI applications fit this pattern very much and so a large new sad should not present a problem. > > For your case: > - Do you care that your application use some more Mb of RAM ? > - Do you care if your application takes a couple extra ms to answer a > request ? (In any case, the full GC takes much more time and also delays > the answer right now) > If you don't care, you can use larger Eden. In any case, an Eden of 12 or > 16 Mb should be safe. > > There are other settings that can be useful in your case. I will try to > write a post about it. > > >> Because in our case we deploy a server that might need to read some MSE. >> We cannot restart it with our current solution. In that case it would be >> good to have more info to select the best EdenSize for the server. >> >> Thank you! >> >> -- >> Cyril Ferlicot >> >> http://www.synectique.eu >> >> 2 rue Jacques PrÃ©vert 01, >> 59650 Villeneuve d'ascq France >> >> > -- _,,,^..^,,,_ best, Eliot

March 3, 2017

Re: [Pharo-dev] Growing large images: the case of Moose models

by phil＠highoctane.be

Eliot, Script? Best, Phil On Fri, Mar 3, 2017 at 5:05 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Phil, > > On Fri, Mar 3, 2017 at 5:04 AM, philippe.back(a)highoctane.be < > philippe.back(a)gmail.com> wrote: > >> >> >> Le 3 mars 2017 13:56, "ClÃ©ment Bera" <bera.clement(a)gmail.com> a Ã©crit : >> >> >> >> On Fri, Mar 3, 2017 at 12:12 PM, Cyril Ferlicot D. < >> cyril.ferlicot(a)gmail.com> wrote: >> >>> On 03/03/2017 11:56, ClÃ©ment Bera wrote: >>> > Hello everyone, >>> > >>> > This morning I investigated with Vincent Blondeau a problem reported by >>> > the Moose community a while ago: loading Moose model is slower in Spur >>> > (Pharo 5+) than in pre-Spur (Pharo 4 and older). In general, this >>> > problem was present for anyone growing images to a significant size. >>> > >>> > To investigate the problem, we loaded a 200Mb[3] Moose model on a 250Mb >>> > image, growing the image to 450Mb. Loading such a model takes 2 minutes >>> > in Spur and 1m30s in pre-Spur VMs. >>> > >>> > Using the stable Pharo VM, the analysis results were the following: >>> > - total time spent to load the Model: 2 minutes >>> > - time spent in full GC: 1 minute (4 fullGCs) >>> > - time spent in scavenges[1]: 15 seconds >>> > On the 2 minutes spent, we have 50% of the time spent in full GCs, >>> 12.5% >>> > in scavenges, 37.5% executing code. >>> > >>> > We then used the latest VM that features the new compactor (VM from >>> > beginning of March 2017 and over). The full GC execution time went down >>> > from 1 minute to 2 seconds. >>> > >>> > In addition, we increased the size of Eden[2] from 4Mb to 12Mb. Time >>> > spent in scavenges decreased from 15 seconds to 5 seconds. >>> > >>> > Overall, loading the model is now taking ~50 seconds instead of 2 >>> minutes. >>> > >>> > To increase Eden size, one needs to run a script similar to: >>> > >>> > | currentEdenSize desiredEdenSize | >>> > currentEdenSize := Smalltalk vm parameterAt: 44. >>> > desiredEdenSize := currentEdenSize * 4. >>> > Smalltalk vm parameterAt: 45 put: desiredEdenSize. >>> > >>> > _*And then restart the image.*_ >>> > >>> > I hope this report can be useful for some of you. I will try to make a >>> > blog post out of it, detailing other GC settings one can change from >>> the >>> > image to improve performance. >>> > _* >>> > *_ >>> > Best, >>> > >>> > Clement >>> > >>> > [1] A scavenge is basically the garbage collection of only young >>> objects >>> > [2] Eden is basically the space where objects are initially allocated. >>> > [3] All numbers in the report are order of magnitudes and not precise >>> > numbers >>> > >>> > >>> > >>> >>> Hi, >>> >>> This is great! We will probably try it soon on our models. >>> >>> Guillaume had a question also, what is the counterparty if we let the >>> EdenSize at this size when we are not loading/exporting a MSE? >>> >> >> There are 2 main counterparts: >> - You waste a bit of memory. If you increase from 4Mb to 12Mb, you waste >> 8Mb. >> - The user-pauses for scavenges may be more significant. >> >> There are customers using 64Mb Eden in production. It improves their >> performance, they do not care about wasting 60Mb on machine with 16Gb RAM >> and their application does heavy computation and does not need to be that >> responsive. >> >> However for UI applications (typically the IDE), the scavenge pauses may >> become significant enough to be noticed by the programmer. Maybe not at >> 12Mb, but certainly at 64Mb. >> >> For your case: >> - Do you care that your application use some more Mb of RAM ? >> - Do you care if your application takes a couple extra ms to answer a >> request ? (In any case, the full GC takes much more time and also delays >> the answer right now) >> If you don't care, you can use larger Eden. In any case, an Eden of 12 or >> 16 Mb should be safe. >> >> There are other settings that can be useful in your case. I will try to >> write a post about it. >> >> >>> Because in our case we deploy a server that might need to read some MSE. >>> We cannot restart it with our current solution. In that case it would be >>> good to have more info to select the best EdenSize for the server. >>> >>> Thank you! >>> >> >> I have images that do not shrink with the new compactor and as there are >> no leaks I am wondering if there is a way for me to diagnose why this >> happens. I am routinely loading 100MB XML in the image but these are only >> transient. >> > > First of all run the attached shell script on your image and report back > what the "freespace" field value us. In the old compactor this would > usually be tens to hundreds of kilobytes. With the new compactor this > should be zero or perhaps a few hundred bytes (and I suspect the few > hundred bytes should only show up in Newspeak images). > > >> Shrinking works with a Pharo6 image but not with a 5.0 based. Strange. >> > > Look at the vm parameters for growth and shrinkage. > > >> Anyone willing to pair over the internet to have a look? >> > > Yes, but sometime next week. > > >> It is annoying because I am providing those images to customers and an >> ever growing thing is not really great especially with a non standard tech >> I am trying to push forward there. >> >> Phil >> >> >>> -- >>> Cyril Ferlicot >>> >>> http://www.synectique.eu >>> >>> 2 rue Jacques PrÃ©vert 01, >>> 59650 Villeneuve d'ascq France >>> >>> >> >> > > > -- > _,,,^..^,,,_ > best, Eliot >

March 3, 2017

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/60427 Home: https://github.com/pharo-project/pharo-core

March 3, 2017

[pharo-project/pharo-core] 4389a1: 60427

by GitHub

Branch: refs/heads/6.0 Home: https://github.com/pharo-project/pharo-core Commit: 4389a1ae3caf1381f4d33eb97d9849a5a8ead0a0 https://github.com/pharo-project/pharo-core/commit/4389a1ae3caf1381f4d33eb9… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2017-03-03 (Fri, 03 Mar 2017) Changed paths: M Monticello.package/MCClassDefinition.class/instance/installing/createClass.st M Morphic-Widgets-FastTable.package/FTTableContainerMorph.class/instance/private/calculateStartIndexWhenShowing_.st M Morphic-Widgets-FastTable.package/FTTableMorph.class/instance/initialization/initialize.st M Morphic-Widgets-FastTable.package/FTTableMorph.class/instance/private/resizeAllSubviews.st M Morphic-Widgets-FastTable.package/FTTableMorph.class/instance/private/rowIndexToVerticalScrollBarValue_.st A Reflectivity-Tools-Tests.package/BreakpointTest.class/README.md A Reflectivity-Tools-Tests.package/BreakpointTest.class/definition.st A Reflectivity-Tools-Tests.package/BreakpointTest.class/instance/initialization/setUp.st A Reflectivity-Tools-Tests.package/BreakpointTest.class/instance/initialization/tearDown.st A Reflectivity-Tools-Tests.package/BreakpointTest.class/instance/tests/testRemoveClassWithBreakpoint.st A Reflectivity-Tools-Tests.package/BreakpointTest.class/instance/tests/testRemoveMethodWithBreakpoint.st A Reflectivity-Tools-Tests.package/BreakpointTest.class/instance/utils/newDummyClass.st A Reflectivity-Tools-Tests.package/extension/Breakpoint/class/all.st R Reflectivity-Tools.package/Breakpoint.class/class/class initialization/initialize.st A Reflectivity-Tools.package/Breakpoint.class/class/class initialization/registerInsterestToSystemAnnouncement.st A Reflectivity-Tools.package/Breakpoint.class/class/initialization/initialize.st A Reflectivity-Tools.package/Breakpoint.class/class/removing/removeBreakpointsFromMethod_.st A Reflectivity-Tools.package/Breakpoint.class/class/system announcements/handleClassRemoved_.st A Reflectivity-Tools.package/Breakpoint.class/class/system announcements/handleMethodRemoved_.st R ScriptLoader60.package/ScriptLoader.class/instance/pharo - scripts/script60426.st A ScriptLoader60.package/ScriptLoader.class/instance/pharo - scripts/script60427.st R ScriptLoader60.package/ScriptLoader.class/instance/pharo - updates/update60426.st A ScriptLoader60.package/ScriptLoader.class/instance/pharo - updates/update60427.st M ScriptLoader60.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 60427 18207 Breakpoints still held after a method/class with breakpoints is removed https://pharo.fogbugz.com/f/cases/18207 19669 FastTable jumps to the selected element when its resized https://pharo.fogbugz.com/f/cases/19669 19775 Bootstrapped image does not contain class comment of the Context class https://pharo.fogbugz.com/f/cases/19775 http://files.pharo.org/image/60/60427.zip

March 3, 2017

Re: [Pharo-dev] Growing large images: the case of Moose models

by Eliot Miranda

Hi Phil, On Fri, Mar 3, 2017 at 5:04 AM, philippe.back(a)highoctane.be < philippe.back(a)gmail.com> wrote: > > > Le 3 mars 2017 13:56, "ClÃ©ment Bera" <bera.clement(a)gmail.com> a Ã©crit : > > > > On Fri, Mar 3, 2017 at 12:12 PM, Cyril Ferlicot D. < > cyril.ferlicot(a)gmail.com> wrote: > >> On 03/03/2017 11:56, ClÃ©ment Bera wrote: >> > Hello everyone, >> > >> > This morning I investigated with Vincent Blondeau a problem reported by >> > the Moose community a while ago: loading Moose model is slower in Spur >> > (Pharo 5+) than in pre-Spur (Pharo 4 and older). In general, this >> > problem was present for anyone growing images to a significant size. >> > >> > To investigate the problem, we loaded a 200Mb[3] Moose model on a 250Mb >> > image, growing the image to 450Mb. Loading such a model takes 2 minutes >> > in Spur and 1m30s in pre-Spur VMs. >> > >> > Using the stable Pharo VM, the analysis results were the following: >> > - total time spent to load the Model: 2 minutes >> > - time spent in full GC: 1 minute (4 fullGCs) >> > - time spent in scavenges[1]: 15 seconds >> > On the 2 minutes spent, we have 50% of the time spent in full GCs, 12.5% >> > in scavenges, 37.5% executing code. >> > >> > We then used the latest VM that features the new compactor (VM from >> > beginning of March 2017 and over). The full GC execution time went down >> > from 1 minute to 2 seconds. >> > >> > In addition, we increased the size of Eden[2] from 4Mb to 12Mb. Time >> > spent in scavenges decreased from 15 seconds to 5 seconds. >> > >> > Overall, loading the model is now taking ~50 seconds instead of 2 >> minutes. >> > >> > To increase Eden size, one needs to run a script similar to: >> > >> > | currentEdenSize desiredEdenSize | >> > currentEdenSize := Smalltalk vm parameterAt: 44. >> > desiredEdenSize := currentEdenSize * 4. >> > Smalltalk vm parameterAt: 45 put: desiredEdenSize. >> > >> > _*And then restart the image.*_ >> > >> > I hope this report can be useful for some of you. I will try to make a >> > blog post out of it, detailing other GC settings one can change from the >> > image to improve performance. >> > _* >> > *_ >> > Best, >> > >> > Clement >> > >> > [1] A scavenge is basically the garbage collection of only young objects >> > [2] Eden is basically the space where objects are initially allocated. >> > [3] All numbers in the report are order of magnitudes and not precise >> > numbers >> > >> > >> > >> >> Hi, >> >> This is great! We will probably try it soon on our models. >> >> Guillaume had a question also, what is the counterparty if we let the >> EdenSize at this size when we are not loading/exporting a MSE? >> > > There are 2 main counterparts: > - You waste a bit of memory. If you increase from 4Mb to 12Mb, you waste > 8Mb. > - The user-pauses for scavenges may be more significant. > > There are customers using 64Mb Eden in production. It improves their > performance, they do not care about wasting 60Mb on machine with 16Gb RAM > and their application does heavy computation and does not need to be that > responsive. > > However for UI applications (typically the IDE), the scavenge pauses may > become significant enough to be noticed by the programmer. Maybe not at > 12Mb, but certainly at 64Mb. > > For your case: > - Do you care that your application use some more Mb of RAM ? > - Do you care if your application takes a couple extra ms to answer a > request ? (In any case, the full GC takes much more time and also delays > the answer right now) > If you don't care, you can use larger Eden. In any case, an Eden of 12 or > 16 Mb should be safe. > > There are other settings that can be useful in your case. I will try to > write a post about it. > > >> Because in our case we deploy a server that might need to read some MSE. >> We cannot restart it with our current solution. In that case it would be >> good to have more info to select the best EdenSize for the server. >> >> Thank you! >> > > I have images that do not shrink with the new compactor and as there are > no leaks I am wondering if there is a way for me to diagnose why this > happens. I am routinely loading 100MB XML in the image but these are only > transient. > First of all run the attached shell script on your image and report back what the "freespace" field value us. In the old compactor this would usually be tens to hundreds of kilobytes. With the new compactor this should be zero or perhaps a few hundred bytes (and I suspect the few hundred bytes should only show up in Newspeak images). > Shrinking works with a Pharo6 image but not with a 5.0 based. Strange. > Look at the vm parameters for growth and shrinkage. > Anyone willing to pair over the internet to have a look? > Yes, but sometime next week. > It is annoying because I am providing those images to customers and an > ever growing thing is not really great especially with a non standard tech > I am trying to push forward there. > > Phil > > >> -- >> Cyril Ferlicot >> >> http://www.synectique.eu >> >> 2 rue Jacques PrÃ©vert 01, >> 59650 Villeneuve d'ascq France >> >> > > -- _,,,^..^,,,_ best, Eliot

March 3, 2017

[pharo-project/pharo-core] 8c2236: 60426

by GitHub

Branch: refs/heads/6.0 Home: https://github.com/pharo-project/pharo-core Commit: 8c22361d29c1d1209e0c75b01446436f92be20b2 https://github.com/pharo-project/pharo-core/commit/8c22361d29c1d1209e0c75b0… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2017-03-03 (Fri, 03 Mar 2017) Changed paths: M ConfigurationOfUnifiedFFI.package/ConfigurationOfUnifiedFFI.class/instance/symbolic versions/stable_.st A ConfigurationOfUnifiedFFI.package/ConfigurationOfUnifiedFFI.class/instance/versions/v0%5F26%5F2_.st M Rubric.package/RubAbstractTextArea.class/instance/geometry/extent_.st M Rubric.package/RubFindReplaceService.class/instance/accessing/convertedFindString.st A Rubric.package/RubParagraphDecorator.class/instance/accessing/withoutDecorator.st R ScriptLoader60.package/ScriptLoader.class/instance/pharo - scripts/script60425.st A ScriptLoader60.package/ScriptLoader.class/instance/pharo - scripts/script60426.st R ScriptLoader60.package/ScriptLoader.class/instance/pharo - updates/update60425.st A ScriptLoader60.package/ScriptLoader.class/instance/pharo - updates/update60426.st M ScriptLoader60.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st M UnifiedFFI.package/FFIExternalStructure.class/class/instance creation/fromHandle_.st Log Message: ----------- 60426 19788 sometimes external structures does not initialise properly OFFSET fields before use https://pharo.fogbugz.com/f/cases/19788 19221 Rub Find And Replace can not search for "?" https://pharo.fogbugz.com/f/cases/19221 19789 CI tests often fail on Windows https://pharo.fogbugz.com/f/cases/19789 http://files.pharo.org/image/60/60426.zip

March 3, 2017