- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Nautilus memory leak/hog crashing Pharo

by Sven Van Caekenberghe

On 26 Jun 2014, at 15:17, Johan Brichau <johan(a)inceptive.be> wrote: > I will do that, but the main problem is not the loading speed. > The real problem is that the image blows up (i.e. crashes) when a browser is open because it runs out of memory. > > But yes, I will make a ticket and get some more profiling with that so we can fix it. > Right now, I have to tell my co-workers to close all browsers when they do a full code load interactively... it works but it's a bit ridiculous ;-) I disagree a bit ;-) Like Thierry said before: we all expect open browsers to react to anything that happens, so when you load a very large amount of code, lots of things change and I can imagine the internal notifications (announcements) overloading the browsers who are struggling to keep up. Now, there is probably something wrong, and things can probably be tuned, but I think this will always be slower. Like StÃ©phane said, loading a big batch of code should happen in a special way, disabling updates and firing a full reset at the end - or something along those lines. > cheers > Johan > > On 26 Jun 2014, at 09:47, Goubier Thierry <thierry.goubier(a)cea.fr> wrote: > >> Sorry to ask then, but: >> >> would it be possible to profile a configurationOf... load with and without a Nautilus open? Time spent handling announcements should be visible, and, yes, when loading code, Browsers have to be aware the code is being changed. >> >> I spent some time optimizing that for AltBrowser, and it can very significantly increase the loading time, without counting in side effects such as over-caching (and I know Nautilus does some caching, but I'm not sure I understand why). >> >> Thierry >> >> Le 25/06/2014 20:20, phil(a)highoctane.be a Ã©crit : >>> Maybe due to some Annoucement being picked up by Nautilus? >>> >>> Phil >>> >>> >>> >>> On Wed, Jun 25, 2014 at 7:57 PM, Johan Brichau <johan(a)inceptive.be >>> <mailto:johan@inceptive.be>> wrote: >>> >>> Hi, >>> >>> It seems that when even a single Nautilus system browser is open and >>> you do a load (using Metacello), there is a huge amount of objects >>> that get created and persisted in the image. >>> This even leads to the point that the image crashes when I try to >>> load our code using a ConfigurationOf. After some time, the Pharo >>> process is stuck at 100%, image size is over 500MB and the entire >>> image becomes unresponsive, finally crashing after an hour or so. >>> >>> I have not yet found which objects or why, but I just wrestled with >>> this all day to find out what was going on. I first thought that >>> Metacello was in an infinite loop but after noticing that the image >>> was so large (500MB) and that it got reduced to 20% of its size >>> after closing the browser window, I can say that Nautilus is >>> gathering garbage. It is definitely not Metacello because I can >>> trigger the same problem doing a load via Monticello only. >>> >>> When I load the ConfigurationOf without a single browser open, it >>> loads 5x faster and the image size stays reasonable. >>> >>> Is this a known issue? Any thoughts on what may be causing this? >>> >>> regards! >>> Johan >>> >>> >> >> -- >> Thierry Goubier >> CEA list >> Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s >> 91191 Gif sur Yvette Cedex >> France >> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95 > >

June 26, 2014

Re: [Pharo-dev] [Pharo4] Slots: API for Classes

by Sven Van Caekenberghe

Thanks for taking care, these things are aways more work than expected. On 26 Jun 2014, at 15:06, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > On 25 Jun 2014, at 17:15, Marcus Denker <marcus.denker(a)inria.fr> wrote: >> >>> - Opal needs to delegate code generation to the Slot. >>> >> This is now done. Opal now, in the semantic analysis phase, annotates slot accesses >> as OCSlotVariable (OCInstanceVariable we can remove later). >> >> Then when generating, it just forwards to the slot: >> >> emitStore: methodBuilder >> >> slot emitStore: methodBuilder >> > > This is now finally in the image 4.0 046 > > Marcus > >

June 26, 2014

Re: [Pharo-dev] Nautilus memory leak/hog crashing Pharo

by Johan Brichau

I will do that, but the main problem is not the loading speed. The real problem is that the image blows up (i.e. crashes) when a browser is open because it runs out of memory. But yes, I will make a ticket and get some more profiling with that so we can fix it. Right now, I have to tell my co-workers to close all browsers when they do a full code load interactively... it works but it's a bit ridiculous ;-) cheers Johan On 26 Jun 2014, at 09:47, Goubier Thierry <thierry.goubier(a)cea.fr> wrote: > Sorry to ask then, but: > > would it be possible to profile a configurationOf... load with and without a Nautilus open? Time spent handling announcements should be visible, and, yes, when loading code, Browsers have to be aware the code is being changed. > > I spent some time optimizing that for AltBrowser, and it can very significantly increase the loading time, without counting in side effects such as over-caching (and I know Nautilus does some caching, but I'm not sure I understand why). > > Thierry > > Le 25/06/2014 20:20, phil(a)highoctane.be a Ã©crit : >> Maybe due to some Annoucement being picked up by Nautilus? >> >> Phil >> >> >> >> On Wed, Jun 25, 2014 at 7:57 PM, Johan Brichau <johan(a)inceptive.be >> <mailto:johan@inceptive.be>> wrote: >> >> Hi, >> >> It seems that when even a single Nautilus system browser is open and >> you do a load (using Metacello), there is a huge amount of objects >> that get created and persisted in the image. >> This even leads to the point that the image crashes when I try to >> load our code using a ConfigurationOf. After some time, the Pharo >> process is stuck at 100%, image size is over 500MB and the entire >> image becomes unresponsive, finally crashing after an hour or so. >> >> I have not yet found which objects or why, but I just wrestled with >> this all day to find out what was going on. I first thought that >> Metacello was in an infinite loop but after noticing that the image >> was so large (500MB) and that it got reduced to 20% of its size >> after closing the browser window, I can say that Nautilus is >> gathering garbage. It is definitely not Metacello because I can >> trigger the same problem doing a load via Monticello only. >> >> When I load the ConfigurationOf without a single browser open, it >> loads 5x faster and the image size stays reasonable. >> >> Is this a known issue? Any thoughts on what may be causing this? >> >> regards! >> Johan >> >> > > -- > Thierry Goubier > CEA list > Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s > 91191 Gif sur Yvette Cedex > France > Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

June 26, 2014

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/40046 Home: https://github.com/pharo-project/pharo-core

June 26, 2014

[pharo-project/pharo-core] 888b8a: 40046

by GitHub

Branch: refs/heads/4.0 Home: https://github.com/pharo-project/pharo-core Commit: 888b8a3fd125ec2904abded3e2de62130d59deb1 https://github.com/pharo-project/pharo-core/commit/888b8a3fd125ec2904abded3… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2014-06-26 (Thu, 26 Jun 2014) Changed paths: A AST-Interpreter-Core.package/extension/InstanceVariableSlot/instance/accept_assign_inNode_.st A AST-Interpreter-Core.package/extension/InstanceVariableSlot/instance/readWith_inNode_.st A AST-Interpreter-Core.package/extension/OCSlotVariable/instance/accept_assign_inNode_.st A AST-Interpreter-Core.package/extension/OCSlotVariable/instance/readWith_inNode_.st A Kernel.package/ClassDescription.class/instance/slots/allSlots.st A Kernel.package/ClassDescription.class/instance/slots/slotNamed_.st A Kernel.package/ClassDescription.class/instance/slots/slots.st M OpalCompiler-Core.package/OCClassScope.class/instance/levels/instanceScope.st A OpalCompiler-Core.package/OCInstanceScope.class/instance/initializing/slots_.st R OpalCompiler-Core.package/OCInstanceScope.class/instance/initializing/vars_.st M OpalCompiler-Core.package/OCSemanticWarning.class/instance/accessing/compilationContext.st A OpalCompiler-Core.package/OCSlotVariable.class/README.md A OpalCompiler-Core.package/OCSlotVariable.class/definition.st A OpalCompiler-Core.package/OCSlotVariable.class/instance/accessing/name.st A OpalCompiler-Core.package/OCSlotVariable.class/instance/accessing/name_.st A OpalCompiler-Core.package/OCSlotVariable.class/instance/accessing/slot.st A OpalCompiler-Core.package/OCSlotVariable.class/instance/accessing/slot_.st A OpalCompiler-Core.package/OCSlotVariable.class/instance/emitting/emitStore_.st A OpalCompiler-Core.package/OCSlotVariable.class/instance/emitting/emitValue_.st A OpalCompiler-Core.package/OCSlotVariable.class/instance/testing/isInstance.st M OpalCompiler-Core.package/OCUndeclaredVariableWarning.class/instance/correcting/declareInstVar_.st A ScriptLoader40.package/ScriptLoader.class/instance/pharo - scripts/script46.st A ScriptLoader40.package/ScriptLoader.class/instance/pharo - updates/update40046.st M ScriptLoader40.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st A Slot.package/InstanceVariableSlot.class/instance/code generation/emitStore_.st A Slot.package/InstanceVariableSlot.class/instance/code generation/emitValue_.st M Slot.package/extension/Symbol/instance/asSlot.st A SmartSuggestions.package/extension/OCSlotVariable/instance/specialCommands.st R Traits.package/TClassDescription.class/instance/slots/allSlots.st R Traits.package/TClassDescription.class/instance/slots/slotNamed_.st R Traits.package/TClassDescription.class/instance/slots/slots.st A Traits.package/TraitDescription.class/instance/slots/allSlots.st A Traits.package/TraitDescription.class/instance/slots/slots.st Log Message: ----------- 40046 13406 Opal should delegate code generation for Slots to the Slot meta object https://pharo.fogbugz.com/f/cases/13406 http://files.pharo.org/image/40/40046.zip

June 26, 2014

Re: [Pharo-dev] evicted zombie process from run queue

by Esteban Lorenzano

you are basically creating 1000 * 100000 morphs (and complex ones). I would just do a matrix morph with just one morph and rendering the matrix in the canvas (overriding the #drawOn:) BTW, I do not think VW would work like that either: IMO, that design (pre-generating the cells) is bad and it never works. Other frameworks use a datasource design (like cocoa, swtâ¦ and basically all the serious frameworks I know about). and no, we do not have a default design for this kind of big components (a serious absence, IMO). Esteban On 26 Jun 2014, at 10:00, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: > Why not? > Do you know a better grid morph? > > I have tested several ones but there is no way to visualize big matrices in Pharo (I am avoiding to switch to VisualWorks for now). > > Cheers, > > HernÃ¡n > > > > 2014-06-26 9:51 GMT-03:00 Esteban Lorenzano <estebanlm(a)gmail.com>: > he, you are trying to open a tree with 1000 columns and 10000 rows. > that will never work, not like that. > > Esteban > > > On 26 Jun 2014, at 09:45, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: > >> Yes, sorry I forgot that. Pharo 3.0 clean image Latest update: #30850, on Windows 8 >> The VM is the one downloaded when you do >> >> $ wget -O- get.pharo.org/30+vm | bash >> >> Then you open and press F2, Display version information, CogVM 4.0.0 from May 15 2014. >> >> Script to reproduce it: >> >> Smalltalk garbageCollect. >> [ | matrix rows cols tree | >> rows := 10000. >> cols := 1000. >> matrix := Matrix rows: rows columns: cols. >> 1 to: rows do: [ : r | >> 1 to: cols do: [ : c | >> matrix at: r at: c put: SmallInteger maxVal atRandom ] ]. >> >> tree := TreeModel new. >> tree columns: ( >> (1 to: cols) collect: [ : col | >> TreeColumnModel new >> displayBlock: [ : node | (node content at: col) asString ]; >> headerLabel: col asString ]). >> tree roots: ((1 to: rows) collect: [ : nrow | matrix atRow: nrow ]). >> tree openWithSpec ] timeToRun >> >> HernÃ¡n >> >> >> >> 2014-06-26 2:48 GMT-03:00 stepharo <stepharo(a)free.fr>: >> Hi hernan >> >> Which image? >> What did you loaded? >> Can you reproduce it? >> >> Can someone explain what does it mean or why it happens? >> Is that a common scenario for you? >> >> I am measuring execution time for big matrices in CogVM 4.0.0. Image is unresponsive. >> >> Cheers, >> >> HernÃ¡n >> >> >> >> > >

June 26, 2014

Re: [Pharo-dev] [Pharo4] Slots: API for Classes

by Marcus Denker

On 25 Jun 2014, at 17:15, Marcus Denker <marcus.denker(a)inria.fr> wrote: > >> - Opal needs to delegate code generation to the Slot. >> > This is now done. Opal now, in the semantic analysis phase, annotates slot accesses > as OCSlotVariable (OCInstanceVariable we can remove later). > > Then when generating, it just forwards to the slot: > > emitStore: methodBuilder > > slot emitStore: methodBuilder > This is now finally in the image 4.0 046 Marcus

June 26, 2014

Re: [Pharo-dev] evicted zombie process from run queue

by HernÃ¡n Morales Durand

Why not? Do you know a better grid morph? I have tested several ones but there is no way to visualize big matrices in Pharo (I am avoiding to switch to VisualWorks for now). Cheers, HernÃ¡n 2014-06-26 9:51 GMT-03:00 Esteban Lorenzano <estebanlm(a)gmail.com>: > he, you are trying to open a tree with 1000 columns and 10000 rows. > that will never work, not like that. > > Esteban > > > On 26 Jun 2014, at 09:45, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> > wrote: > > Yes, sorry I forgot that. Pharo 3.0 clean image Latest update: #30850, on > Windows 8 > The VM is the one downloaded when you do > > $ wget -O- get.pharo.org/30+vm | bash > > Then you open and press F2, Display version information, CogVM 4.0.0 from > May 15 2014. > > Script to reproduce it: > > Smalltalk garbageCollect. > [ | matrix rows cols tree | > rows := 10000. > cols := 1000. > matrix := Matrix rows: rows columns: cols. > 1 to: rows do: [ : r | > 1 to: cols do: [ : c | > matrix at: r at: c put: SmallInteger maxVal atRandom ] ]. > > tree := TreeModel new. > tree columns: ( > (1 to: cols) collect: [ : col | > TreeColumnModel new > displayBlock: [ : node | (node content at: col) asString ]; > headerLabel: col asString ]). > tree roots: ((1 to: rows) collect: [ : nrow | matrix atRow: nrow ]). > tree openWithSpec ] timeToRun > > HernÃ¡n > > > > 2014-06-26 2:48 GMT-03:00 stepharo <stepharo(a)free.fr>: > >> Hi hernan >> >> Which image? >> What did you loaded? >> Can you reproduce it? >> >> Can someone explain what does it mean or why it happens? >>> Is that a common scenario for you? >>> >>> I am measuring execution time for big matrices in CogVM 4.0.0. Image is >>> unresponsive. >>> >>> Cheers, >>> >>> HernÃ¡n >>> >>> >> >> > >

June 26, 2014

Re: [Pharo-dev] evicted zombie process from run queue

by Esteban Lorenzano

he, you are trying to open a tree with 1000 columns and 10000 rows. that will never work, not like that. Esteban On 26 Jun 2014, at 09:45, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: > Yes, sorry I forgot that. Pharo 3.0 clean image Latest update: #30850, on Windows 8 > The VM is the one downloaded when you do > > $ wget -O- get.pharo.org/30+vm | bash > > Then you open and press F2, Display version information, CogVM 4.0.0 from May 15 2014. > > Script to reproduce it: > > Smalltalk garbageCollect. > [ | matrix rows cols tree | > rows := 10000. > cols := 1000. > matrix := Matrix rows: rows columns: cols. > 1 to: rows do: [ : r | > 1 to: cols do: [ : c | > matrix at: r at: c put: SmallInteger maxVal atRandom ] ]. > > tree := TreeModel new. > tree columns: ( > (1 to: cols) collect: [ : col | > TreeColumnModel new > displayBlock: [ : node | (node content at: col) asString ]; > headerLabel: col asString ]). > tree roots: ((1 to: rows) collect: [ : nrow | matrix atRow: nrow ]). > tree openWithSpec ] timeToRun > > HernÃ¡n > > > > 2014-06-26 2:48 GMT-03:00 stepharo <stepharo(a)free.fr>: > Hi hernan > > Which image? > What did you loaded? > Can you reproduce it? > > Can someone explain what does it mean or why it happens? > Is that a common scenario for you? > > I am measuring execution time for big matrices in CogVM 4.0.0. Image is unresponsive. > > Cheers, > > HernÃ¡n > > > >

June 26, 2014

Re: [Pharo-dev] Spec, groups and version-specific packages

by Roberto Minelli

Hi Christophe, Thanks for the answer. I managed to do that! Am I interpreting something wrong or the CoolBrowser-Platform itself is an empty package? Cheers, R > On Jun 26, 2014, at 2:08 PM, Christophe Demarey <christophe.demarey(a)inria.fr> wrote: > Hi, > > You can find useful information into the 'Deep into Pharo' book, chapter 9 (Managing projects with Metacello), section 9.12 (platform specific package). > > Example: > ConfigurationOfCoolBrowser>>baseline09: spec > <version: '0.9-baseline'> > spec for: #common do: [ > spec blessing: #baseline. > spec repository: 'http://www.example.com/CoolBrowser'. > spec > package: 'CoolBrowser-Core'; > package: 'CoolBrowser-Tests' with: [ spec requires: 'CoolBrowser-Core' ]; > package: 'CoolBrowser-Addons' with: [ spec requires: 'CoolBrowser-Core' ]; > package: 'CoolBrowser-AddonsTests' with: [ > spec requires: #('CoolBrowser-Addons' 'CoolBrowser-Tests' ) ]. > spec > group: 'default' with: #('CoolBrowser-Core' 'CoolBrowser-Addons' ); > group: 'Core' with: #('CoolBrowser-Core' 'CoolBrowser-Platform' ); > group: 'Extras' with: #('CoolBrowser-Addon'); > group: 'Tests' with: #('CoolBrowser-Tests' 'CoolBrowser-AddonsTests' ); > group: 'CompleteWithoutTests' with: #('Core', 'Extras' ); > group: 'CompleteWithTests' with: #('CompleteWithoutTests', 'Tests' )]. > spec for: #gemstone do: [ > spec package: 'CoolBrowser-Platform' with: 'CoolBrowser-PlatformGemstone']. > spec for: #pharo do: [ > spec package: 'CoolBrowser-Platform' with: 'CoolBrowser-PlatformPharo']. > > Le 26 juin 2014 Ã 13:43, Roberto Minelli a Ã©crit : > >> Hi, >> >> I have my project with my Configuration and I am using spec groups to load different sets of packages (e.g., development and user groups). >> >> With the porting to Pharo 4.0 I discovered that there are some functionalities specific to Pharo 3.0 and others to Pharo 4.0, for example the fact that MethodContext is no longer part of Pharo 4.0. but it has been replaced by Context. >> >> I moved such functionalities in a package, say 'MyProject-Extension30â and Pharo 4.0 specific functionalities in 'MyProject-Extension40â. >> >> Now the question is, suppose that I have a group like >> spec group: âUser' with: #(âAâ âBâ âCâ). >> >> and another one: >> spec group: âDev' with: #(âAâ âBâ âCâ âDâ âEâ). >> >> Until now, the developer (i.e., me) would do something like >> (self project version: #bleedingEdge) load: #('Devâ) >> >> and a used of my project would do >> (self project version: #bleedingEdge) load: #(âUserâ) >> >> >> >> How can I factor inside the dynamic load of the two Pharo-version specific packages? >> i.e., no matter if âUserâ or âDevâ group is loaded I need to load 'MyProject-Extension40â if I am on Pharo 4.0 and 'MyProject-Extension30â if on Pharo 3.0. >> >> Thanks for your help, >> Roberto >

June 26, 2014