- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Nautilus memory leak/hog crashing Pharo

by Goubier Thierry

I'm trying to see if I could do it in a cleaner way, and I found it! On partial updates, AltBrowser is able to cleanly delete the node morphs because it sends #noteRemovalOfAll: aCollection to MorphTreeMorph, which has the selection testing and removal code. Which means changing MorphTreeMorph>>updateContentsWithPreviouslyExpanded: aNodeList nodeList := nil. self noteRemovalOfAll: self allNodeMorphs. "<-- Changed" (self nodeList isNil or: [ self nodeList isEmpty ]) ifTrue: [ nodeList := nil. ^ self emptySelection ]. self addSubmorphsFromNodeList: self currentNodelist previouslyExpanded: aNodeList It solves the problem. However, I don't like this that much overall, because delete is never sent to the node morphs themselves, and this seems very wrong to me. Thierry Le 30/06/2014 16:56, Goubier Thierry a Ã©crit : > It would solve the selection kept while updating the full list, but it > won't solve partial updates. You would also be apparently deselecting > and reselecting when appending elements to the list. > > Thierry > > Le 30/06/2014 16:15, Nicolai Hess a Ã©crit : >> How about this change, if it rebuilds the whole list, we can savely >> remove the selection, or not? >> >> MorphTreeMorph>> >> addSubmorphsFromNodeList: aNodeList previouslyExpanded: expandedNodeList >> | morphList | >> morphList := OrderedCollection new. >> self >> addMorphsTo: morphList >> from: aNodeList >> withExpandedItems: expandedNodeList >> atLevel: 0. >> self insertNewMorphs: morphList. >> self listManager emptySelection. "<--- >> added" >> self listManager updateSelectionFromModel. >> self roots do: [:r | r updateChildrenRecursively]. >> self updateColumnMorphs >> >> >> >> >> 2014-06-30 15:35 GMT+02:00 Goubier Thierry <thierry.goubier(a)cea.fr >> <mailto:thierry.goubier@cea.fr>>: >> >> Ok, I have the test case showing the problem. >> >> | c w t | >> c := ClassTreeExample new. >> w := c openOn: Collection. >> t := c dependents last. >> t expandAll. >> t selectAll. >> c updateList. >> t listManager selectedMorphList do: [ :each | self assert: ( t >> allNodeMorphs includes: each ) ] >> >> Fail on the final assert. Switching to single selection hold the >> same error. >> >> Interestingly, reselecting cleans the selectedMorphList. >> >> | c w t | >> c := ClassTreeExample new. >> w := c openOn: Collection. >> t := c dependents last. >> t expandAll. >> t selectAll. >> c updateList. >> t setSelectedMorph: t allNodeMorphs first. >> t listManager selectedMorphList do: [ :each | self assert: ( t >> allNodeMorphs includes: each ) ] >> >> This one has no error. >> >> Changing the way Nautilus maintain the selection would work. Forcing >> selectedMorphList to be cleaned up on addition / removal in the list >> would be nice too (but possible update code can delete the node >> morphs by itself). >> >> Thierry >> >> >> Le 30/06/2014 10:15, Johan Brichau a Ã©crit : >> >> Hi Nicolai, >> >> >> Do you have a method selected in the browser? >> As the memory leak happens via the selectedMorphs instvar, it >> requires a package to be selected (and maybe even a method). >> >> Before and after the load. I do this and notice the increase. >> >> Smalltalk garbageCollect. >> MorphTreeNodeMorph allInstances size. >> >> It really only becomes problematic (i.e. out-of-mem) when you >> load large projects. Maybe MOOSE will qualify? >> >> Yes, I was astonished as well when I noticed that every update >> was rebuilding the entire set of Morphs. It's the easiest >> solution of course but it does impose a lot of overhead. >> First, I will see how to solve that leaking problem. Then I will >> take a look at the update itself. >> >> I can only spend some of my free time on this, so I will >> continue tonight. >> >> best, >> Johan >> >> On 30 Jun 2014, at 09:09, Nicolai Hess <nicolaihess(a)web.de >> <mailto:nicolaihess@web.de>> wrote: >> >> 2014-06-29 22:31 GMT+02:00 Johan Brichau <johan(a)inceptive.be >> <mailto:johan@inceptive.be>>: >> >> On 27 Jun 2014, at 14:00, Goubier Thierry >> <thierry.goubier(a)cea.fr <mailto:thierry.goubier@cea.fr>> >> wrote: >> >> It seems to depend on the Nautilus window state. If >> you've just opened it, then nothing seems to be amiss. >> If you start to select things in it, you start to see >> time spent in updateClassView, but why? >> >> >> I just found that the instances are being kept by the >> MorphTreeListManager#__selectedMorphs instvar. So, that >> observation is correct: you need to have a package selected. >> >> What seems to happen is that on every package load, the >> Nautilus package tree is updated (which means a >> PackageTreePackageNodeModel is created for each package in >> the image via #asNautilusNodeWithModel:). >> This update does not clear the selectedMorphs list and just >> this single reference seem to keep the entire package tree >> model and its Morphs in memory. >> >> There were 496 morphs in the selectedMorphs list and when I >> cleaned that, all trailing Morphs and PackageNodeModelNodes >> (and all the other garbage) could be gc'ed. >> >> On to investigating the growth of the selectedMorphs >> variable... >> >> Johan >> >> >> Yes, we had quite some bugs with this package tree update >> in the past. What I don't understand is, why >> is the whole tree removed and rebuild, maybe this is a >> common strategy in Morphic, update a list ore tree >> will always rebuild the whole morph structure? But it >> happens even if the structure is the same and >> just this little package/dirty package icon is updated. >> >> Another issue is this "selected package/package selection". >> The tree (and for example the >> category list and/or method list" as well), stores the >> selection on multiple ways, in the model >> (PackageTreeNautilus), the >> UI (PackageTreeNautilusUi) and the treelist morph. Once as a >> SelectedTreeNode, a Package from Nautilus Model and >> once as a PackgeTreeSelection/__PackageTreeTagSelection. >> I find it pretty confusing. >> >> >> BTW, I still can not reproduce this memory leak behavior. I >> tried this: >> >> [ Gofer new smalltalkhubUser: 'ObjectProfile' >> project: 'Roassal2'; >> package: 'ConfigurationOfRoassal2'; >> load. >> (Smalltalk at: #ConfigurationOfRoassal2) load ] timeToRun. >> >> with and without an open SystemBrowser. But the load time and >> memory consumption is the same. >> >> >> Nicolai >> >> >> >> >> >> >> >> -- >> Thierry Goubier >> CEA list >> Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s >> 91191 Gif sur Yvette Cedex >> France >> Phone/Fax: +33 (0) 1 69 08 32 92 >> <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95 >> >> > -- Thierry Goubier CEA list Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s 91191 Gif sur Yvette Cedex France Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

June 30, 2014

Re: [Pharo-dev] Nautilus memory leak/hog crashing Pharo

by Goubier Thierry

It would solve the selection kept while updating the full list, but it won't solve partial updates. You would also be apparently deselecting and reselecting when appending elements to the list. Thierry Le 30/06/2014 16:15, Nicolai Hess a Ã©crit : > How about this change, if it rebuilds the whole list, we can savely > remove the selection, or not? > > MorphTreeMorph>> > addSubmorphsFromNodeList: aNodeList previouslyExpanded: expandedNodeList > | morphList | > morphList := OrderedCollection new. > self > addMorphsTo: morphList > from: aNodeList > withExpandedItems: expandedNodeList > atLevel: 0. > self insertNewMorphs: morphList. > self listManager emptySelection. "<--- > added" > self listManager updateSelectionFromModel. > self roots do: [:r | r updateChildrenRecursively]. > self updateColumnMorphs > > > > > 2014-06-30 15:35 GMT+02:00 Goubier Thierry <thierry.goubier(a)cea.fr > <mailto:thierry.goubier@cea.fr>>: > > Ok, I have the test case showing the problem. > > | c w t | > c := ClassTreeExample new. > w := c openOn: Collection. > t := c dependents last. > t expandAll. > t selectAll. > c updateList. > t listManager selectedMorphList do: [ :each | self assert: ( t > allNodeMorphs includes: each ) ] > > Fail on the final assert. Switching to single selection hold the > same error. > > Interestingly, reselecting cleans the selectedMorphList. > > | c w t | > c := ClassTreeExample new. > w := c openOn: Collection. > t := c dependents last. > t expandAll. > t selectAll. > c updateList. > t setSelectedMorph: t allNodeMorphs first. > t listManager selectedMorphList do: [ :each | self assert: ( t > allNodeMorphs includes: each ) ] > > This one has no error. > > Changing the way Nautilus maintain the selection would work. Forcing > selectedMorphList to be cleaned up on addition / removal in the list > would be nice too (but possible update code can delete the node > morphs by itself). > > Thierry > > > Le 30/06/2014 10:15, Johan Brichau a Ã©crit : > > Hi Nicolai, > > > Do you have a method selected in the browser? > As the memory leak happens via the selectedMorphs instvar, it > requires a package to be selected (and maybe even a method). > > Before and after the load. I do this and notice the increase. > > Smalltalk garbageCollect. > MorphTreeNodeMorph allInstances size. > > It really only becomes problematic (i.e. out-of-mem) when you > load large projects. Maybe MOOSE will qualify? > > Yes, I was astonished as well when I noticed that every update > was rebuilding the entire set of Morphs. It's the easiest > solution of course but it does impose a lot of overhead. > First, I will see how to solve that leaking problem. Then I will > take a look at the update itself. > > I can only spend some of my free time on this, so I will > continue tonight. > > best, > Johan > > On 30 Jun 2014, at 09:09, Nicolai Hess <nicolaihess(a)web.de > <mailto:nicolaihess@web.de>> wrote: > > 2014-06-29 22:31 GMT+02:00 Johan Brichau <johan(a)inceptive.be > <mailto:johan@inceptive.be>>: > > On 27 Jun 2014, at 14:00, Goubier Thierry > <thierry.goubier(a)cea.fr <mailto:thierry.goubier@cea.fr>> wrote: > > It seems to depend on the Nautilus window state. If > you've just opened it, then nothing seems to be amiss. > If you start to select things in it, you start to see > time spent in updateClassView, but why? > > > I just found that the instances are being kept by the > MorphTreeListManager#__selectedMorphs instvar. So, that > observation is correct: you need to have a package selected. > > What seems to happen is that on every package load, the > Nautilus package tree is updated (which means a > PackageTreePackageNodeModel is created for each package in > the image via #asNautilusNodeWithModel:). > This update does not clear the selectedMorphs list and just > this single reference seem to keep the entire package tree > model and its Morphs in memory. > > There were 496 morphs in the selectedMorphs list and when I > cleaned that, all trailing Morphs and PackageNodeModelNodes > (and all the other garbage) could be gc'ed. > > On to investigating the growth of the selectedMorphs variable... > > Johan > > > Yes, we had quite some bugs with this package tree update > in the past. What I don't understand is, why > is the whole tree removed and rebuild, maybe this is a > common strategy in Morphic, update a list ore tree > will always rebuild the whole morph structure? But it > happens even if the structure is the same and > just this little package/dirty package icon is updated. > > Another issue is this "selected package/package selection". > The tree (and for example the > category list and/or method list" as well), stores the > selection on multiple ways, in the model > (PackageTreeNautilus), the > UI (PackageTreeNautilusUi) and the treelist morph. Once as a > SelectedTreeNode, a Package from Nautilus Model and > once as a PackgeTreeSelection/__PackageTreeTagSelection. > I find it pretty confusing. > > > BTW, I still can not reproduce this memory leak behavior. I > tried this: > > [ Gofer new smalltalkhubUser: 'ObjectProfile' > project: 'Roassal2'; > package: 'ConfigurationOfRoassal2'; > load. > (Smalltalk at: #ConfigurationOfRoassal2) load ] timeToRun. > > with and without an open SystemBrowser. But the load time and > memory consumption is the same. > > > Nicolai > > > > > > > > -- > Thierry Goubier > CEA list > Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s > 91191 Gif sur Yvette Cedex > France > Phone/Fax: +33 (0) 1 69 08 32 92 > <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95 > > -- Thierry Goubier CEA list Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s 91191 Gif sur Yvette Cedex France Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

June 30, 2014

Re: [Pharo-dev] Versioner

by kilon alios

I just don't see it the way you do. For me Versioner is a tool to make life a bit easier with Configurations, its not "the one ring to rule them all" , far from it. What stops anyone from implementing his own configuration tool ? On Mon, Jun 30, 2014 at 10:23 AM, Diego Lont <diego.lont(a)delware.nl> wrote: > I am afraid this mail is going to be too long, but I feel it needs to be > written. I will start with the conclusion, so you can skip the mail if you > feel it does not interest you. Please keep the discussion on the Moose > list, as there the discussion was started. > > I am very happy that versioner is being built. We need more and better > software configuration tools, and versioner will make our life easier. That > being said, I am getting a bit nervous about how versioner is released. It > should aim to improve the way people can work, and *not* to create a > single workflow for all people working on pharo projects. I hope my fears > are unfounded. > > Software configuration management (SCM) is in my opinion an undervalued > topic within the software world. One cannot find very many good books on > them, and I am afraid this lack of knowledge shows in the way people go > around with their SCM. So I would like to write down some observations of > the SCM processes in Pharo that I believe we should keep in mind. > > The main target of SCM is to keep the impact of changes as minimal as > possible. Good SCM allows all people working on a certain project, to make > their changes, without causing trouble for other project members and people > using this project. Open source projects are (in SCM terms) rather complex, > as they involve a lot of people, including some people we do not know. > > 1. One of the best things of Metacello is that it does not require you to > follow a certain workflow. It is very flexible and supports a large range > of possible workflows. Within the Pharo community we have at least 3 > different workflows: > A. For Pharo core, we have a pharo inbox, where suggested fixes for > reported bugs are put. These fixes are reviewed and, if accepted, > integrated into Pharo. This ensures that the changes in Pharo hold (most of > the time) to quite some quality standards. > B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all > suggested fixes are put in the main repository on smalltalkhub. The fixes > are, after review and tests, released by putting them in a âreleasedâ > version of the configuration. Since not all fixes should be in all > versions, and sometimes only concern a certain platform, the process for > Pharo core would not work: releasing fixes are more complex because of the > cross-platform issues. > C. For the âmooseâ projects all suggested fixes are put in the main > repository and are accepted by default. Since most projects evolve very > fast, quality assurance is done afterwards, by submitting bug fixes. > > I can say a lot about why I believe these workflows are appropriate for > their different projects, but the point I am trying to make here, is that > these workflows have come to existence because of the different demands on > them. And to stress my point about the flexibility (thank you Dale): > When there was a problem with the workflow in the cross platform projects, > I could find a solution (using releases) that Metacello supported. > > 2. Unfortunately Metacello does not have a good UI. And for large > configurations like Seaside, it is a lot of work to keep them maintained. I > dream of a tool that acts as a UI for Metacello: this tool is flexible as > Metacello, while making the simple things real easy (hitting a button and > done). Does versioner aim to do this? > > 3. I always get very nervous when someone says: They do it like this, and > that works very well for them. So we should do that too. I do not believe > that is always a good idea. The conclusion should be a question: Can we use > that too? And now I cannot find the mail about the java SCM practice, but I > do not think we can apply it to all our processes ... > > 4. Modularity is very important. Also for performing good SCM. And yes, we > lack sufficient modularity in Moose. It should be better. I.e. because of > the lack of modularity, it is sometimes hard to distinguish between where > the change needs to be put in. So there is no clear group to point out who > should accept the changes. So this is also an SCM problem. > > Cleaning up the configuration will help here very much. Thank you Steff > for the good work here. And maybe you also have a point that the people > working on Moose should have more focus on keeping things modular, because > it makes the system better maintainable in a lot of ways. > > 5. Forking things will only make SCM worse, as it adds a complexity. So I > very much hope we can come to an agreement how the process works, without > resorting to drastic measures. > > Regards, > Diego >

June 30, 2014

Re: [Pharo-dev] Nautilus memory leak/hog crashing Pharo

by Nicolai Hess

How about this change, if it rebuilds the whole list, we can savely remove the selection, or not? MorphTreeMorph>> addSubmorphsFromNodeList: aNodeList previouslyExpanded: expandedNodeList | morphList | morphList := OrderedCollection new. self addMorphsTo: morphList from: aNodeList withExpandedItems: expandedNodeList atLevel: 0. self insertNewMorphs: morphList. self listManager emptySelection. "<--- added" self listManager updateSelectionFromModel. self roots do: [:r | r updateChildrenRecursively]. self updateColumnMorphs 2014-06-30 15:35 GMT+02:00 Goubier Thierry <thierry.goubier(a)cea.fr>: > Ok, I have the test case showing the problem. > > | c w t | > c := ClassTreeExample new. > w := c openOn: Collection. > t := c dependents last. > t expandAll. > t selectAll. > c updateList. > t listManager selectedMorphList do: [ :each | self assert: ( t > allNodeMorphs includes: each ) ] > > Fail on the final assert. Switching to single selection hold the same > error. > > Interestingly, reselecting cleans the selectedMorphList. > > | c w t | > c := ClassTreeExample new. > w := c openOn: Collection. > t := c dependents last. > t expandAll. > t selectAll. > c updateList. > t setSelectedMorph: t allNodeMorphs first. > t listManager selectedMorphList do: [ :each | self assert: ( t > allNodeMorphs includes: each ) ] > > This one has no error. > > Changing the way Nautilus maintain the selection would work. Forcing > selectedMorphList to be cleaned up on addition / removal in the list would > be nice too (but possible update code can delete the node morphs by itself). > > Thierry > > > Le 30/06/2014 10:15, Johan Brichau a Ã©crit : > >> Hi Nicolai, >> >> >> Do you have a method selected in the browser? >> As the memory leak happens via the selectedMorphs instvar, it requires a >> package to be selected (and maybe even a method). >> >> Before and after the load. I do this and notice the increase. >> >> Smalltalk garbageCollect. >> MorphTreeNodeMorph allInstances size. >> >> It really only becomes problematic (i.e. out-of-mem) when you load large >> projects. Maybe MOOSE will qualify? >> >> Yes, I was astonished as well when I noticed that every update was >> rebuilding the entire set of Morphs. It's the easiest solution of course >> but it does impose a lot of overhead. >> First, I will see how to solve that leaking problem. Then I will take a >> look at the update itself. >> >> I can only spend some of my free time on this, so I will continue tonight. >> >> best, >> Johan >> >> On 30 Jun 2014, at 09:09, Nicolai Hess <nicolaihess(a)web.de> wrote: >> >> 2014-06-29 22:31 GMT+02:00 Johan Brichau <johan(a)inceptive.be>: >>> >>> On 27 Jun 2014, at 14:00, Goubier Thierry <thierry.goubier(a)cea.fr> >>> wrote: >>> >>> It seems to depend on the Nautilus window state. If you've just opened >>>> it, then nothing seems to be amiss. If you start to select things in it, >>>> you start to see time spent in updateClassView, but why? >>>> >>> >>> I just found that the instances are being kept by the >>> MorphTreeListManager#selectedMorphs instvar. So, that observation is >>> correct: you need to have a package selected. >>> >>> What seems to happen is that on every package load, the Nautilus package >>> tree is updated (which means a PackageTreePackageNodeModel is created for >>> each package in the image via #asNautilusNodeWithModel:). >>> This update does not clear the selectedMorphs list and just this single >>> reference seem to keep the entire package tree model and its Morphs in >>> memory. >>> >>> There were 496 morphs in the selectedMorphs list and when I cleaned >>> that, all trailing Morphs and PackageNodeModelNodes (and all the other >>> garbage) could be gc'ed. >>> >>> On to investigating the growth of the selectedMorphs variable... >>> >>> Johan >>> >>> >>> Yes, we had quite some bugs with this package tree update in the past. >>> What I don't understand is, why >>> is the whole tree removed and rebuild, maybe this is a common strategy >>> in Morphic, update a list ore tree >>> will always rebuild the whole morph structure? But it happens even if >>> the structure is the same and >>> just this little package/dirty package icon is updated. >>> >>> Another issue is this "selected package/package selection". The tree >>> (and for example the >>> category list and/or method list" as well), stores the selection on >>> multiple ways, in the model (PackageTreeNautilus), the >>> UI (PackageTreeNautilusUi) and the treelist morph. Once as a >>> SelectedTreeNode, a Package from Nautilus Model and >>> once as a PackgeTreeSelection/PackageTreeTagSelection. >>> I find it pretty confusing. >>> >>> >>> BTW, I still can not reproduce this memory leak behavior. I tried this: >>> >>> [ Gofer new smalltalkhubUser: 'ObjectProfile' >>> project: 'Roassal2'; >>> package: 'ConfigurationOfRoassal2'; >>> load. >>> (Smalltalk at: #ConfigurationOfRoassal2) load ] timeToRun. >>> >>> with and without an open SystemBrowser. But the load time and >>> memory consumption is the same. >>> >>> >>> Nicolai >>> >>> >>> >> >> >> >> > -- > Thierry Goubier > CEA list > Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s > 91191 Gif sur Yvette Cedex > France > Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95 > >

June 30, 2014

Re: [Pharo-dev] Versioner

by Christophe Demarey

Le 30 juin 2014 Ã 15:30, Diego Lont a Ã©crit : > Hi Christophe, > >> Hi Diego, >> >> Le 30 juin 2014 Ã 09:23, Diego Lont a Ã©crit : >> >>> I am very happy that versioner is being built. We need more and better software configuration tools, and versioner will make our life easier. That being said, I am getting a bit nervous about how versioner is released. It should aim to improve the way people can work, and not to create a single workflow for all people working on pharo projects. I hope my fears are unfounded. >> >> You're right, Versionner enforces a single workflow. We tried to do a simple tool that can fit most people and very easy to use. Versionner is far from perfect but allows to easily manage project dependencies and release for a "classical" workflow. There are also some known limitations (e.g., no support of platform-specific sections). >> Versionner does not aim that ALL people use a single workfow but tries to provide guidelines. If one needs another workflow, it is possible by using Metacello directly. If needed, we can also add features to Versionner. It may be done from feedbacks. >> To summarize, it is not "One Ring to rule them all" but rather one tool to fit more needs and make configuration management easy. > > That is exactly my point. But Stef currently tries to enforce this workflow of working to Moose. And since this workflow will not work well for Moose (because of the demands on the process), I object to this. I don't think Stef wants to impose the workflow. Stef wants to improve code modularity in Moose and avoid broken / bad configs. Moose is a big project and one solution to ease the maintenance is to use a tool (Versionner for example). Now it is clear that Versionner does not have all features to support the Moose process and I will take some time to see how to implement a smart deep release. With that, it should be enough to support Moose workflow. >>> Within the Pharo community we have at least 3 different workflows: >>> A. For Pharo core, we have a pharo inbox, where suggested fixes for reported bugs are put. These fixes are reviewed and, if accepted, integrated into Pharo. This ensures that the changes in Pharo hold (most of the time) to quite some quality standards. >> It is the same workflow that GitHub favors: you fork the code, do your changes and provide a pull request that will be accepted or not. It works quite well. >>> B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all suggested fixes are put in the main repository on smalltalkhub. The fixes are, after review and tests, released by putting them in a âreleasedâ version of the configuration. Since not all fixes should be in all versions, and sometimes only concern a certain platform, the process for Pharo core would not work: releasing fixes are more complex because of the cross-platform issues. >> I don't see your point here. What is really different except that you don't have an inbox (some kind of dedicated branch)? Instead, you publish in the same repository, creating new branches. >>> C. For the âmooseâ projects all suggested fixes are put in the main repository and are accepted by default. Since most projects evolve very fast, quality assurance is done afterwards, by submitting bug fixes. >> Here, there is no review, ok. > > Your response shows that I have not made my point well. I wanted to point out that we need different workflows because of the different demands. A workflow is more than only a tool, but starts with people building something and ends with people using something. And you point out that versioner has one workflow implemented, and therefor will not be applicable to all projects. > > A. For pharo core all projects are included in pharo. The number of external influences (VM, plugins) is very limited. It works here to have no configuration at all, since there is only 1 release-branch: the current one. And since the file server creates a version of each build, we can always go back to a specific build. Quality is very important to pharo as a lot of users depend on a good working pharo. > > B. For the âcross-platformâ projects this workflow is not sufficient, because there are much more external influences. So here we use the configurations to manage the releases. And the complexity of the configuration of Seaside shows this is a very complex thing to do. This complexity is not from Metacello. Metacello allows us to do a decent job here for the supported platforms. For the unsupported platforms there is some additional manual work involved, with all risks to this. The complexity here stems from the Software Configuration challenges Seaside faces. A simple inbox wonât do the job for us. Quality here is also very important as a lot of users depend on a good working Seaside. Also there are more demands on reproducibility, as there are quite some systems depending on Seaside that should function 24/7. > > C. For the âmooseâ projects, we have a few advantages above the cross-platform projects, and a few disadvantages: > + Moose only needs to work for pharo. So we can cut out a lot of platform specific junk. Some projects are suited for multiple platforms, but not for the amount of platform Seaside should work on. > + There are far less projects depending on Moose. Most of them are integrated, so tests will automatically be fired. And the projects that do depend on Moose usually do not have run 24/7. > - Moose depends on more projects then Seaside. To create an exact version is therefor much harder. > - A large part of the Moose bugs comes from the change rate of the projects Moose depends on. Pharo has a very drastic strategy to improve its architecture. This causes quite some maintenance work. > So that there is no review is not the point. The point is that because of these different demands we need to accept a lower quality standard for moose. This lower quality standard is maybe undesirable, but is not easily improved. Having no review is a consequence of this, not the cause. If we want to improve this quality standard, better SCM tools will not suffice. On the other hand, this lower quality standard has so far worked for Moose, as there are less people who depend on Moose. Ok, I got it. Just to let you know, we want to move the process you describe for Pharo to build it from a seed + configurations. The Pharo process will change too and we will see if our tools are enough powerful to achieve that.

June 30, 2014

Re: [Pharo-dev] Nautilus memory leak/hog crashing Pharo

by Goubier Thierry

Ok, I have the test case showing the problem. | c w t | c := ClassTreeExample new. w := c openOn: Collection. t := c dependents last. t expandAll. t selectAll. c updateList. t listManager selectedMorphList do: [ :each | self assert: ( t allNodeMorphs includes: each ) ] Fail on the final assert. Switching to single selection hold the same error. Interestingly, reselecting cleans the selectedMorphList. | c w t | c := ClassTreeExample new. w := c openOn: Collection. t := c dependents last. t expandAll. t selectAll. c updateList. t setSelectedMorph: t allNodeMorphs first. t listManager selectedMorphList do: [ :each | self assert: ( t allNodeMorphs includes: each ) ] This one has no error. Changing the way Nautilus maintain the selection would work. Forcing selectedMorphList to be cleaned up on addition / removal in the list would be nice too (but possible update code can delete the node morphs by itself). Thierry Le 30/06/2014 10:15, Johan Brichau a Ã©crit : > Hi Nicolai, > > Do you have a method selected in the browser? > As the memory leak happens via the selectedMorphs instvar, it requires a package to be selected (and maybe even a method). > > Before and after the load. I do this and notice the increase. > > Smalltalk garbageCollect. > MorphTreeNodeMorph allInstances size. > > It really only becomes problematic (i.e. out-of-mem) when you load large projects. Maybe MOOSE will qualify? > > Yes, I was astonished as well when I noticed that every update was rebuilding the entire set of Morphs. It's the easiest solution of course but it does impose a lot of overhead. > First, I will see how to solve that leaking problem. Then I will take a look at the update itself. > > I can only spend some of my free time on this, so I will continue tonight. > > best, > Johan > > On 30 Jun 2014, at 09:09, Nicolai Hess <nicolaihess(a)web.de> wrote: > >> 2014-06-29 22:31 GMT+02:00 Johan Brichau <johan(a)inceptive.be>: >> >> On 27 Jun 2014, at 14:00, Goubier Thierry <thierry.goubier(a)cea.fr> wrote: >> >>> It seems to depend on the Nautilus window state. If you've just opened it, then nothing seems to be amiss. If you start to select things in it, you start to see time spent in updateClassView, but why? >> >> I just found that the instances are being kept by the MorphTreeListManager#selectedMorphs instvar. So, that observation is correct: you need to have a package selected. >> >> What seems to happen is that on every package load, the Nautilus package tree is updated (which means a PackageTreePackageNodeModel is created for each package in the image via #asNautilusNodeWithModel:). >> This update does not clear the selectedMorphs list and just this single reference seem to keep the entire package tree model and its Morphs in memory. >> >> There were 496 morphs in the selectedMorphs list and when I cleaned that, all trailing Morphs and PackageNodeModelNodes (and all the other garbage) could be gc'ed. >> >> On to investigating the growth of the selectedMorphs variable... >> >> Johan >> >> >> Yes, we had quite some bugs with this package tree update in the past. What I don't understand is, why >> is the whole tree removed and rebuild, maybe this is a common strategy in Morphic, update a list ore tree >> will always rebuild the whole morph structure? But it happens even if the structure is the same and >> just this little package/dirty package icon is updated. >> >> Another issue is this "selected package/package selection". The tree (and for example the >> category list and/or method list" as well), stores the selection on multiple ways, in the model (PackageTreeNautilus), the >> UI (PackageTreeNautilusUi) and the treelist morph. Once as a SelectedTreeNode, a Package from Nautilus Model and >> once as a PackgeTreeSelection/PackageTreeTagSelection. >> I find it pretty confusing. >> >> >> BTW, I still can not reproduce this memory leak behavior. I tried this: >> >> [ Gofer new smalltalkhubUser: 'ObjectProfile' >> project: 'Roassal2'; >> package: 'ConfigurationOfRoassal2'; >> load. >> (Smalltalk at: #ConfigurationOfRoassal2) load ] timeToRun. >> >> with and without an open SystemBrowser. But the load time and >> memory consumption is the same. >> >> >> Nicolai >> >> > > > > -- Thierry Goubier CEA list Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s 91191 Gif sur Yvette Cedex France Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

June 30, 2014

Re: [Pharo-dev] Versioner

by Diego Lont

Hi Christophe, > Hi Diego, > > Le 30 juin 2014 Ã 09:23, Diego Lont a Ã©crit : > >> I am very happy that versioner is being built. We need more and better software configuration tools, and versioner will make our life easier. That being said, I am getting a bit nervous about how versioner is released. It should aim to improve the way people can work, and not to create a single workflow for all people working on pharo projects. I hope my fears are unfounded. > > You're right, Versionner enforces a single workflow. We tried to do a simple tool that can fit most people and very easy to use. Versionner is far from perfect but allows to easily manage project dependencies and release for a "classical" workflow. There are also some known limitations (e.g., no support of platform-specific sections). > Versionner does not aim that ALL people use a single workfow but tries to provide guidelines. If one needs another workflow, it is possible by using Metacello directly. If needed, we can also add features to Versionner. It may be done from feedbacks. > To summarize, it is not "One Ring to rule them all" but rather one tool to fit more needs and make configuration management easy. That is exactly my point. But Stef currently tries to enforce this workflow of working to Moose. And since this workflow will not work well for Moose (because of the demands on the process), I object to this. >> Within the Pharo community we have at least 3 different workflows: >> A. For Pharo core, we have a pharo inbox, where suggested fixes for reported bugs are put. These fixes are reviewed and, if accepted, integrated into Pharo. This ensures that the changes in Pharo hold (most of the time) to quite some quality standards. > It is the same workflow that GitHub favors: you fork the code, do your changes and provide a pull request that will be accepted or not. It works quite well. >> B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all suggested fixes are put in the main repository on smalltalkhub. The fixes are, after review and tests, released by putting them in a âreleasedâ version of the configuration. Since not all fixes should be in all versions, and sometimes only concern a certain platform, the process for Pharo core would not work: releasing fixes are more complex because of the cross-platform issues. > I don't see your point here. What is really different except that you don't have an inbox (some kind of dedicated branch)? Instead, you publish in the same repository, creating new branches. >> C. For the âmooseâ projects all suggested fixes are put in the main repository and are accepted by default. Since most projects evolve very fast, quality assurance is done afterwards, by submitting bug fixes. > Here, there is no review, ok. Your response shows that I have not made my point well. I wanted to point out that we need different workflows because of the different demands. A workflow is more than only a tool, but starts with people building something and ends with people using something. And you point out that versioner has one workflow implemented, and therefor will not be applicable to all projects. A. For pharo core all projects are included in pharo. The number of external influences (VM, plugins) is very limited. It works here to have no configuration at all, since there is only 1 release-branch: the current one. And since the file server creates a version of each build, we can always go back to a specific build. Quality is very important to pharo as a lot of users depend on a good working pharo. B. For the âcross-platformâ projects this workflow is not sufficient, because there are much more external influences. So here we use the configurations to manage the releases. And the complexity of the configuration of Seaside shows this is a very complex thing to do. This complexity is not from Metacello. Metacello allows us to do a decent job here for the supported platforms. For the unsupported platforms there is some additional manual work involved, with all risks to this. The complexity here stems from the Software Configuration challenges Seaside faces. A simple inbox wonât do the job for us. Quality here is also very important as a lot of users depend on a good working Seaside. Also there are more demands on reproducibility, as there are quite some systems depending on Seaside that should function 24/7. C. For the âmooseâ projects, we have a few advantages above the cross-platform projects, and a few disadvantages: + Moose only needs to work for pharo. So we can cut out a lot of platform specific junk. Some projects are suited for multiple platforms, but not for the amount of platform Seaside should work on. + There are far less projects depending on Moose. Most of them are integrated, so tests will automatically be fired. And the projects that do depend on Moose usually do not have run 24/7. - Moose depends on more projects then Seaside. To create an exact version is therefor much harder. - A large part of the Moose bugs comes from the change rate of the projects Moose depends on. Pharo has a very drastic strategy to improve its architecture. This causes quite some maintenance work. So that there is no review is not the point. The point is that because of these different demands we need to accept a lower quality standard for moose. This lower quality standard is maybe undesirable, but is not easily improved. Having no review is a consequence of this, not the cause. If we want to improve this quality standard, better SCM tools will not suffice. On the other hand, this lower quality standard has so far worked for Moose, as there are less people who depend on Moose. Regards, Diego

June 30, 2014

Re: [Pharo-dev] Versioner

by Christophe Demarey

Hi Diego, Le 30 juin 2014 Ã 09:23, Diego Lont a Ã©crit : > I am very happy that versioner is being built. We need more and better software configuration tools, and versioner will make our life easier. That being said, I am getting a bit nervous about how versioner is released. It should aim to improve the way people can work, and not to create a single workflow for all people working on pharo projects. I hope my fears are unfounded. You're right, Versionner enforces a single workflow. We tried to do a simple tool that can fit most people and very easy to use. Versionner is far from perfect but allows to easily manage project dependencies and release for a "classical" workflow. There are also some known limitations (e.g., no support of platform-specific sections). Versionner does not aim that ALL people use a single workfow but tries to provide guidelines. If one needs another workflow, it is possible by using Metacello directly. If needed, we can also add features to Versionner. It may be done from feedbacks. To summarize, it is not "One Ring to rule them all" but rather one tool to fit more needs and make configuration management easy. > Software configuration management (SCM) is in my opinion an undervalued topic within the software world. One cannot find very many good books on them, and I am afraid this lack of knowledge shows in the way people go around with their SCM. So I would like to write down some observations of the SCM processes in Pharo that I believe we should keep in mind. > > The main target of SCM is to keep the impact of changes as minimal as possible. Good SCM allows all people working on a certain project, to make their changes, without causing trouble for other project members and people using this project. Open source projects are (in SCM terms) rather complex, as they involve a lot of people, including some people we do not know. > > 1. One of the best things of Metacello is that it does not require you to follow a certain workflow. It is very flexible and supports a large range of possible workflows. True, it's really valuable. In the same time, it's difficult for people with few notions on SCM to know what to do. > Within the Pharo community we have at least 3 different workflows: > A. For Pharo core, we have a pharo inbox, where suggested fixes for reported bugs are put. These fixes are reviewed and, if accepted, integrated into Pharo. This ensures that the changes in Pharo hold (most of the time) to quite some quality standards. It is the same workflow that GitHub favors: you fork the code, do your changes and provide a pull request that will be accepted or not. It works quite well. > B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all suggested fixes are put in the main repository on smalltalkhub. The fixes are, after review and tests, released by putting them in a âreleasedâ version of the configuration. Since not all fixes should be in all versions, and sometimes only concern a certain platform, the process for Pharo core would not work: releasing fixes are more complex because of the cross-platform issues. I don't see your point here. What is really different except that you don't have an inbox (some kind of dedicated branch)? Instead, you publish in the same repository, creating new branches. > C. For the âmooseâ projects all suggested fixes are put in the main repository and are accepted by default. Since most projects evolve very fast, quality assurance is done afterwards, by submitting bug fixes. Here, there is no review, ok. > > I can say a lot about why I believe these workflows are appropriate for their different projects, but the point I am trying to make here, is that these workflows have come to existence because of the different demands on them. > And to stress my point about the flexibility (thank you Dale): > When there was a problem with the workflow in the cross platform projects, I could find a solution (using releases) that Metacello supported. I can only agree. There is not one language for everything, one process for everything, one tool for everything => they are languages, processes, tools. Some of them more adapted to a given situation. > 2. Unfortunately Metacello does not have a good UI. And for large configurations like Seaside, it is a lot of work to keep them maintained. I dream of a tool that acts as a UI for Metacello: this tool is flexible as Metacello, while making the simple things real easy (hitting a button and done). Does versioner aim to do this? Clearly not. Versionner was not thought to open all doors Metacello provides but rather provide a tool to manage easily configurations with a classical workflow. That said, Versionner is not against any improvement / feature. If there are need uncovered, we should think on how to cover them. > 3. I always get very nervous when someone says: They do it like this, and that works very well for them. So we should do that too. I do not believe that is always a good idea. The conclusion should be a question: Can we use that too? +1 > And now I cannot find the mail about the java SCM practice, but I do not think we can apply it to all our processes ... > > 4. Modularity is very important. +100 > Also for performing good SCM. And yes, we lack sufficient modularity in Moose. It should be better. I.e. because of the lack of modularity, it is sometimes hard to distinguish between where the change needs to be put in. So there is no clear group to point out who should accept the changes. So this is also an SCM problem. > > Cleaning up the configuration will help here very much. Thank you Steff for the good work here. And maybe you also have a point that the people working on Moose should have more focus on keeping things modular, because it makes the system better maintainable in a lot of ways. > > 5. Forking things will only make SCM worse, as it adds a complexity. So I very much hope we can come to an agreement how the process works, without resorting to drastic measures. To summarize, Versionner (may) lacks functionalities and there is place to enhancement. The first thing is to identify them clearly (I don't want to bloat Versionner with unused functionalities) and then to implement it. If some of you wish to join efforts, they are welcomed :) Regards, Christophe.

June 30, 2014

Re: [Pharo-dev] [ANN]: Little NeoJSON Helper

by Sven Van Caekenberghe

Hi Sean, This has been on my list for quite some time, sorry about that - the problem was also that I do/did not really have a good opinion about this, see further. On one hand, I am really happy that you produced this hack (in the good definition of the word: a neat trick to solve a problem). I am impressed by it, you mix what is available in NeoJSON with the dynamic abilities in Pharo to do something cool (much like coding in the debugger, or using available input to dynamically derive a mapping). You clearly got deep into it (carried away you said yourself) and you do understand NeoJSON very well. But on the other hand, it took me quite some time, browsing your code intensively, reading your mail again and again, to even understand what you did, how it (could) work(s), the goals, the limitations - and I wrote NeoJSON. So the problem is, IMHO, that we cannot expect this to be explainable/understandable to others. This could maybe be fixed by writing more code, maybe some kind of GUI tool, more documentation - but then the question is, is it worth it ? does this really solve a common problem ? It feels like an extension/add-on to me, not something to be part of the core of NeoJSON. I just don't know, what do you think about it in retrospect ? Sven On 01 May 2014, at 20:26, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > It is on my list ... > > On 01 May 2014, at 20:03, Sean P. DeNigris <sean(a)clipperadams.com> wrote: > >> I'd like for Sven to take a look and see if it's generally useful and >> well-designed, but sure... >> >> >> >> ----- >> Cheers, >> Sean >> -- >> View this message in context: http://forum.world.st/ANN-Little-NeoJSON-Helper-tp4756763p4757481.html >> Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com. >> >

June 30, 2014

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/40054 Home: https://github.com/pharo-project/pharo-core

June 30, 2014