March 2012 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] pharo-seed and pharo

by Marcus Denker

On Mar 3, 2012, at 12:03 PM, Tudor Girba wrote: > Hi, > > I did not manage to explain myself, so let me try again with a longer email. > > We all want to get to a small kernel. To get there, we should have a process that makes the progress possible in small increments. At this point, I would just be interested in putting in place a simple job that takes a seed image, and builds the image in which the core team works in. > > As far as I understood, the only reason why we now have a single image is that the old Core image had poor tools, and building the Distribution was long. However, now, you do not have RB in there anymore. Removing RB was the right decision, because it means it can evolve rather independently. This also means that to do your work you would benefit from another image. > > So, the simplest thing at this point, would be to start with just making the current image to be the seed and build another one with RB (and maybe Nautilus?) inside where you do your work. And then we try to see how it goes in terms your process. If it works Ok, we can start pulling out other packages. > > For example, currently, there are at least two other parts that are not required to be in the seed image: AST and Filesystem. Of course, when OPAL will arrive, the AST will have to go in the seed image, but until then, it does not have to. The same with Filesystem: at this moment, it does not have to be in. > > One thing that we need to watch for though is the process of including something in the Distribution. The old Dev image failed because the policy was one of "it looks interesting -> it should be in". Instead, the Distribution should be focused on providing a small set of extras that make development better. > > In the end, the goal is not to come up with the kernel now, but it would be to have the infrastructure and process in place that enables us to remove things from the seed image gradually, and still get the core team to have a good toolset. > > Would that be Ok? > What we need: -> One Artefact of release. *This is Pharo* (and yes, there is a small core somewhere, but that's for specialists) - The goal in that is to have everything just once. Not a "crapy X for the Core, but everyone uses Y". - No need to manage bugs for Full and Core differently - Stable. No "Dev now is different and I don't know why, nobody changed anything" (as happend with 1.3, again...) (stable in the sense: two builds of the same version should not magically be different) -> The development happens in that one release. Developers and integrators can just use that. No thinking. -> The way to commit is the same. What killed me in the Dev image approach is that it was different and difficult *in addtion* to a difficult and boring aproach for the core. (e.g. editing Metacello config by hand for every trivial change.. and breaking something every time). -> I can press "update system to latest" to update the image, no need to always download a fresh image for every trivial change. -- Marcus Denker -- http://marcusdenker.de

March 3, 2012

Re: [Pharo-project] pharo-seed and pharo

by Marcus Denker

> > > For example, currently, there are at least two other parts that are not required to be in the seed image: AST and Filesystem. FileSystem is needed. It replaced the old Filesystem in some places already. e.g. in FileList. Marcus -- Marcus Denker -- http://marcusdenker.de

March 3, 2012

Re: [Pharo-project] pharo-seed and pharo

by Tudor Girba

Hi, I did not manage to explain myself, so let me try again with a longer email. We all want to get to a small kernel. To get there, we should have a process that makes the progress possible in small increments. At this point, I would just be interested in putting in place a simple job that takes a seed image, and builds the image in which the core team works in. As far as I understood, the only reason why we now have a single image is that the old Core image had poor tools, and building the Distribution was long. However, now, you do not have RB in there anymore. Removing RB was the right decision, because it means it can evolve rather independently. This also means that to do your work you would benefit from another image. So, the simplest thing at this point, would be to start with just making the current image to be the seed and build another one with RB (and maybe Nautilus?) inside where you do your work. And then we try to see how it goes in terms your process. If it works Ok, we can start pulling out other packages. For example, currently, there are at least two other parts that are not required to be in the seed image: AST and Filesystem. Of course, when OPAL will arrive, the AST will have to go in the seed image, but until then, it does not have to. The same with Filesystem: at this moment, it does not have to be in. One thing that we need to watch for though is the process of including something in the Distribution. The old Dev image failed because the policy was one of "it looks interesting -> it should be in". Instead, the Distribution should be focused on providing a small set of extras that make development better. In the end, the goal is not to come up with the kernel now, but it would be to have the infrastructure and process in place that enables us to remove things from the seed image gradually, and still get the core team to have a good toolset. Would that be Ok? Cheers, Doru On 3 Mar 2012, at 09:43, Marcus Denker wrote: > >>>> It's not possible to turn 1.4000 into the current one by loading packages in one step. >>>> because there are lots of objects that need to be migrated... it's a living system. >>>> >>>> There was *a lot* of manual fiddling involved, even just from one step to the next. >>> >>> but can we try and see :) >>> > > > Another question: is this really the next thing to solve? > > There are two things: > > A) Building the image from a small kernel (where all of them are already at the same version: they in priciple fit together) > B) Updating an image version X to X+1, or even X to X + 15, as fast as possible, with as little data as possible. > > These are two different problems... just the result is the same... and one needs both. But B) is not "loading packages". > > Marcus > > -- > Marcus Denker -- http://marcusdenker.de > > -- www.tudorgirba.com "Being happy is a matter of choice."

March 3, 2012

Re: [Pharo-project] pharo-seed and pharo

by Frank Shearar

On 2 March 2012 23:18, Igor Stasenko <siguctua(a)gmail.com> wrote: > On 3 March 2012 01:13, Marcus Denker <marcus.denker(a)inria.fr> wrote: >> >> On Mar 2, 2012, at 11:11 PM, Marcus Denker wrote: >> >>> >>> On Mar 2, 2012, at 9:21 PM, Tudor Girba wrote: >>>> >>>> Ok. So I downloaded the first Pharo Core 1.4: >>>> https://gforge.inria.fr/frs/download.php/28660/PharoCore-1.4-14000.zip >>>> >>>> but, I still do not know what packages to load. Is there an easy way to figure these ones out (and in particular the loading order)? >> >> >> The update is done by ScriptLoaders #update* methods. >> >> e.g. one of the first ones: >> >> update14002 >> Â Â Â Â "self new update14002" >> Â Â Â Â self withUpdateLog: '- remove ToolBuilder'. >> Â Â Â Â self loadTogether: self script267 merge: false. >> Â Â Â Â (MCPackage named: 'ToolBuilder-Morphic') unload. >> Â Â Â Â (MCPackage named: 'ToolBuilder-Kernel') unload. >> Â Â Â Â self flushCaches. >> >> >> #script267 defines which MC packages are to be loaded for 14002. The list of packages slowly changes... >> (e.g. in 14003 there will be no ToolsBuilder anymore, as we removed it here). >> >> The #loadTogether:merge: shows that there is actually no load order... the changes are loaded together as >> if it would be one package. So we just have the load-order of MC to worry about, which can be wrong... >> In that case, one has split the update in two: first add the classes/methods, then change the calling code. >> If everything fails, we do have changesets where we can hard-reorder stuff. >> >> Besides that, there is the proble of object migration. E.g. we removed things from the startuplist, the specialObjectArray... >> e.g. in MorphicUIManager there is class var with the UI process. This used to be in Project, which is not there anymore... >> >> I think what one could do is to combine updates of the simple kind. E.g. if we would have atomic loading, load order >> problems would disapear. Then one could batch all updates together where there is no reflective change done in a postscript >> (which happens, but not that often). >> > well, atomic loading helps, except from cases when you deal with > external resources or VM contracts. fortunately there are not so many > of them. It's off-topic for this thread, but note that Lukas had to completely remove MC's attempts at atomic loading in LanguageBoxes. In LanguageBoxes you often - perhaps even _usually_? - have instance methods not written in Smalltalk. The environment can parse these correctly only because class-side methods say "no, the instance methods of this class are in <whatever language>, so use <this parser/compiler instead." As a result, you cannot load those instance methods until the class-side methods have been loaded. Really, LanguageBoxes' special tricks apply only to when you can't load method A before you've loaded method B. It's not so much about _atomicity_ as it is about _controllable load order_. frank >> Another thing that one would need to do is to destinguish between "hot" code that is run when updating and the other. >> One could structure the system in a way that the "active" code when updating is well known and controlled, and the non-active >> one could be done in a way that state manipulations are part of a general inialization scheme and expicit... >> >> There are lots of things to think about for the problem of updating a running system... difficult problem. Nice research area. >> > Indeed. And largely unknown, since most people living in > edit-compile-run world :) > >> Â Â Â Â Marcus >> >> >> -- >> Marcus Denker -- http://marcusdenker.de >> >> > > > > -- > Best regards, > Igor Stasenko. >

March 3, 2012

[Pharo-project] [TODO] things tagged 1.4

by Marcus Denker

Hi, Even though peope hate it, I think we just can't affort to not priorityze what we need to do... So I, evil as I am, started to remove 1.4 tags to get a feeling for what we really really need to fix and integrate... I should repeat that this un-tagging is not meant to judge on the submitter of the report, it does not mean that the issue should not be worked on at all, it just means that nobody needs to stay awake all night to fix it before we release 1.4... http://code.google.com/p/pharo/issues/list?can=2&q=Milestone%3D1.4+&colspec… 43 issues, 30 of those open, 13 already to be integrated. It does not mean that the list is complete, nor correct, nor that we really will fix all the issues for 1.4. It meant as a tool for you and me to have a manageble list of things that need attention. I personally need this, others do not. Those are invited to look at the complete list of reports: http://code.google.com/p/pharo/issues/list?can=2&q=&colspec=ID+Type+Status+… Here 446 Issues are listed *ALL* of them important to be fixed. Marcus -- Marcus Denker -- http://marcusdenker.de

March 3, 2012

Re: [Pharo-project] pharo-seed and pharo

by Marcus Denker

>>> It's not possible to turn 1.4000 into the current one by loading packages in one step. >>> because there are lots of objects that need to be migrated... it's a living system. >>> >>> There was *a lot* of manual fiddling involved, even just from one step to the next. >> >> but can we try and see :) >> Another question: is this really the next thing to solve? There are two things: A) Building the image from a small kernel (where all of them are already at the same version: they in priciple fit together) B) Updating an image version X to X+1, or even X to X + 15, as fast as possible, with as little data as possible. These are two different problems... just the result is the same... and one needs both. But B) is not "loading packages". Marcus -- Marcus Denker -- http://marcusdenker.de

March 3, 2012

Re: [Pharo-project] Looking for a package to do some statistics in Pharo

by StÃ©phane Ducasse

Thanks I was reviewer for the book of didier. Now I think that it would be good to revive the project and make it even better. I started to have a look. I will publish a configuration to start with. Stef On Mar 3, 2012, at 9:13 AM, Douglas McPherson wrote: > Indeed, you should check out DHBNumerical. I'm using it for Kalman filter development/simulation, and have previously done Galois field simulation using Bessett's DHBPolynomial. These are many included probability distributions and statistical queries built in, so quite likely has what you need. No underscores needed :) > > Doug > > On Mar 2, 2012, at 10:12 , StÃ©phane Ducasse wrote: > >> Hi guys >> >> I would like to know if there is a package doing some statistics in Pharo. >> I should check the work of didier besset but I want to know if there is more. >> >> It would be good to have such kind of package. >> >> >> Stef >> > >

March 3, 2012

Re: [Pharo-project] pharo-seed and pharo

by Marcus Denker

On Mar 3, 2012, at 9:27 AM, StÃ©phane Ducasse wrote: > > On Mar 2, 2012, at 11:11 PM, Marcus Denker wrote: > >> >> On Mar 2, 2012, at 9:21 PM, Tudor Girba wrote: >>> >>> Ok. So I downloaded the first Pharo Core 1.4: >>> https://gforge.inria.fr/frs/download.php/28660/PharoCore-1.4-14000.zip >>> >>> but, I still do not know what packages to load. Is there an easy way to figure these ones out (and in particular the loading order)? >> >> >> It's not possible to turn 1.4000 into the current one by loading packages in one step. >> because there are lots of objects that need to be migrated... it's a living system. >> >> There was *a lot* of manual fiddling involved, even just from one step to the next. > > but can we try and see :) > If you have a PhD student... it's a nice topic. If just making updates faster is the goal, we should think about doing the update on a server and recording the changes applied. The can load just those, with the Doits needed at the right spot... Marcus -- Marcus Denker -- http://marcusdenker.de

March 3, 2012

Re: [Pharo-project] pharo-seed and pharo

by StÃ©phane Ducasse

you can find the current list of packages in script loader latest version. Now marcus is right on the difficulties we will find :) Stef On Mar 2, 2012, at 1:11 PM, StÃ©phane Ducasse wrote: > Hi doru > > We already changed to build not from a snapshot (the one published on regular basis but manually by marcus) > but from the latest jenkins build. Like that we do not need to load a bunch of versions. > In addition benjamin showed to marcus the tools he built with camillo to support integration. > I should sync with them because we would like to have more and more tools to help us avoiding manual work > and that other people can review fixes and press the button ok without stress, like that sven, alain, ben, igor > can integrate without stress. > > Now for the smaller image: > Context: we need to see because we cannot simply take a small image and load the latest packages. > Why? because some changes were made knowing that the system was in a given state. > I hope that in the future we will always be able to load from the latest package versions but this is not as simple as that. > > Exp1: > Now we can try and see. For example if we take the first 1.4alpha and load the list of packages listed in the latest versions > of scriptLoader package. This could show some potential problems to upgrade directly to the latest versions of packages. > If you want to help this would be a cool experience to do. With script loader you can simple ask scriptLoader to load a set of packages. > > Exp2: > The second step is to see if it makes sense to manage Core Image with Metacello or if in addition to the list of packages > we can load the tools manage with metacello (notice that it would mean also the packages that we will move out of current core): > How much times does it take to load the packages and projects of tools that would be in Pharo (not Core) and check the time to do it. > We can measure that from a version of core that we consider as a potential seed and/or > with a version that would get some stuff loaded (and would correspond to the latest build). > > > Stef > > > On Mar 2, 2012, at 10:06 AM, Tudor Girba wrote: > >> Hi, >> >> I believe everyone agrees that one important goal in Pharo is to get >> to a small code/kernel/seed. During the Pharo Sprint at Bern we saw >> that the only reason for now having a unified image is just a >> temporary convenience that stems from two things: >> 1. Until Pharo 1.3, the core team worked in the core image, and this >> was frustrating because they were missing proper tools >> 2. After 1.3, building a pharo image out of a seed image was perceived >> as being too long and not fitting for the process of integration >> >> It's clear that the core team needs development tools. However, we >> think that 2 can be actually quite cheap if the build starts from the >> latest seed, and not from the original seed of 1.4 as it currently >> seem to happen. If this would be fast, we could go back to having one >> seed image (I call it seed just to avoid confusion and wrong feelings, >> but you can call it anyway you want) out of which the Pharo >> distribution (like the current image is) is built. The core team would >> work on this distribution, and Jenkins would do the rest. By switching >> the distribution build to work with the latest seed, we can probably >> reduce the amount of build time to a couple of minutes (at least this >> is what we saw during a couple of quick tests). >> >> Regarding the flow of the integrator, we saw that actually, when >> multiple changes need to be integrated, you can base the second change >> on the image that already exists locally, without needing to wait >> until Jenkins finishes. >> >> I would be interested in providing help. So, what do you say? >> >> Cheers, >> Doru >> >> -- >> www.tudorgirba.com >> >> "Every thing has its own flow" >> > >

March 3, 2012

Re: [Pharo-project] pharo-seed and pharo

by StÃ©phane Ducasse

On Mar 2, 2012, at 11:11 PM, Marcus Denker wrote: > > On Mar 2, 2012, at 9:21 PM, Tudor Girba wrote: >> >> Ok. So I downloaded the first Pharo Core 1.4: >> https://gforge.inria.fr/frs/download.php/28660/PharoCore-1.4-14000.zip >> >> but, I still do not know what packages to load. Is there an easy way to figure these ones out (and in particular the loading order)? > > > It's not possible to turn 1.4000 into the current one by loading packages in one step. > because there are lots of objects that need to be migrated... it's a living system. > > There was *a lot* of manual fiddling involved, even just from one step to the next. but can we try and see :) > > Marcus > > > -- > Marcus Denker -- http://marcusdenker.de > >

March 3, 2012