[Pharo-project] pharo-seed and pharo
Hi, I believe everyone agrees that one important goal in Pharo is to get to a small code/kernel/seed. During the Pharo Sprint at Bern we saw that the only reason for now having a unified image is just a temporary convenience that stems from two things: 1. Until Pharo 1.3, the core team worked in the core image, and this was frustrating because they were missing proper tools 2. After 1.3, building a pharo image out of a seed image was perceived as being too long and not fitting for the process of integration It's clear that the core team needs development tools. However, we think that 2 can be actually quite cheap if the build starts from the latest seed, and not from the original seed of 1.4 as it currently seem to happen. If this would be fast, we could go back to having one seed image (I call it seed just to avoid confusion and wrong feelings, but you can call it anyway you want) out of which the Pharo distribution (like the current image is) is built. The core team would work on this distribution, and Jenkins would do the rest. By switching the distribution build to work with the latest seed, we can probably reduce the amount of build time to a couple of minutes (at least this is what we saw during a couple of quick tests). Regarding the flow of the integrator, we saw that actually, when multiple changes need to be integrated, you can base the second change on the image that already exists locally, without needing to wait until Jenkins finishes. I would be interested in providing help. So, what do you say? Cheers, Doru -- www.tudorgirba.com "Every thing has its own flow"
On Fri, Mar 2, 2012 at 10:06 AM, Tudor Girba <tudor@tudorgirba.com> wrote:
So, what do you say?
Crazy idea: What about doing the opposite where integrators work on an image with all the tools they want and building a seed out of this image for people who need a deployment image or an image with different tools? -- Damien Cassou http://damiencassou.seasidehosting.st "Lambdas are relegated to relative obscurity until Java makes them popular by not having them." James Iry
Hi doru We already changed to build not from a snapshot (the one published on regular basis but manually by marcus) but from the latest jenkins build. Like that we do not need to load a bunch of versions. In addition benjamin showed to marcus the tools he built with camillo to support integration. I should sync with them because we would like to have more and more tools to help us avoiding manual work and that other people can review fixes and press the button ok without stress, like that sven, alain, ben, igor can integrate without stress. Now for the smaller image: Context: we need to see because we cannot simply take a small image and load the latest packages. Why? because some changes were made knowing that the system was in a given state. I hope that in the future we will always be able to load from the latest package versions but this is not as simple as that. Exp1: Now we can try and see. For example if we take the first 1.4alpha and load the list of packages listed in the latest versions of scriptLoader package. This could show some potential problems to upgrade directly to the latest versions of packages. If you want to help this would be a cool experience to do. With script loader you can simple ask scriptLoader to load a set of packages. Exp2: The second step is to see if it makes sense to manage Core Image with Metacello or if in addition to the list of packages we can load the tools manage with metacello (notice that it would mean also the packages that we will move out of current core): How much times does it take to load the packages and projects of tools that would be in Pharo (not Core) and check the time to do it. We can measure that from a version of core that we consider as a potential seed and/or with a version that would get some stuff loaded (and would correspond to the latest build). Stef On Mar 2, 2012, at 10:06 AM, Tudor Girba wrote:
Hi,
I believe everyone agrees that one important goal in Pharo is to get to a small code/kernel/seed. During the Pharo Sprint at Bern we saw that the only reason for now having a unified image is just a temporary convenience that stems from two things: 1. Until Pharo 1.3, the core team worked in the core image, and this was frustrating because they were missing proper tools 2. After 1.3, building a pharo image out of a seed image was perceived as being too long and not fitting for the process of integration
It's clear that the core team needs development tools. However, we think that 2 can be actually quite cheap if the build starts from the latest seed, and not from the original seed of 1.4 as it currently seem to happen. If this would be fast, we could go back to having one seed image (I call it seed just to avoid confusion and wrong feelings, but you can call it anyway you want) out of which the Pharo distribution (like the current image is) is built. The core team would work on this distribution, and Jenkins would do the rest. By switching the distribution build to work with the latest seed, we can probably reduce the amount of build time to a couple of minutes (at least this is what we saw during a couple of quick tests).
Regarding the flow of the integrator, we saw that actually, when multiple changes need to be integrated, you can base the second change on the image that already exists locally, without needing to wait until Jenkins finishes.
I would be interested in providing help. So, what do you say?
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
Hi Stef, On 2 Mar 2012, at 13:11, Stéphane Ducasse wrote:
Hi doru
We already changed to build not from a snapshot (the one published on regular basis but manually by marcus) but from the latest jenkins build. Like that we do not need to load a bunch of versions.
Excellent.
In addition benjamin showed to marcus the tools he built with camillo to support integration. I should sync with them because we would like to have more and more tools to help us avoiding manual work and that other people can review fixes and press the button ok without stress, like that sven, alain, ben, igor can integrate without stress.
Even better :)
Now for the smaller image: Context: we need to see because we cannot simply take a small image and load the latest packages. Why? because some changes were made knowing that the system was in a given state. I hope that in the future we will always be able to load from the latest package versions but this is not as simple as that.
I understand, but we need to start somewhere :)
Exp1: Now we can try and see. For example if we take the first 1.4alpha and load the list of packages listed in the latest versions of scriptLoader package. This could show some potential problems to upgrade directly to the latest versions of packages. If you want to help this would be a cool experience to do. With script loader you can simple ask scriptLoader to load a set of packages.
Ok. So I downloaded the first Pharo Core 1.4: https://gforge.inria.fr/frs/download.php/28660/PharoCore-1.4-14000.zip but, I still do not know what packages to load. Is there an easy way to figure these ones out (and in particular the loading order)?
Exp2: The second step is to see if it makes sense to manage Core Image with Metacello or if in addition to the list of packages we can load the tools manage with metacello (notice that it would mean also the packages that we will move out of current core): How much times does it take to load the packages and projects of tools that would be in Pharo (not Core) and check the time to do it. We can measure that from a version of core that we consider as a potential seed and/or with a version that would get some stuff loaded (and would correspond to the latest build).
Exactly. If you tell me from which image to start and the list of packages I will try to go through this experience. But, I need help because I am an outsider when it comes to this problem. Cheers, Doru
Stef
On Mar 2, 2012, at 10:06 AM, Tudor Girba wrote:
Hi,
I believe everyone agrees that one important goal in Pharo is to get to a small code/kernel/seed. During the Pharo Sprint at Bern we saw that the only reason for now having a unified image is just a temporary convenience that stems from two things: 1. Until Pharo 1.3, the core team worked in the core image, and this was frustrating because they were missing proper tools 2. After 1.3, building a pharo image out of a seed image was perceived as being too long and not fitting for the process of integration
It's clear that the core team needs development tools. However, we think that 2 can be actually quite cheap if the build starts from the latest seed, and not from the original seed of 1.4 as it currently seem to happen. If this would be fast, we could go back to having one seed image (I call it seed just to avoid confusion and wrong feelings, but you can call it anyway you want) out of which the Pharo distribution (like the current image is) is built. The core team would work on this distribution, and Jenkins would do the rest. By switching the distribution build to work with the latest seed, we can probably reduce the amount of build time to a couple of minutes (at least this is what we saw during a couple of quick tests).
Regarding the flow of the integrator, we saw that actually, when multiple changes need to be integrated, you can base the second change on the image that already exists locally, without needing to wait until Jenkins finishes.
I would be interested in providing help. So, what do you say?
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
-- www.tudorgirba.com "One cannot do more than one can do."
you can find the current list of packages in script loader latest version. Now marcus is right on the difficulties we will find :) Stef On Mar 2, 2012, at 1:11 PM, Stéphane Ducasse wrote:
Hi doru
We already changed to build not from a snapshot (the one published on regular basis but manually by marcus) but from the latest jenkins build. Like that we do not need to load a bunch of versions. In addition benjamin showed to marcus the tools he built with camillo to support integration. I should sync with them because we would like to have more and more tools to help us avoiding manual work and that other people can review fixes and press the button ok without stress, like that sven, alain, ben, igor can integrate without stress.
Now for the smaller image: Context: we need to see because we cannot simply take a small image and load the latest packages. Why? because some changes were made knowing that the system was in a given state. I hope that in the future we will always be able to load from the latest package versions but this is not as simple as that.
Exp1: Now we can try and see. For example if we take the first 1.4alpha and load the list of packages listed in the latest versions of scriptLoader package. This could show some potential problems to upgrade directly to the latest versions of packages. If you want to help this would be a cool experience to do. With script loader you can simple ask scriptLoader to load a set of packages.
Exp2: The second step is to see if it makes sense to manage Core Image with Metacello or if in addition to the list of packages we can load the tools manage with metacello (notice that it would mean also the packages that we will move out of current core): How much times does it take to load the packages and projects of tools that would be in Pharo (not Core) and check the time to do it. We can measure that from a version of core that we consider as a potential seed and/or with a version that would get some stuff loaded (and would correspond to the latest build).
Stef
On Mar 2, 2012, at 10:06 AM, Tudor Girba wrote:
Hi,
I believe everyone agrees that one important goal in Pharo is to get to a small code/kernel/seed. During the Pharo Sprint at Bern we saw that the only reason for now having a unified image is just a temporary convenience that stems from two things: 1. Until Pharo 1.3, the core team worked in the core image, and this was frustrating because they were missing proper tools 2. After 1.3, building a pharo image out of a seed image was perceived as being too long and not fitting for the process of integration
It's clear that the core team needs development tools. However, we think that 2 can be actually quite cheap if the build starts from the latest seed, and not from the original seed of 1.4 as it currently seem to happen. If this would be fast, we could go back to having one seed image (I call it seed just to avoid confusion and wrong feelings, but you can call it anyway you want) out of which the Pharo distribution (like the current image is) is built. The core team would work on this distribution, and Jenkins would do the rest. By switching the distribution build to work with the latest seed, we can probably reduce the amount of build time to a couple of minutes (at least this is what we saw during a couple of quick tests).
Regarding the flow of the integrator, we saw that actually, when multiple changes need to be integrated, you can base the second change on the image that already exists locally, without needing to wait until Jenkins finishes.
I would be interested in providing help. So, what do you say?
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
On 2012-03-02, at 10:06, Tudor Girba wrote:
Hi,
I believe everyone agrees that one important goal in Pharo is to get to a small code/kernel/seed. During the Pharo Sprint at Bern we saw that the only reason for now having a unified image is just a temporary convenience that stems from two things: 1. Until Pharo 1.3, the core team worked in the core image, and this was frustrating because they were missing proper tools 2. After 1.3, building a pharo image out of a seed image was perceived as being too long and not fitting for the process of integration
this mainly comes from the fact that MC is not particularly fast. I was recently amazed how fast simple CS loading is compared to the same with a SLICE. So I think that rebuilding a dev image from a core image might even work locally given that MC can be made 10 to 100 times faster. I recently made updates on my machines some 10 times faster by actually using the package-cache and not always reloading the versions from the remote server! So MC has quite some room for performance improvements.
It's clear that the core team needs development tools. However, we think that 2 can be actually quite cheap if the build starts from the latest seed, and not from the original seed of 1.4 as it currently seem to happen. If this would be fast, we could go back to having one seed image (I call it seed just to avoid confusion and wrong feelings, but you can call it anyway you want) out of which the Pharo distribution (like the current image is) is built. The core team would work on this distribution, and Jenkins would do the rest. By switching the distribution build to work with the latest seed, we can probably reduce the amount of build time to a couple of minutes (at least this is what we saw during a couple of quick tests).
Regarding the flow of the integrator, we saw that actually, when multiple changes need to be integrated, you can base the second change on the image that already exists locally, without needing to wait until Jenkins finishes.
I would be interested in providing help. So, what do you say?
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
Hi, On Fri, Mar 2, 2012 at 2:34 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2012-03-02, at 10:06, Tudor Girba wrote:
Hi,
I believe everyone agrees that one important goal in Pharo is to get to a small code/kernel/seed. During the Pharo Sprint at Bern we saw that the only reason for now having a unified image is just a temporary convenience that stems from two things: 1. Until Pharo 1.3, the core team worked in the core image, and this was frustrating because they were missing proper tools 2. After 1.3, building a pharo image out of a seed image was perceived as being too long and not fitting for the process of integration
this mainly comes from the fact that MC is not particularly fast. I was recently amazed how fast simple CS loading is compared to the same with a SLICE. So I think that rebuilding a dev image from a core image might even work locally given that MC can be made 10 to 100 times faster.
I recently made updates on my machines some 10 times faster by actually using the package-cache and not always reloading the versions from the remote server! So MC has quite some room for performance improvements.
That is even better. But, even without, I think that the current image can be loaded quite fast if we work on it. So, how can I find/obtain a seed image? Cheers, Doru
It's clear that the core team needs development tools. However, we think that 2 can be actually quite cheap if the build starts from the latest seed, and not from the original seed of 1.4 as it currently seem to happen. If this would be fast, we could go back to having one seed image (I call it seed just to avoid confusion and wrong feelings, but you can call it anyway you want) out of which the Pharo distribution (like the current image is) is built. The core team would work on this distribution, and Jenkins would do the rest. By switching the distribution build to work with the latest seed, we can probably reduce the amount of build time to a couple of minutes (at least this is what we saw during a couple of quick tests).
Regarding the flow of the integrator, we saw that actually, when multiple changes need to be integrated, you can base the second change on the image that already exists locally, without needing to wait until Jenkins finishes.
I would be interested in providing help. So, what do you say?
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
-- www.tudorgirba.com "Every thing has its own flow"
On 2 March 2012 13:34, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2012-03-02, at 10:06, Tudor Girba wrote:
Hi,
I believe everyone agrees that one important goal in Pharo is to get to a small code/kernel/seed. During the Pharo Sprint at Bern we saw that the only reason for now having a unified image is just a temporary convenience that stems from two things: 1. Until Pharo 1.3, the core team worked in the core image, and this was frustrating because they were missing proper tools 2. After 1.3, building a pharo image out of a seed image was perceived as being too long and not fitting for the process of integration
this mainly comes from the fact that MC is not particularly fast. I was recently amazed how fast simple CS loading is compared to the same with a SLICE. So I think that rebuilding a dev image from a core image might even work locally given that MC can be made 10 to 100 times faster.
I recently made updates on my machines some 10 times faster by actually using the package-cache and not always reloading the versions from the remote server! So MC has quite some room for performance improvements.
To be fair, that's not MC's fault. A usability fault, perhaps. When you load a package you load from a repository. Your package-cache is but one such repository. Metacello will (Dale can correct me as necessary!) check your package-cache before downloading something, if it's configured correctly in some ConfigurationOfFoo. So I guess what I'm saying is that the tools _around_ MC might need adjusting, but otherwise MC's just doing what you tell it to do! I don't want to give the impression that I have it perfect, but if you look at http://www.squeaksource.com/MetacelloRepository/ConfigurationOfSqueakCheck-f... you'll see in the #ensureMetacello how to make your configuration use the package-cache. (It's not just performance either: using the package-cache first lets you work offline, provided you have a properly populated package-cache. For me, working offline is crucial: a lot of my Smalltalk hacking happens on my train commute.) frank
It's clear that the core team needs development tools. However, we think that 2 can be actually quite cheap if the build starts from the latest seed, and not from the original seed of 1.4 as it currently seem to happen. If this would be fast, we could go back to having one seed image (I call it seed just to avoid confusion and wrong feelings, but you can call it anyway you want) out of which the Pharo distribution (like the current image is) is built. The core team would work on this distribution, and Jenkins would do the rest. By switching the distribution build to work with the latest seed, we can probably reduce the amount of build time to a couple of minutes (at least this is what we saw during a couple of quick tests).
Regarding the flow of the integrator, we saw that actually, when multiple changes need to be integrated, you can base the second change on the image that already exists locally, without needing to wait until Jenkins finishes.
I would be interested in providing help. So, what do you say?
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
participants (6)
-
Camillo Bruni -
Damien Cassou -
Frank Shearar -
Stéphane Ducasse -
Stéphane Ducasse
-
Tudor Girba