[Pharo-project] Metacello as a package management system for Pharo
Hi folks. First of all, sorry for the long email (I guess it will be long). After thinking and discussing hundred of times in the mailing lists, last week we all meet and talk about Metacello and Pharo. Dale, Stef, Marcus, Esteban, all were present. I want to write down all the ideas and solution in order to: not to forget them; and to get feedback and opinions from you. *Problems we want to face: * 1) Right now a user CANNOT know which version of a project is the stable for a specific Pharo version. For example, suppose ConfigurationOfXXX. The user cannot know that XXX 3.45 is the stable for 1.0 and that XXX 5.7.3 is the one for 1.1....etc...so users end up confused not knowing which version to download. In addition, #latestVersion is not enough, because that will answer you the latest version, but that may not be the correct one for older Pharo images. Suppose in the previous case, latestVersion may answer 5.7.3 which may not work in Pharo 1.0..... We want the user to be able to autaticallly load the stable version for each pharo version without needing to know that. 2) Have a repository for each Pharo version so that someone can easily browse the available configurations for that Pharo version, and load them. Not only one as it is now woth MetacelloRepository. 3) Be self contained. Sometimes developers removed packages of versions from their repositories. In such case, Metacello cannot do anything and the load may not work anymore. We want to be able to reproduce the load and be able to load the same 10 years after. *Solution proposed: * The proposed solution may not even involve Metacello. You may have heard about Metacello Project Loader developer by Esteban Lorenzano. See http://www.smallworks.com.ar/en/community/GoferProjectLoader The idea is rename that project to that it is a general name like GoferProjectManager or something like that, because now it will do load but also write :) I will call it GoferProjectManager in this email. The idea is that every developer of a configuration define the stable version for each Dialect version. There are two possible alternatives: a) That each Conf class implements #stableVersion: spec, which could be something like: ConfigurationOfXXX >> stableVersion: spec spec for: #Pharo1.0 do: [ ^ self version: '3.45' ]. spec for: #Pharo1.1 do: [ ^ self version: '5.7.3' ]. spec for: #Pharo1.2 do: [ self error: 'There is no stable version for Pharo 1.2' ]. spec for: #Gemstone do: [ ^ self version: '4.2' ]. ..... or something like that...the idea is that you define stable versions for each platform. The idea is that now the user could do something like this: ConfigurationOfXXX project stableVersion load but he won't do that ;) b) another approach is to specify something in each version, for example: ConfigurationOfXXX >> version345: spec <version: '3.45' imports: #('1.2-baseline')> spec for: #common do: [ spec blessing: #release. spec description: 'Blah balh...'. spec platformVersion: 'Pharo 1.0' and ConfigurationOfXXX >> version573: spec <version: '5.7.3' imports: #('1.2-baseline')> spec for: #common do: [ spec blessing: #release. spec description: 'Blah balh...'. spec platformVersion: 'Pharo 1.1' Now...any of those alternatives would solve problem 1) Now to solve 3) the idea is that GoferProjectManager could provide something like this: (Gofer project 'XXX') publishVersion: '3.45' for: 'Pharo 1.0' or then (Gofer project 'XXX') publishVersion: '5.7.3' for: 'Pharo 1.1' etc... Then this would do different things: a) Automatically take the conf class (it is important the name conventions!!) b) Commit it into the correct Pharo repository version. For example for 3.4.5 it will commit it in PharoMetacelloRepository10 and 5.7.3 to PharoMetacelloRepository11 c) Traverse all the dependencies of all needed packages and commit all of them to PharoPackagesContainer10 or PharoPackagesContainer11....etc Dale sent few weeks ago a script that does that: takes a conf, a version and a repo and commits all the files to that repo. With all this we solve 2). The idea is that for EACH pharo version we will have 2 repositories, one for the confs and another one for the packages (as a backup). The last point is that when loading we should load everything from the PharoPackagesContainer10 or PharoPackagesContainer11...but we don't want to change all our confs (for example, they are pointing to they own repos or MetacelloRepository)....so the solution is that we use GoferProjectManager to load. Example: Gofer project: 'XXX' loadStable and loadStable would be something like: self stableVersion repositoryOverrides: (self containerRepo); load The idea is that it can use #repositoriesOverride to override with the correct repository... So.....that's all. What do you think? does this make sense? Cheers Mariano
El lun, 20-09-2010 a las 22:31 +0200, Mariano Martinez Peck escribió:
Hi folks. First of all, sorry for the long email (I guess it will be long). After thinking and discussing hundred of times in the mailing lists, last week we all meet and talk about Metacello and Pharo. Dale, Stef, Marcus, Esteban, all were present. I want to write down all the ideas and solution in order to: not to forget them; and to get feedback and opinions from you.
Problems we want to face:
1) Right now a user CANNOT know which version of a project is the stable for a specific Pharo version. For example, suppose ConfigurationOfXXX. The user cannot know that XXX 3.45 is the stable for 1.0 and that XXX 5.7.3 is the one for 1.1....etc...so users end up confused not knowing which version to download. In addition, #latestVersion is not enough, because that will answer you the latest version, but that may not be the correct one for older Pharo images. Suppose in the previous case, latestVersion may answer 5.7.3 which may not work in Pharo 1.0..... We want the user to be able to autaticallly load the stable version for each pharo version without needing to know that.
2) Have a repository for each Pharo version so that someone can easily browse the available configurations for that Pharo version, and load them. Not only one as it is now woth MetacelloRepository.
3) Be self contained. Sometimes developers removed packages of versions from their repositories. In such case, Metacello cannot do anything and the load may not work anymore. We want to be able to reproduce the load and be able to load the same 10 years after.
ok
Solution proposed:
The proposed solution may not even involve Metacello. You may have heard about Metacello Project Loader developer by Esteban Lorenzano. See http://www.smallworks.com.ar/en/community/GoferProjectLoader The idea is rename that project to that it is a general name like GoferProjectManager or something like that, because now it will do load but also write :) I will call it GoferProjectManager in this email.
The idea is that every developer of a configuration define the stable version for each Dialect version. There are two possible alternatives:
a) That each Conf class implements #stableVersion: spec, which could be something like:
ConfigurationOfXXX >> stableVersion: spec spec for: #Pharo1.0 do: [ ^ self version: '3.45' ]. spec for: #Pharo1.1 do: [ ^ self version: '5.7.3' ]. spec for: #Pharo1.2 do: [ self error: 'There is no stable version for Pharo 1.2' ]. spec for: #Gemstone do: [ ^ self version: '4.2' ]. .....
This implies that the stableVersion method will be heavily modified as the software is ported to new platforms and new releases. Not a big deal. Sometimes will imply that a merge will be needed when 2 developers want to maintain the configuration. Also, I feel that disconnects the version method from the version string declaration/publication. again, not a big issue either.
or something like that...the idea is that you define stable versions for each platform.
The idea is that now the user could do something like this: ConfigurationOfXXX project stableVersion load but he won't do that ;)
b) another approach is to specify something in each version, for example:
ConfigurationOfXXX >> version345: spec <version: '3.45' imports: #('1.2-baseline')>
spec for: #common do: [ spec blessing: #release. spec description: 'Blah balh...'. spec platformVersion: 'Pharo 1.0'
and
ConfigurationOfXXX >> version573: spec <version: '5.7.3' imports: #('1.2-baseline')>
spec for: #common do: [ spec blessing: #release. spec description: 'Blah balh...'. spec platformVersion: 'Pharo 1.1'
Now...any of those alternatives would solve problem 1)
As you say they are equivalent ways to state the same information. I like this because it is only a place to modify the new version number. I personally like this, but also I can imagine that this could lead to big methods with conditionals for functionality that applies to more than one release. Maybe the first option requires less work for Dale to support this functionality.
Now to solve 3) the idea is that GoferProjectManager could provide something like this:
(Gofer project 'XXX') publishVersion: '3.45' for: 'Pharo 1.0' or then (Gofer project 'XXX') publishVersion: '5.7.3' for: 'Pharo 1.1' etc...
Then this would do different things:
a) Automatically take the conf class (it is important the name conventions!!) b) Commit it into the correct Pharo repository version. For example for 3.4.5 it will commit it in PharoMetacelloRepository10 and 5.7.3 to PharoMetacelloRepository11 c) Traverse all the dependencies of all needed packages and commit all of them to PharoPackagesContainer10 or PharoPackagesContainer11....etc Dale sent few weeks ago a script that does that: takes a conf, a version and a repo and commits all the files to that repo.
With all this we solve 2). The idea is that for EACH pharo version we will have 2 repositories, one for the confs and another one for the packages (as a backup).
I like it. Now, when traversing dependencies, wouldn't be good to also rewrite the repositories they point to so that they point to self (repository). This way the problem 3 is solved too, because the configuration already points to self and not the original repo? This maybe could be easier if Metacello could have a way to indirect repository names from the configuration so that the metacello version projects only referred a keyworkd and not the real package and then on commiting the package to the repo, it would be rewritten or overriden with the container repository? For example, instead of: spec for: #squeakCommon do: [ spec project: 'OSProcess' with: [ spec className: 'ConfigurationOfOSProcess'; loads: #('default'); file: 'ConfigurationOfOSProcess'; repository: 'http://www.squeaksource.com/MetacelloRepository' ]. spec repository: 'http://www.squeaksource.com/Magma'; package: 'WriteBarrier' with: [ spec repository: 'http://www.squeaksource.com/WriteBarrier' ]; you'll write a baseline like: spec for: #squeakCommon do: [ spec project: 'OSProcess' with: [ spec className: 'ConfigurationOfOSProcess'; loads: #('default'); file: 'ConfigurationOfOSProcess'; repository: #osProcessRepository ]. spec repository: 'http://www.squeaksource.com/Magma'; package: 'WriteBarrier' with: [ spec repository: #writeBarrierRepository ]; and will define the repositories somewhere else: ConfigurationOfXXX>>repositoryFor: aSymbol ^ repoMap at: #aSymbol "This will return the real repo" And when commiting, maybe the GoferProjectManager will create a subclass with a method overriding the repository map to ConfigurationOfXXX>>repositoryFor: aSymbol ^ 'http://www.squeaksource.com/PharoPackagesContainerXX' I haven't used repositoryOverrides so maybe this is nonsense and not necessary and can be handled cleanly and better by Metacello itself. :)
The last point is that when loading we should load everything from the PharoPackagesContainer10 or PharoPackagesContainer11...but we don't want to change all our confs (for example, they are pointing to they own repos or MetacelloRepository)....so the solution is that we use GoferProjectManager to load. Example:
Gofer project: 'XXX' loadStable
and loadStable would be something like:
self stableVersion repositoryOverrides: (self containerRepo); load
The idea is that it can use #repositoriesOverride to override with the correct repository...
So.....that's all. What do you think? does this make sense?
Cheers
Mariano
Overall I agree. The less the user has to type and know from where the packages come the better for we all. -- Miguel Cobá http://miguel.leugim.com.mx
On Mon, Sep 20, 2010 at 11:25 PM, Miguel Cobá <miguel.coba@gmail.com> wrote:
El lun, 20-09-2010 a las 22:31 +0200, Mariano Martinez Peck escribió:
Hi folks. First of all, sorry for the long email (I guess it will be long). After thinking and discussing hundred of times in the mailing lists, last week we all meet and talk about Metacello and Pharo. Dale, Stef, Marcus, Esteban, all were present. I want to write down all the ideas and solution in order to: not to forget them; and to get feedback and opinions from you.
Problems we want to face:
1) Right now a user CANNOT know which version of a project is the stable for a specific Pharo version. For example, suppose ConfigurationOfXXX. The user cannot know that XXX 3.45 is the stable for 1.0 and that XXX 5.7.3 is the one for 1.1....etc...so users end up confused not knowing which version to download. In addition, #latestVersion is not enough, because that will answer you the latest version, but that may not be the correct one for older Pharo images. Suppose in the previous case, latestVersion may answer 5.7.3 which may not work in Pharo 1.0..... We want the user to be able to autaticallly load the stable version for each pharo version without needing to know that.
2) Have a repository for each Pharo version so that someone can easily browse the available configurations for that Pharo version, and load them. Not only one as it is now woth MetacelloRepository.
3) Be self contained. Sometimes developers removed packages of versions from their repositories. In such case, Metacello cannot do anything and the load may not work anymore. We want to be able to reproduce the load and be able to load the same 10 years after.
ok
Solution proposed:
The proposed solution may not even involve Metacello. You may have heard about Metacello Project Loader developer by Esteban Lorenzano. See http://www.smallworks.com.ar/en/community/GoferProjectLoader The idea is rename that project to that it is a general name like GoferProjectManager or something like that, because now it will do load but also write :) I will call it GoferProjectManager in this email.
The idea is that every developer of a configuration define the stable version for each Dialect version. There are two possible alternatives:
a) That each Conf class implements #stableVersion: spec, which could be something like:
ConfigurationOfXXX >> stableVersion: spec spec for: #Pharo1.0 do: [ ^ self version: '3.45' ]. spec for: #Pharo1.1 do: [ ^ self version: '5.7.3' ]. spec for: #Pharo1.2 do: [ self error: 'There is no stable version for Pharo 1.2' ]. spec for: #Gemstone do: [ ^ self version: '4.2' ]. .....
This implies that the stableVersion method will be heavily modified as the software is ported to new platforms and new releases. Not a big deal. Sometimes will imply that a merge will be needed when 2 developers want to maintain the configuration.
Yes...what how many times would you have a conflic? Anyway, as you said, a simple merge solves the problem.
Also, I feel that disconnects the version method from the version string declaration/publication. again, not a big issue either.
True. That's why we have also option b)
or something like that...the idea is that you define stable versions for each platform.
The idea is that now the user could do something like this: ConfigurationOfXXX project stableVersion load but he won't do that ;)
b) another approach is to specify something in each version, for example:
ConfigurationOfXXX >> version345: spec <version: '3.45' imports: #('1.2-baseline')>
spec for: #common do: [ spec blessing: #release. spec description: 'Blah balh...'. spec platformVersion: 'Pharo 1.0'
and
ConfigurationOfXXX >> version573: spec <version: '5.7.3' imports: #('1.2-baseline')>
spec for: #common do: [ spec blessing: #release. spec description: 'Blah balh...'. spec platformVersion: 'Pharo 1.1'
Now...any of those alternatives would solve problem 1)
As you say they are equivalent ways to state the same information. I like this because it is only a place to modify the new version number. I personally like this, but also I can imagine that this could lead to big methods with conditionals for functionality that applies to more than one release.
Exactly. And this way it is a bit more complex to calculate. Suppose the user does ConfigurationOfXXX project stableVersion. There you have to check in which Pharo you are, then, check ALL selectors that are version methods, and get the selector that as in the code the pharo version you are in. But how to do that? maybe you have "Pharo 1.0" as a comment and maybe it is found? another is to run the execution over each method and then ask to the spec :) Fnally, you will have several versions for the same Pharo version. Suppose first I set 3.45 as stable for Pharo 1.0 but then I do it for 3.46 so....we have to take the last one.
Maybe the first option requires less work for Dale to support this functionality.
Yes, I think it is easier altought you may want to do the second one.
Now to solve 3) the idea is that GoferProjectManager could provide something like this:
(Gofer project 'XXX') publishVersion: '3.45' for: 'Pharo 1.0' or then (Gofer project 'XXX') publishVersion: '5.7.3' for: 'Pharo 1.1' etc...
Then this would do different things:
a) Automatically take the conf class (it is important the name conventions!!) b) Commit it into the correct Pharo repository version. For example for 3.4.5 it will commit it in PharoMetacelloRepository10 and 5.7.3 to PharoMetacelloRepository11 c) Traverse all the dependencies of all needed packages and commit all of them to PharoPackagesContainer10 or PharoPackagesContainer11....etc Dale sent few weeks ago a script that does that: takes a conf, a version and a repo and commits all the files to that repo.
With all this we solve 2). The idea is that for EACH pharo version we will have 2 repositories, one for the confs and another one for the packages (as a backup).
I like it. Now, when traversing dependencies, wouldn't be good to also rewrite the repositories they point to so that they point to self (repository). This way the problem 3 is solved too, because the configuration already points to self and not the original repo?
That's a good idea :) You mean...in order not to traverse it twice? with this we could save the call to #repositoriesOverrides. I guess it would be possible.
This maybe could be easier if Metacello could have a way to indirect repository names from the configuration so that the metacello version projects only referred a keyworkd and not the real package and then on commiting the package to the repo, it would be rewritten or overriden with the container repository?
For example, instead of:
spec for: #squeakCommon do: [ spec project: 'OSProcess' with: [ spec className: 'ConfigurationOfOSProcess'; loads: #('default'); file: 'ConfigurationOfOSProcess'; repository: 'http://www.squeaksource.com/MetacelloRepository' ]. spec repository: 'http://www.squeaksource.com/Magma'; package: 'WriteBarrier' with: [ spec repository: 'http://www.squeaksource.com/WriteBarrier' ];
you'll write a baseline like:
spec for: #squeakCommon do: [ spec project: 'OSProcess' with: [ spec className: 'ConfigurationOfOSProcess'; loads: #('default'); file: 'ConfigurationOfOSProcess'; repository: #osProcessRepository ]. spec repository: 'http://www.squeaksource.com/Magma'; package: 'WriteBarrier' with: [ spec repository: #writeBarrierRepository ];
and will define the repositories somewhere else:
ConfigurationOfXXX>>repositoryFor: aSymbol ^ repoMap at: #aSymbol "This will return the real repo"
And when commiting, maybe the GoferProjectManager will create a subclass with a method overriding the repository map to
ConfigurationOfXXX>>repositoryFor: aSymbol ^ 'http://www.squeaksource.com/PharoPackagesContainerXX'
Ok....I got the idea.
I haven't used repositoryOverrides so maybe this is nonsense and not necessary and can be handled cleanly and better by Metacello itself. :)
And yes, I think itis handle by Metacello. But, as far as I understand, you can send the "#repositoryOverrides: newRepository" to a version. That will traverse all dependencies and then for each of them it traverse it does a "#repository: newRepository" . Once this is done, you can said "load" to the version. It seems they are 2 different approaches.
The last point is that when loading we should load everything from the PharoPackagesContainer10 or PharoPackagesContainer11...but we don't want to change all our confs (for example, they are pointing to they own repos or MetacelloRepository)....so the solution is that we use GoferProjectManager to load. Example:
Gofer project: 'XXX' loadStable
and loadStable would be something like:
self stableVersion repositoryOverrides: (self containerRepo); load
The idea is that it can use #repositoriesOverride to override with the correct repository...
So.....that's all. What do you think? does this make sense?
Cheers
Mariano
Overall I agree. The less the user has to type and know from where the packages come the better for we all.
:)
-- Miguel Cobá http://miguel.leugim.com.mx
participants (2)
-
Mariano Martinez Peck -
Miguel Cobá