[Pharo-project] Metacello --- How to make sure that we will be able to load... in 3 years from now
Hi all I would like that we have a discussion about how do we put in place an infrastructure to make sure that we can get metacello configuration working in 3/5 years from now. I think that there are two scenarios: MyProject ====== in my project I used metacello and I publish a config and if it breaks in the future this is my responsibility PublishedMyProjectIn ============= When I publish my project (for now I copy the configuration file from my repository to the one of metacello) in the metacelloPharo1.0 project, we could automatically pulled up all the necessary files and copy them in the 1.0 repository. This way we can shield and make sure that a project can be loaded. I would like to have MetacelloMetaRepository1.0 MetacelloMetaRepository1.1 MetacelloMetaRepository1.2 ... what do you think? Stef
El mar, 23-02-2010 a las 11:14 +0100, stephane ducasse escribió: I was thinking about this problem yesterday in the light of the packages moved by Lukas. We have a problem with the packages referenced by a ConfigurationOfXXX. The can disappear anytime. The repository can disappear anytime. So I thought that we need a dedicated hosting server where: - We have all the ConfigurationOfXXX (in the MetacelloRepository repo, just as now) - We copy there *all* the packages that conform a giver ConfigurationOfXXX>>versionXX: from the original repo to this dedicated repo. - There is *no way* to delete anything. This is a *write only, read always, delete never* repository. So maybe we need a modified squeaksource image there that disables delete/rename/move packages. Or maybe the metasource from Torsten (is that the name of the new app designed for Metacello) can add this requirement. Besides, the app can scan all the ConfigurationOfXXX and check for what ones are affected by a deletion of a package before permiting delete of the package. This way we can delete packages but also see what that will affect and correctly promptly. The point is, we need an independent, mostly never breaking metacello repository. Other idea will be to use google or github versioned repositories and from them build the published repo, so that when a package is deleted by accident or by mistake, it can be restored quickly. But I like more the previous version. What do you think? -- Miguel Cobá http://miguel.leugim.com.mx
Miguel Enrique Cobá Martinez wrote:
El mar, 23-02-2010 a las 11:14 +0100, stephane ducasse escribió: I was thinking about this problem yesterday in the light of the packages moved by Lukas. We have a problem with the packages referenced by a ConfigurationOfXXX. The can disappear anytime. The repository can disappear anytime.
So I thought that we need a dedicated hosting server where:
- We have all the ConfigurationOfXXX (in the MetacelloRepository repo, just as now) - We copy there *all* the packages that conform a giver ConfigurationOfXXX>>versionXX: from the original repo to this dedicated repo. - There is *no way* to delete anything. This is a *write only, read always, delete never* repository.
Sidenote: SqueakMap actually caches all URLs used for versions and if the original URL fails an SHA check it falls back on the cached file on the SqueakMap server. In that way it was meant to handle original URLs going stale or disappearing. regards, Göran
El mar, 23-02-2010 a las 14:12 +0100, Göran Krampe escribió:
Sidenote: SqueakMap actually caches all URLs used for versions and if the original URL fails an SHA check it falls back on the cached file on the SqueakMap server.
In that way it was meant to handle original URLs going stale or disappearing.
Does that mean that everything pushed to squeakmap can never be deleted? If that isn't true, then the problem is just pushed one step further.
regards, Göran
Thanks -- Miguel Cobá http://miguel.leugim.com.mx
neat!
El mar, 23-02-2010 a las 11:14 +0100, stephane ducasse escribió: I was thinking about this problem yesterday in the light of the packages moved by Lukas. We have a problem with the packages referenced by a ConfigurationOfXXX. The can disappear anytime. The repository can disappear anytime.
So I thought that we need a dedicated hosting server where:
- We have all the ConfigurationOfXXX (in the MetacelloRepository repo, just as now) - We copy there *all* the packages that conform a giver ConfigurationOfXXX>>versionXX: from the original repo to this dedicated repo. - There is *no way* to delete anything. This is a *write only, read always, delete never* repository.
Sidenote: SqueakMap actually caches all URLs used for versions and if the original URL fails an SHA check it falls back on the cached file on the SqueakMap server.
In that way it was meant to handle original URLs going stale or disappearing.
regards, Göran
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
I like the notion of a secondary repository for packages. It is absolutely necessary to protect us from a catastrophic failure with any one of the SqueakSource repositories. Perhaps Gofer can be provided with a secondary repository location in the case when the primary repository fails. With this kind of approach, the metacello configurations can continue to reference the primary development repository, in this way we might just be able to have our cake and eat it too:) The Pharo secondary repository would protect users from repository volatility and package volatility ... Once a version is published in Pharo you can count of being able to load that version over time. This secondary repository could be maintained by periodically scanning the Pharo configurations and copying the referenced packages from the primary repository to a known secondary repository (this repository could just be done with Apache file access since a fancy UI for the secondary repository isn't essential). Having code and processes in place means that individuals can reuse the code and process to maintain their own secondary repository for their own production environment. I have code that does most of what is needed, as I use this process at GemStone for archiving the configurations and mcz packages in our company source code control system. Dale ----- "Göran Krampe" <goran@krampe.se> wrote: | Miguel Enrique Cobá Martinez wrote: | > El mar, 23-02-2010 a las 11:14 +0100, stephane ducasse escribió: | > I was thinking about this problem yesterday in the light of the | packages | > moved by Lukas. | > We have a problem with the packages referenced by a | ConfigurationOfXXX. | > The can disappear anytime. The repository can disappear anytime. | > | > So I thought that we need a dedicated hosting server where: | > | > - We have all the ConfigurationOfXXX (in the MetacelloRepository | repo, | > just as now) | > - We copy there *all* the packages that conform a giver | > ConfigurationOfXXX>>versionXX: from the original repo to this | dedicated | > repo. | > - There is *no way* to delete anything. This is a *write only, read | > always, delete never* repository. | | Sidenote: SqueakMap actually caches all URLs used for versions and if | the original URL fails an SHA check it falls back on the cached file | on | the SqueakMap server. | | In that way it was meant to handle original URLs going stale or | disappearing. | | regards, Göran | | | _______________________________________________ | Pharo-project mailing list | Pharo-project@lists.gforge.inria.fr | http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Perhaps Gofer can be provided with a secondary repository location in the case when the primary repository fails.
Yes, it can. You just specify multiple repositories and it will use them all. If you tell Gofer to #disableRepositoryErrors then it will silently ignore repositories that are not available. Lukas -- Lukas Renggli http://www.lukas-renggli.ch
----- "Lukas Renggli" <renggli@gmail.com> wrote: | > Perhaps Gofer can be provided with a secondary repository location | in the case when the primary repository fails. | | Yes, it can. You just specify multiple repositories and it will use | them all. If you tell Gofer to #disableRepositoryErrors then it will | silently ignore repositories that are not available. Lukas, The only thing that worries about including such repository all of the time, is the cost of building the cache of repository when resolving repositories ... the secondary repository would get pretty large over time and most of the time (only in the case of errors) would it need to be scanned. I suppose Metacello could handle load errors in the code that calls Gofer and do a retry using the secondary repository...the secondary repository _could_ be specified as part of the configuration, so this probably isn't a bad idea... Thanks, Dale
Hi Dale, I believe the original problem is that you do not know upfront about where the actual files will be. In this case, perhaps the easiest solution is to provide some scripting facility to provide a repository mapping that should could be modified at loading time, and not be stored in the actual configuration. Suppose you have something like this: ConfigurationOfXXX>>baselineX: spec <version: 'X-baseline'> spec package: 'SomePackage' with: [ spec repository: 'SOME_URL' ]. ... ConfigurationOfXXX>>versionX: spec <version: 'X' imports: #('X-baseline')> spec package: 'SomePackage' with: 'SomePackage.tg.1'. ... Maybe a solution would be that when loading you could say: (ConfigurationOfXXX project version: 'X') withRepositoryMapping: 'SOME_URL' -> 'SOME_NEW_URL'; withRepositoryMapping: '...' -> '...'; load What do you think? In this way, the original configuration specifies everything, and the URL works either as the actual URL or as an alias that can be mapped to something else later on when you know the actual URL. Actually, this can already be accomplished at engine level. I believe a simple scripting like above would just make it practical. What do you think? Cheers, Doru On 23 Feb 2010, at 21:18, Dale Henrichs wrote:
----- "Lukas Renggli" <renggli@gmail.com> wrote:
| > Perhaps Gofer can be provided with a secondary repository location | in the case when the primary repository fails. | | Yes, it can. You just specify multiple repositories and it will use | them all. If you tell Gofer to #disableRepositoryErrors then it will | silently ignore repositories that are not available.
Lukas,
The only thing that worries about including such repository all of the time, is the cost of building the cache of repository when resolving repositories ... the secondary repository would get pretty large over time and most of the time (only in the case of errors) would it need to be scanned.
I suppose Metacello could handle load errors in the code that calls Gofer and do a retry using the secondary repository...the secondary repository _could_ be specified as part of the configuration, so this probably isn't a bad idea...
Thanks,
Dale
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- www.tudorgirba.com "Every thing has its own flow."
What if use redudancy features of Internet itself? As you know the Internet was designed to withstand a nuclear attack, with a lot of redudant features, which we can explore to harden the Smalltalk infracture as well. In the similar ways as guyss like Google do. You know, they have always a single entry, http://www.google.com, but actually they direct your search to 1000s and milions of their servers behind. A mini version of Google can be use for Squeak or PharoSource as well. As I can see the Monticello repositories are basically the stateless WebDAV enabled Apache web servers. So having a mirror is a first thing to do. To still have a single entry point like http://www.squeaksource.com), we can exploit the redundancy feature of DNS: round robin entries. That is, you enter two entries in DNS zone, with the same name but different IP addresses, say: squeaksource.com www IN A 87.54.32.108 www IN A 212.18.45.87 This way in the case of failure of one server, the another one will responds. If both are working, DNS will return one of them in roun robin fashion. This will work for WebDAV part of the story, but we still need to tackle of the statefull web application of SqueakSource... hmm, a bit more thinking is needed therefore... Best regards Janko On 23. 02. 2010 19:48, Dale Henrichs wrote:
I like the notion of a secondary repository for packages. It is absolutely necessary to protect us from a catastrophic failure with any one of the SqueakSource repositories.
Perhaps Gofer can be provided with a secondary repository location in the case when the primary repository fails.
With this kind of approach, the metacello configurations can continue to reference the primary development repository, in this way we might just be able to have our cake and eat it too:) The Pharo secondary repository would protect users from repository volatility and package volatility ... Once a version is published in Pharo you can count of being able to load that version over time.
This secondary repository could be maintained by periodically scanning the Pharo configurations and copying the referenced packages from the primary repository to a known secondary repository (this repository could just be done with Apache file access since a fancy UI for the secondary repository isn't essential).
Having code and processes in place means that individuals can reuse the code and process to maintain their own secondary repository for their own production environment.
I have code that does most of what is needed, as I use this process at GemStone for archiving the configurations and mcz packages in our company source code control system.
Dale ----- "Göran Krampe" <goran@krampe.se> wrote:
| Miguel Enrique Cobá Martinez wrote: | > El mar, 23-02-2010 a las 11:14 +0100, stephane ducasse escribió: | > I was thinking about this problem yesterday in the light of the | packages | > moved by Lukas. | > We have a problem with the packages referenced by a | ConfigurationOfXXX. | > The can disappear anytime. The repository can disappear anytime. | > | > So I thought that we need a dedicated hosting server where: | > | > - We have all the ConfigurationOfXXX (in the MetacelloRepository | repo, | > just as now) | > - We copy there *all* the packages that conform a giver | > ConfigurationOfXXX>>versionXX: from the original repo to this | dedicated | > repo. | > - There is *no way* to delete anything. This is a *write only, read | > always, delete never* repository. | | Sidenote: SqueakMap actually caches all URLs used for versions and if | the original URL fails an SHA check it falls back on the cached file | on | the SqueakMap server. | | In that way it was meant to handle original URLs going stale or | disappearing. | | regards, Göran | | | _______________________________________________ | Pharo-project mailing list | Pharo-project@lists.gforge.inria.fr | http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Janko Mivšek Svetovalec za informatiko Eranova d.o.o. Ljubljana, Slovenija www.eranova.si tel: 01 514 22 55 faks: 01 514 22 56 gsm: 031 674 565
Hi, I am very interested in the whole configuration management, metacello, monticello, public/private repository, project, library/package, object, project, environment, release, version stuff ... some clarity around is badly needed. My first contact with the whole Squeak/Pharo packages and versioning was very confusing with SqueakSource, SqueakMap, Sake/Packages, Package Universe, Monticello. Why not look at other distributed version control systems like Darcs, Bazaar, GIT ... and see what could work for smalltalk? -- View this message in context: http://n4.nabble.com/Metacello-How-to-make-sure-that-we-will-be-able-to-load... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
and with that I did not mean to "use" those other DVCS but maybe take ideas from them :) -- View this message in context: http://n4.nabble.com/Metacello-How-to-make-sure-that-we-will-be-able-to-load... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
and with that I did not mean to "use" those other DVCS but maybe take ideas from them :)
This is not a problem any DVCS tackles. Rather take ideas of package management systems like aptitude, rpm, portage, fink, ... As you see in all such system, they ship the code (binary or source) together with the package definition. None of these mechanisms refer to the VCS where the code is developed, but they include all the dependencies they need. I was thinking more along of something of a SAR file that would include the Metacello configuration and all the referenced mcz files in one file. Dragging and dropping that file onto the image (or using some GUI) would bootstrap Metacello, execute the Metacello definition, and if necessary download and pull in other SAR files from a well known place. That's how Linux works quite successful for over a decade. Lukas -- Lukas Renggli http://www.lukas-renggli.ch
participants (9)
-
Dale Henrichs -
Geert Claes -
Göran Krampe -
Janko Mivšek -
Lukas Renggli -
Miguel Enrique Cobá Martinez -
stephane ducasse -
Stéphane Ducasse -
Tudor Girba