How does Metacello decide what is already loaded ?
Hi, How does Metacello decide what is already loaded ? Specifically in the context of 'shared dependencies' ? Say project A depends on X and project B also depends on X ? How does Metacello then decide they are talking about the same X ? Is that by name only ? Does the repository count as well ? Does the version/group count ? How can I see what is known as loaded and what the dependencies are ? How can we, as a community, make sure that we properly share dependencies ? Sven
Sven Van Caekenberghe-2 wrote
How does Metacello decide what is already loaded ?
IIRC, if you load via the scripting API (i.e. 'Metacello new..."), versions are kept in a registry, but if you use e.g. Gofer, it "guesses" the version based on the package versions present. I can't answer your more specific questions, but since one can "lock" based e.g. on repository location, that must be considered somewhere. Also, NB, I doubt Dale actively follows this list, so you may want to ping him. ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
On 14 Feb 2019, at 18:40, Sean P. DeNigris <sean@clipperadams.com> wrote:
Sven Van Caekenberghe-2 wrote
How does Metacello decide what is already loaded ?
IIRC, if you load via the scripting API (i.e. 'Metacello new..."), versions are kept in a registry, but if you use e.g. Gofer, it "guesses" the version based on the package versions present. I can't answer your more specific questions, but since one can "lock" based e.g. on repository location, that must be considered somewhere. Also, NB, I doubt Dale actively follows this list, so you may want to ping him.
But is it not a bit sad that we don't know ? We're using it all the time, but we lack important information. This is probably because somehow Metacello does the right thing automagically, but still.
As Sean said, when using Metacello scripting, it uses a registry, and that's how it achieves the version "lock", "use local" or "use incoming" version. And yes... there are a lot of things that "just work", we thank it, take for granted, and complain when they fail :) Regards, Esteban A. Maringolo El jue., 14 feb. 2019 a las 14:50, Sven Van Caekenberghe (<sven@stfx.eu>) escribió:
On 14 Feb 2019, at 18:40, Sean P. DeNigris <sean@clipperadams.com> wrote:
Sven Van Caekenberghe-2 wrote
How does Metacello decide what is already loaded ?
IIRC, if you load via the scripting API (i.e. 'Metacello new..."), versions are kept in a registry, but if you use e.g. Gofer, it "guesses" the version based on the package versions present. I can't answer your more specific questions, but since one can "lock" based e.g. on repository location, that must be considered somewhere. Also, NB, I doubt Dale actively follows this list, so you may want to ping him.
But is it not a bit sad that we don't know ?
We're using it all the time, but we lack important information.
This is probably because somehow Metacello does the right thing automagically, but still.
OK, where is the registry, how does it work, where can I read about it ? Let's make this more concrete: I load Bootstrap which will load Seaside and I load Material Design Lite which will load Seaside. Will Seaside be loaded twice ? On what does this depend ? How can we as a community make sure we are talking about the same Seaside ? That is essentially my question. I know it is not an easy question, but I find it an important one.
On 14 Feb 2019, at 20:09, Esteban Maringolo <emaringolo@gmail.com> wrote:
As Sean said, when using Metacello scripting, it uses a registry, and that's how it achieves the version "lock", "use local" or "use incoming" version.
And yes... there are a lot of things that "just work", we thank it, take for granted, and complain when they fail :)
Regards,
Esteban A. Maringolo
El jue., 14 feb. 2019 a las 14:50, Sven Van Caekenberghe (<sven@stfx.eu>) escribió:
On 14 Feb 2019, at 18:40, Sean P. DeNigris <sean@clipperadams.com> wrote:
Sven Van Caekenberghe-2 wrote
How does Metacello decide what is already loaded ?
IIRC, if you load via the scripting API (i.e. 'Metacello new..."), versions are kept in a registry, but if you use e.g. Gofer, it "guesses" the version based on the package versions present. I can't answer your more specific questions, but since one can "lock" based e.g. on repository location, that must be considered somewhere. Also, NB, I doubt Dale actively follows this list, so you may want to ping him.
But is it not a bit sad that we don't know ?
We're using it all the time, but we lack important information.
This is probably because somehow Metacello does the right thing automagically, but still.
Hi Sven, https://github.com/dalehenrich/metacello-work/blob/master/docs/MetacelloUser... includes a chapter on load conflicts. Thx T.
On 2/14/19 11:28 AM, Sven Van Caekenberghe wrote:
OK, where is the registry, how does it work, where can I read about it ?
`MetacelloProjectRegistration registry` ... It is a private registry of loaded baselines and configurations used by Metacello. It's been around roughly 6-7 years or so (ever since I added git support to Metacello and when I create the `Metacello new` style of launching Metacello loads) ... 6-7 years ago I wrote a first cut of documentation that Torsten references, but at that time no one cared about the features I had added to Metacello, because, well they pretty much applied to git-based support and using and the `Metacello new` style of launching Metacello and it was hard enought to get folks to use `Metacello new` by itself, let alone any of the "new features" ... Metacello started out (2009) supporting Monticello repositories (ConfigurationOf) and then transitioned (2012) to supporting git repositories (BaselineOf). Much of the complexity in Metacello is a carry over from Monticello support ... and isn't really needed. Today (2018) I am working on Rowan which is/will be a wholesale replacement for Monticello/Metacello and the specifications for what packages will be loaded is simplified quite a bit ... So it is a bit ironic, that real interest in how Metacello works comes at a time when I am well on my way to finishing a replacement for Metacello:) Okay this is not entirely true ... over the years I have been quizzed by folks and if you look through the pharo mailing list archives, you'll probably see that I've written a few long-winded mails answering questions or trying to describe how things work:)
Let's make this more concrete: I load Bootstrap which will load Seaside and I load Material Design Lite which will load Seaside. Will Seaside be loaded twice ? On what does this depend ? How can we as a community make sure we are talking about the same Seaside ?
That is essentially my question.
I know it is not an easy question, but I find it an important one.
First there is the Monticello "mcz file" answer, which applies to real .mcz files and filetree repositories with Monticello meta data. The simplest case is a project without dependencies. When a Metacello load is kicked off, Metacello evaluates the configuration or baseline specification and determines the list of packages in load order that are needed to satisfy the specification. Metacello then traverses the list and compares the mcz version of the Monticello package working copy in the image with the mcz version of the Monticello package on disk those Monticello packages that are newer than the loaded packages are then loaded into the image .... (how the packages are loaded is dependent upon whether not using are using #atomic or #linear loads). If there is a project dependency, Metacello determines the version of the project that is loaded in the image (looking in the registry) and compares that to the incoming version of the project. If the incoming project version is greater than or equal to the loaded project (this is a simplification of reality, since individual loads can be customized to allow or disallow a number of conflicts and or default decisions made by Metacello), the packages from the dependent project are added to the list of packages in load order ... This process is repeated until all dependent projects are accounted for ... If a dependency loop is encountered as you suggest, Metacello will choose to load the latest version of the dependent project ... In general Metacello loads the latest version of a project encountered and the latest version of package encountered ... If you inspect/print the result `Metacello new ... record` you will see a collection of load directives, which details which packages will be loaded the order that they will be loaded. Here's an example printed load directive from a GemStone image I happen to have open (Seaside is not loaded): [ Metacello new baseline: 'Seaside3'; record ] on: Warning do: [:ex | ex resume ]. linear load : linear load : baseline [BaselineOfSeaside3] load : BaselineOfGLASS1 linear load : baseline [BaselineOfSeaside3] load : BaselineOfGrease linear load : baseline [BaselineOfSeaside3] load : BaselineOfGrease linear load : baseline [BaselineOfSeaside3] linear load : baseline [BaselineOfGLASS1] load : BaselineOfGrease linear load : baseline [BaselineOfGLASS1] load : BaselineOfMetacello linear load : baseline [BaselineOfGLASS1] linear load : 1.0-alpha2.2 [ConfigurationOfGoferProjectLoader] load : Network-Url linear load : baseline [BaselineOfGrease] atomic load : 0.250 [ConfigurationOfGsCore] load : Network-Url-dkh.2 atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-Core atomic load : 0.243 [ConfigurationOfGsMisc] atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-GemStone-Core load : Grease-GemStone330-Core load : Grease-Tests-Core load : Grease-Tests-GemStone-Core.v32 linear load : baseline [BaselineOfMetacello] load : BaselineOfFileTree linear load : baseline [BaselineOfMetacello] load : Metacello-Base load : Metacello-Core load : Metacello-MC load : Metacello-Platform.gemstone load : Metacello-ToolBox load : Metacello-Cypress load : Metacello-Tutorial load : Metacello-TestsCore load : Metacello-TestsMCResources load : Metacello-TestsCommonMC.common load : Metacello-TestsMCCore load : Metacello-TestsTutorial linear load : baseline [BaselineOfFileTree] load : MonticelloFileTree-Core load : MonticelloFileTree-FileDirectory-Utilities load : Metacello-GS3x-Platform load : Metacello-FileTree load : Metacello-GitBasedRepository load : Metacello-GitHub load : Metacello-Bitbucket load : Metacello-Reference load : Metacello-TestsReference load : Metacello-TestsMC load : Metacello-TestsMCB load : Metacello-TestsCypress load : Metacello-TestsMCA load : Metacello-TestsPlatform.gemstone load : Metacello-GemStone-TestsMCB load : GemStone-Interactions load : Core.v3 load : GemStone-Compression load : Core322x load : Base-Bootstrap.v3 load : Bootstrap.v34 load : GemStone-ANSI-Streams load : GemStone-Indexing-Extensions.v34 load : OmniBrowser load : OB-GemStone-Platform load : Squeak.v34 load : Regex-Core load : Regex-Tests-Core load : GsSqueakCommon-Core load : GsSqueakCommon-CoreV30 load : PackageInfo-Base.g load : Monticello.v34 load : MonticelloGs.v33 load : GemStone-Deployment.v310 load : Change-Notification.v310 load : OB-Standard.v3 load : OB-Monticello load : OB-SymbolListBrowser load : OB-SUnitIntegration load : OB-SUnitGUI.g load : Announcements load : OB-Tools.v33 load : JadeServer load : GemStone-Release-Support load : Seaside-Squeak-Compatibility linear load : baseline [BaselineOfGrease] atomic load : 0.250 [ConfigurationOfGsCore] load : Core.v3-dkh.78 load : Core322x-dkh.2 load : Base-Bootstrap.v3-dkh.29 load : Bootstrap.v34-dkh.261 load : GemStone-ANSI-Streams-dkh.9 load : GemStone-Indexing-Extensions-dkh.3 load : Squeak.v34-dkh.339 load : Regex-Core-DaleHenrichs.3 load : Regex-Tests-Core-DaleHenrichs.5 atomic load : 0.243 [ConfigurationOfGsMisc] load : Announcements.v3-dkh.18 load : GemStone-Release-Support-dkh.67 load : Grease-Core atomic load : 0.243 [ConfigurationOfGsMisc] atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-GemStone-Core load : Grease-GemStone330-Core load : Grease-Tests-Core load : Grease-Tests-GemStone-Core.v32 load : Seaside-Continuation linear load : baseline [BaselineOfGrease] atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-Core atomic load : 0.243 [ConfigurationOfGsMisc] atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-GemStone-Core load : Grease-GemStone330-Core load : Grease-Tests-Core load : Grease-Tests-GemStone-Core.v32 load : Seaside-Core load : Seaside-GemStone320-Core load : Seaside-GemStone-Core load : Seaside-GemStone-Basic-Development load : Seaside-Tests-Core load : Seaside-Tests-GemStone-Core load : Seaside-Email load : Seaside-GemStone-Email load : Seaside-Canvas load : Seaside-JSON-Core load : Seaside-GemStone-JSON-Core load : Javascript-Core load : Javascript-GemStone-Core load : JQuery-Core load : JQuery-UI load : JQuery-JSON load : Seaside-Session load : Seaside-GemStone-Session load : Seaside-Component load : Seaside-Widgets load : Seaside-GemStone-Tools-Production load : Seaside-RenderLoop load : Seaside-Tools-Core load : Seaside-Tools-Web load : Seaside-GemStone-Tools-Web load : Seaside-Flow load : Seaside-Development load : Seaside-GemStone-Development load : Seaside-Examples load : Seaside-Environment load : Seaside-GemStone-Environment load : Seaside-Tests-Functional load : Seaside-Tests-GemStone-Functional load : Seaside-Tests-Flow load : Seaside-Tests-GemStone-Flow load : Seaside-Welcome load : Seaside-GemStone-Welcome load : Seaside-GemStone-Continuation load : Seaside-Tests-GemStone-Continuation load : Seaside-GemStone-Flow I started off talking about the Monticello load with Monticello meta data ... If you've got a metadataless repository and you've got a projectClass method in the baseline that references MetacelloCypressBaselineProject, then Metacello will ignore the mcz version calculation and load all of the packages from the target repository (which because of git, are all of the proper version). Metacello relies on the fact that Monticello does a package definition comparison before loading which means that only the changed definitions will actually be loaded ... If you don't have a projectClass method in your baseline in a metadataless repository, then Metacello _will_ apply the mcz version rule and because all of the mcz version for packages in a metadataless repository is 1, Metacello will not load anything at all ... the second time you try to load the project in the repository ... Dale
Hi Dale, Thanks for the explanation. For now I managed to do what I wanted, using #onConflict #useLoaded I could prevent a dependencies' ConfigurationOf from going over a BaselineOf with the same name. Sven
On 15 Feb 2019, at 01:50, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:
On 2/14/19 11:28 AM, Sven Van Caekenberghe wrote:
OK, where is the registry, how does it work, where can I read about it ? `MetacelloProjectRegistration registry` ... It is a private registry of loaded baselines and configurations used by Metacello. It's been around roughly 6-7 years or so (ever since I added git support to Metacello and when I create the `Metacello new` style of launching Metacello loads) ... 6-7 years ago I wrote a first cut of documentation that Torsten references, but at that time no one cared about the features I had added to Metacello, because, well they pretty much applied to git-based support and using and the `Metacello new` style of launching Metacello and it was hard enought to get folks to use `Metacello new` by itself, let alone any of the "new features" ...
Metacello started out (2009) supporting Monticello repositories (ConfigurationOf) and then transitioned (2012) to supporting git repositories (BaselineOf). Much of the complexity in Metacello is a carry over from Monticello support ... and isn't really needed.
Today (2018) I am working on Rowan which is/will be a wholesale replacement for Monticello/Metacello and the specifications for what packages will be loaded is simplified quite a bit ...
So it is a bit ironic, that real interest in how Metacello works comes at a time when I am well on my way to finishing a replacement for Metacello:) Okay this is not entirely true ... over the years I have been quizzed by folks and if you look through the pharo mailing list archives, you'll probably see that I've written a few long-winded mails answering questions or trying to describe how things work:)
Let's make this more concrete: I load Bootstrap which will load Seaside and I load Material Design Lite which will load Seaside. Will Seaside be loaded twice ? On what does this depend ? How can we as a community make sure we are talking about the same Seaside ?
That is essentially my question.
I know it is not an easy question, but I find it an important one.
First there is the Monticello "mcz file" answer, which applies to real .mcz files and filetree repositories with Monticello meta data. The simplest case is a project without dependencies.
When a Metacello load is kicked off, Metacello evaluates the configuration or baseline specification and determines the list of packages in load order that are needed to satisfy the specification. Metacello then traverses the list and compares the mcz version of the Monticello package working copy in the image with the mcz version of the Monticello package on disk those Monticello packages that are newer than the loaded packages are then loaded into the image .... (how the packages are loaded is dependent upon whether not using are using #atomic or #linear loads).
If there is a project dependency, Metacello determines the version of the project that is loaded in the image (looking in the registry) and compares that to the incoming version of the project. If the incoming project version is greater than or equal to the loaded project (this is a simplification of reality, since individual loads can be customized to allow or disallow a number of conflicts and or default decisions made by Metacello), the packages from the dependent project are added to the list of packages in load order ...
This process is repeated until all dependent projects are accounted for ...
If a dependency loop is encountered as you suggest, Metacello will choose to load the latest version of the dependent project ...
In general Metacello loads the latest version of a project encountered and the latest version of package encountered ...
If you inspect/print the result `Metacello new ... record` you will see a collection of load directives, which details which packages will be loaded the order that they will be loaded.
Here's an example printed load directive from a GemStone image I happen to have open (Seaside is not loaded):
[ Metacello new baseline: 'Seaside3'; record ] on: Warning do: [:ex | ex resume ].
linear load : linear load : baseline [BaselineOfSeaside3] load : BaselineOfGLASS1 linear load : baseline [BaselineOfSeaside3] load : BaselineOfGrease linear load : baseline [BaselineOfSeaside3] load : BaselineOfGrease linear load : baseline [BaselineOfSeaside3] linear load : baseline [BaselineOfGLASS1] load : BaselineOfGrease linear load : baseline [BaselineOfGLASS1] load : BaselineOfMetacello linear load : baseline [BaselineOfGLASS1] linear load : 1.0-alpha2.2 [ConfigurationOfGoferProjectLoader] load : Network-Url linear load : baseline [BaselineOfGrease] atomic load : 0.250 [ConfigurationOfGsCore] load : Network-Url-dkh.2 atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-Core atomic load : 0.243 [ConfigurationOfGsMisc] atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-GemStone-Core load : Grease-GemStone330-Core load : Grease-Tests-Core load : Grease-Tests-GemStone-Core.v32 linear load : baseline [BaselineOfMetacello] load : BaselineOfFileTree linear load : baseline [BaselineOfMetacello] load : Metacello-Base load : Metacello-Core load : Metacello-MC load : Metacello-Platform.gemstone load : Metacello-ToolBox load : Metacello-Cypress load : Metacello-Tutorial load : Metacello-TestsCore load : Metacello-TestsMCResources load : Metacello-TestsCommonMC.common load : Metacello-TestsMCCore load : Metacello-TestsTutorial linear load : baseline [BaselineOfFileTree] load : MonticelloFileTree-Core load : MonticelloFileTree-FileDirectory-Utilities load : Metacello-GS3x-Platform load : Metacello-FileTree load : Metacello-GitBasedRepository load : Metacello-GitHub load : Metacello-Bitbucket load : Metacello-Reference load : Metacello-TestsReference load : Metacello-TestsMC load : Metacello-TestsMCB load : Metacello-TestsCypress load : Metacello-TestsMCA load : Metacello-TestsPlatform.gemstone load : Metacello-GemStone-TestsMCB load : GemStone-Interactions load : Core.v3 load : GemStone-Compression load : Core322x load : Base-Bootstrap.v3 load : Bootstrap.v34 load : GemStone-ANSI-Streams load : GemStone-Indexing-Extensions.v34 load : OmniBrowser load : OB-GemStone-Platform load : Squeak.v34 load : Regex-Core load : Regex-Tests-Core load : GsSqueakCommon-Core load : GsSqueakCommon-CoreV30 load : PackageInfo-Base.g load : Monticello.v34 load : MonticelloGs.v33 load : GemStone-Deployment.v310 load : Change-Notification.v310 load : OB-Standard.v3 load : OB-Monticello load : OB-SymbolListBrowser load : OB-SUnitIntegration load : OB-SUnitGUI.g load : Announcements load : OB-Tools.v33 load : JadeServer load : GemStone-Release-Support load : Seaside-Squeak-Compatibility linear load : baseline [BaselineOfGrease] atomic load : 0.250 [ConfigurationOfGsCore] load : Core.v3-dkh.78 load : Core322x-dkh.2 load : Base-Bootstrap.v3-dkh.29 load : Bootstrap.v34-dkh.261 load : GemStone-ANSI-Streams-dkh.9 load : GemStone-Indexing-Extensions-dkh.3 load : Squeak.v34-dkh.339 load : Regex-Core-DaleHenrichs.3 load : Regex-Tests-Core-DaleHenrichs.5 atomic load : 0.243 [ConfigurationOfGsMisc] load : Announcements.v3-dkh.18 load : GemStone-Release-Support-dkh.67 load : Grease-Core atomic load : 0.243 [ConfigurationOfGsMisc] atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-GemStone-Core load : Grease-GemStone330-Core load : Grease-Tests-Core load : Grease-Tests-GemStone-Core.v32 load : Seaside-Continuation linear load : baseline [BaselineOfGrease] atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-Core atomic load : 0.243 [ConfigurationOfGsMisc] atomic load : 0.243 [ConfigurationOfGsMisc] load : Grease-GemStone-Core load : Grease-GemStone330-Core load : Grease-Tests-Core load : Grease-Tests-GemStone-Core.v32 load : Seaside-Core load : Seaside-GemStone320-Core load : Seaside-GemStone-Core load : Seaside-GemStone-Basic-Development load : Seaside-Tests-Core load : Seaside-Tests-GemStone-Core load : Seaside-Email load : Seaside-GemStone-Email load : Seaside-Canvas load : Seaside-JSON-Core load : Seaside-GemStone-JSON-Core load : Javascript-Core load : Javascript-GemStone-Core load : JQuery-Core load : JQuery-UI load : JQuery-JSON load : Seaside-Session load : Seaside-GemStone-Session load : Seaside-Component load : Seaside-Widgets load : Seaside-GemStone-Tools-Production load : Seaside-RenderLoop load : Seaside-Tools-Core load : Seaside-Tools-Web load : Seaside-GemStone-Tools-Web load : Seaside-Flow load : Seaside-Development load : Seaside-GemStone-Development load : Seaside-Examples load : Seaside-Environment load : Seaside-GemStone-Environment load : Seaside-Tests-Functional load : Seaside-Tests-GemStone-Functional load : Seaside-Tests-Flow load : Seaside-Tests-GemStone-Flow load : Seaside-Welcome load : Seaside-GemStone-Welcome load : Seaside-GemStone-Continuation load : Seaside-Tests-GemStone-Continuation load : Seaside-GemStone-Flow
I started off talking about the Monticello load with Monticello meta data ... If you've got a metadataless repository and you've got a projectClass method in the baseline that references MetacelloCypressBaselineProject, then Metacello will ignore the mcz version calculation and load all of the packages from the target repository (which because of git, are all of the proper version). Metacello relies on the fact that Monticello does a package definition comparison before loading which means that only the changed definitions will actually be loaded ...
If you don't have a projectClass method in your baseline in a metadataless repository, then Metacello _will_ apply the mcz version rule and because all of the mcz version for packages in a metadataless repository is 1, Metacello will not load anything at all ... the second time you try to load the project in the repository ...
Dale
Dale Henrichs-3 wrote
`MetacelloProjectRegistration registry`
I'd like to add a clue to MetacelloProjectRegistry that one typically accesses it from MetacelloProjectRegistration, maybe in the class comment. Should I do a PR for MetaC, Pharo, or both (not quite clear on exactly how we're handling that relationship ATM)? ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
participants (5)
-
Dale Henrichs -
Esteban Maringolo -
Sean P. DeNigris -
Sven Van Caekenberghe -
Torsten Bergmann