Hi Dale, The question is how to get all packages that will be loaded for a given configuration version in the correct order. We want to obtain that to provide a flattened configuration version based on what is in the image. Cheers, Doru On 21 Nov 2012, at 19:33, Dale Henrichs <dhenrich@vmware.com> wrote:

Stef,

If you need to leverage the multi-package load, you can do that in Metacello by using the #atomic loadType in the project method of your configuration:

project ^ project ifNil: [ "Bootstrap Metacello if it is not already loaded" self class ensureMetacello. project := self projectClass new projectAttributes: self customProjectAttributes. "Create the Metacello project" project versionNumberClass: self versionNumberClass. project class versionConstructorClass on: self project: project. "Construct the project" project loadType: #'atomic'. "change to #atomic if desired" project ]

With #atomic, all of the packages are loaded in one multi-package load. with #linear (the default) each package is loaded individually.

If you inspect the loadDirectives that are produced by the #record message, the load type is encoded in the class of the directive, so you can tell whether #linear or #atomic loadTypes are used in the configurations...

Does this help?

Dale

----- Original Message ----- | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | To: "Pharo Development" <Pharo-project@lists.gforge.inria.fr> | Sent: Saturday, November 17, 2012 11:28:03 AM | Subject: [Pharo-project] Metacello loading semantics | | Hi dale | | I implemented a little tool that | - create a kind of specification (version, package, repo) for all | the packages reachable from a configurationOf | (see below method method | packagesAndAssociatedRepositoriesFromConfigurationRecord ) | | It generates a script looking as follows: | script3 | ^ #( | #('ConfigurationOfFame-TudorGirba.5.mcz' | 'http://www.squeaksource.com/Fame/' 'ConfigurationOfFame' ) | #('ConfigurationOfMooseAlgos-usmanbhatti.27.mcz' | 'http://www.squeaksource.com/MooseAlgos/' | 'ConfigurationOfMooseAlgos' ) | #('ConfigurationOfGlamour-TudorGirba.98.mcz' | 'http://www.squeaksource.com/Glamour/' 'ConfigurationOfGlamour' | ) | #('ConfigurationOfDSM-AlexandreBergel.56.mcz' | 'http://www.squeaksource.com/dsm/' 'ConfigurationOfDSM' ) | #('ConfigurationOfKumpel-AlexandreBergel.3.mcz' | 'http://www.squeaksource.com/Kumpel/' 'ConfigurationOfKumpel' ) | #('ConfigurationOfSmallDud………….) | | - I keep the order given by Metacello. I extract the information as | follows: | | packagesAndAssociatedRepositoriesFromConfigurationRecord: | aConfigurationRecord | "| record | | record := ((Smalltalk globals at: #ConfigurationOfMoose) project | version: 'default') ignoreImage: true; record. | self packagesAndAssociatedRepositoriesFromConfigurationRecord: | record | " | | | pkgs | | pkgs := OrderedCollection new. | aConfigurationRecord loadDirective packageDirectivesDo: | [:directive | | pkgs add: | {directive spec name, '*'. | (self removeCache: directive spec workingCopy | repositoryGroup repositories) anyOne | locationWithTrailingSlash} ]. | ^ pkgs | | | - then I try to load the package as follows: | | flatVersion47: spec | <version: '4.7-flat'> | | spec for: #common do: [ | self populateSpec: spec with: self script3 ] | | I load them doing | (ConfigurationOfMoose project version: '4.7-flat') load | Now It does not work because after fetching all the packages, the | system complain that some classes that will be loaded later are not | present. | I imagine that you use Gofer and that Gofer does not use the merging | semantics of MC over multiple projects (that you can have multiple | package in any order and that the classes get loaded - as I use it | for the scriptLoader and below). | | | - before I was doing it by doing a diff between the pharo package | and the one loaded and I was loading the specification after | installing the repository | and the working copies using the MCVersionLoader and it worked I | could reload completely Moose! | | | load: aCollectionOfPackageString | "self new | load: #( 'Famix-C-TudorGirba.15' | 'Famix-Core-NicolasAnquetil.201'))" | | | loader | | loader := MCVersionLoader new. | (self newerVersionsIn: aCollectionOfPackageString) | do: [ :fn | loader addVersion: (self loadVersionFromFileNamed: fn). | (Delay forSeconds: 2) wait.] | displayingProgress: 'Adding versions...'. | loader hasVersions ifTrue: [loader load]. | | | So my question is | - Does the packageDirectivesDo: provide an order that I can use to | load package with the gofer-load on project at a time? | May be my code does not keep the same order but I payed attention to | this point. I will check that further. | | - may be I could extend Gofer to support multiple project merge | loading. | | So right now I'm stuck and this is sad because freezing a complex | configuration is really needed. | | Stef | | | | | |

-- www.tudorgirba.com "It's not what we do that matters most, it's how we do it."

On Nov 21, 2012, at 8:51 PM, Dale Henrichs wrote:

Doru,

The loadDirectives structure is the structure that does the actual package load for metacello, so extracting the information from the loadDirectives is the correct technique for producing a list of packages in load order. From a cursory read of Stefs code he appears to be doing it correctly ... atomic/linear loadType is an important distinction that has to be taken into account when attempting to reproduce the Metacello load behavior without using Metacello…

We thought that too but I got a problem with Roassal since it looks like some classes are not loaded first.

Of course in the end, it's the details like which packages/projects specifically are causing trouble that will help me figure out exactly where things are breaking down. I assume that when Metacello does the load, everything gets loaded correctly?

Well when we use the configuration of Moose (not the flatten one) it loads: this is the Moose default configuration.

If you want me to help build/debug a package flattener for Moose I would be glad to help, but I need more detailed information about what you are trying to do and why. Also having the actual code to look at and a reproducible test case makes addressing issues a ton easier...Perhaps making an image available with a workspace doit primed for failure is a good first step:)

The code in MooseReloader on smalltalkhub. I'm going to bed that and tomorrow I follow lectures so I will explain it to you.

Dale

----- Original Message ----- | From: "Tudor Girba" <tudor@tudorgirba.com> | To: Pharo-project@lists.gforge.inria.fr | Sent: Wednesday, November 21, 2012 10:49:12 AM | Subject: Re: [Pharo-project] Metacello loading semantics | | Hi Dale, | | The question is how to get all packages that will be loaded for a | given configuration version in the correct order. We want to obtain | that to provide a flattened configuration version based on what is | in the image. | | Cheers, | Doru | | | | On 21 Nov 2012, at 19:33, Dale Henrichs <dhenrich@vmware.com> wrote: | | > Stef, | > | > If you need to leverage the multi-package load, you can do that in | > Metacello by using the #atomic loadType in the project method of | > your configuration: | > | > project | > ^ project | > ifNil: [ | > "Bootstrap Metacello if it is not already loaded" | > self class ensureMetacello. | > project := self projectClass new projectAttributes: self | > customProjectAttributes. "Create the Metacello project" | > project versionNumberClass: self versionNumberClass. | > project class versionConstructorClass on: self project: | > project. "Construct the project" | > project loadType: #'atomic'. "change to #atomic if | > desired" | > project ] | > | > With #atomic, all of the packages are loaded in one multi-package | > load. with #linear (the default) each package is loaded | > individually. | > | > If you inspect the loadDirectives that are produced by the #record | > message, the load type is encoded in the class of the directive, | > so you can tell whether #linear or #atomic loadTypes are used in | > the configurations... | > | > Does this help? | > | > Dale | > | > ----- Original Message ----- | > | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | > | To: "Pharo Development" <Pharo-project@lists.gforge.inria.fr> | > | Sent: Saturday, November 17, 2012 11:28:03 AM | > | Subject: [Pharo-project] Metacello loading semantics | > | | > | Hi dale | > | | > | I implemented a little tool that | > | - create a kind of specification (version, package, repo) for | > | all | > | the packages reachable from a configurationOf | > | (see below method method | > | packagesAndAssociatedRepositoriesFromConfigurationRecord ) | > | | > | It generates a script looking as follows: | > | script3 | > | ^ #( | > | #('ConfigurationOfFame-TudorGirba.5.mcz' | > | 'http://www.squeaksource.com/Fame/' 'ConfigurationOfFame' ) | > | #('ConfigurationOfMooseAlgos-usmanbhatti.27.mcz' | > | 'http://www.squeaksource.com/MooseAlgos/' | > | 'ConfigurationOfMooseAlgos' ) | > | #('ConfigurationOfGlamour-TudorGirba.98.mcz' | > | 'http://www.squeaksource.com/Glamour/' | > | 'ConfigurationOfGlamour' | > | ) | > | #('ConfigurationOfDSM-AlexandreBergel.56.mcz' | > | 'http://www.squeaksource.com/dsm/' 'ConfigurationOfDSM' ) | > | #('ConfigurationOfKumpel-AlexandreBergel.3.mcz' | > | 'http://www.squeaksource.com/Kumpel/' | > | 'ConfigurationOfKumpel' ) | > | #('ConfigurationOfSmallDud………….) | > | | > | - I keep the order given by Metacello. I extract the information | > | as | > | follows: | > | | > | packagesAndAssociatedRepositoriesFromConfigurationRecord: | > | aConfigurationRecord | > | "| record | | > | record := ((Smalltalk globals at: #ConfigurationOfMoose) | > | project | > | version: 'default') ignoreImage: true; record. | > | self | > | packagesAndAssociatedRepositoriesFromConfigurationRecord: | > | record | > | " | > | | > | | pkgs | | > | pkgs := OrderedCollection new. | > | aConfigurationRecord loadDirective packageDirectivesDo: | > | [:directive | | > | pkgs add: | > | {directive spec name, '*'. | > | (self removeCache: directive spec workingCopy | > | repositoryGroup repositories) anyOne | > | locationWithTrailingSlash} ]. | > | ^ pkgs | > | | > | | > | - then I try to load the package as follows: | > | | > | flatVersion47: spec | > | <version: '4.7-flat'> | > | | > | spec for: #common do: [ | > | self populateSpec: spec with: self script3 ] | > | | > | I load them doing | > | (ConfigurationOfMoose project version: '4.7-flat') load | > | Now It does not work because after fetching all the packages, the | > | system complain that some classes that will be loaded later are | > | not | > | present. | > | I imagine that you use Gofer and that Gofer does not use the | > | merging | > | semantics of MC over multiple projects (that you can have | > | multiple | > | package in any order and that the classes get loaded - as I use | > | it | > | for the scriptLoader and below). | > | | > | | > | - before I was doing it by doing a diff between the pharo | > | package | > | and the one loaded and I was loading the specification after | > | installing the repository | > | and the working copies using the MCVersionLoader and it worked I | > | could reload completely Moose! | > | | > | | > | load: aCollectionOfPackageString | > | "self new | > | load: #( 'Famix-C-TudorGirba.15' | > | 'Famix-Core-NicolasAnquetil.201'))" | > | | > | | loader | | > | loader := MCVersionLoader new. | > | (self newerVersionsIn: aCollectionOfPackageString) | > | do: [ :fn | loader addVersion: (self loadVersionFromFileNamed: | > | fn). | > | (Delay forSeconds: 2) wait.] | > | displayingProgress: 'Adding versions...'. | > | loader hasVersions ifTrue: [loader load]. | > | | > | | > | So my question is | > | - Does the packageDirectivesDo: provide an order that I can use | > | to | > | load package with the gofer-load on project at a time? | > | May be my code does not keep the same order but I payed | > | attention to | > | this point. I will check that further. | > | | > | - may be I could extend Gofer to support multiple project merge | > | loading. | > | | > | So right now I'm stuck and this is sad because freezing a complex | > | configuration is really needed. | > | | > | Stef | > | | > | | > | | > | | > | | > | | > | | -- | www.tudorgirba.com | | "It's not what we do that matters most, it's how we do it." | | |

No stress I promised the same and I know what it is. Stef On Nov 22, 2012, at 12:14 AM, Dale Henrichs wrote:

Stef,

The Thanksgiving holiday is coming up tomorrow and through the weekend ... I promised my wife that I would avoid the computer this weekend since I was away in Argentina for 10 days, but I will probably be able to find a couple of hours poke at an image and try to see where things are falling through the cracks:)

Dale

----- Original Message ----- | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | To: Pharo-project@lists.gforge.inria.fr | Sent: Wednesday, November 21, 2012 2:40:17 PM | Subject: Re: [Pharo-project] Metacello loading semantics | | | On Nov 21, 2012, at 8:51 PM, Dale Henrichs wrote: | | > Doru, | > | > The loadDirectives structure is the structure that does the actual | > package load for metacello, so extracting the information from the | > loadDirectives is the correct technique for producing a list of | > packages in load order. From a cursory read of Stefs code he | > appears to be doing it correctly ... atomic/linear loadType is an | > important distinction that has to be taken into account when | > attempting to reproduce the Metacello load behavior without using | > Metacello… | | We thought that too but I got a problem with Roassal since it looks | like some classes are not loaded first. | | > Of course in the end, it's the details like which packages/projects | > specifically are causing trouble that will help me figure out | > exactly where things are breaking down. I assume that when | > Metacello does the load, everything gets loaded correctly? | | Well when we use the configuration of Moose (not the flatten one) it | loads: this is the Moose default configuration. | | > If you want me to help build/debug a package flattener for Moose I | > would be glad to help, but I need more detailed information about | > what you are trying to do and why. Also having the actual code to | > look at and a reproducible test case makes addressing issues a ton | > easier...Perhaps making an image available with a workspace doit | > primed for failure is a good first step:) | | The code in MooseReloader on smalltalkhub. I'm going to bed that and | tomorrow I follow lectures so I will explain it to you. | | | | > | > Dale | > | > ----- Original Message ----- | > | From: "Tudor Girba" <tudor@tudorgirba.com> | > | To: Pharo-project@lists.gforge.inria.fr | > | Sent: Wednesday, November 21, 2012 10:49:12 AM | > | Subject: Re: [Pharo-project] Metacello loading semantics | > | | > | Hi Dale, | > | | > | The question is how to get all packages that will be loaded for a | > | given configuration version in the correct order. We want to | > | obtain | > | that to provide a flattened configuration version based on what | > | is | > | in the image. | > | | > | Cheers, | > | Doru | > | | > | | > | | > | On 21 Nov 2012, at 19:33, Dale Henrichs <dhenrich@vmware.com> | > | wrote: | > | | > | > Stef, | > | > | > | > If you need to leverage the multi-package load, you can do that | > | > in | > | > Metacello by using the #atomic loadType in the project method | > | > of | > | > your configuration: | > | > | > | > project | > | > ^ project | > | > ifNil: [ | > | > "Bootstrap Metacello if it is not already loaded" | > | > self class ensureMetacello. | > | > project := self projectClass new projectAttributes: | > | > self | > | > customProjectAttributes. "Create the Metacello | > | > project" | > | > project versionNumberClass: self versionNumberClass. | > | > project class versionConstructorClass on: self | > | > project: | > | > project. "Construct the project" | > | > project loadType: #'atomic'. "change to #atomic if | > | > desired" | > | > project ] | > | > | > | > With #atomic, all of the packages are loaded in one | > | > multi-package | > | > load. with #linear (the default) each package is loaded | > | > individually. | > | > | > | > If you inspect the loadDirectives that are produced by the | > | > #record | > | > message, the load type is encoded in the class of the | > | > directive, | > | > so you can tell whether #linear or #atomic loadTypes are used | > | > in | > | > the configurations... | > | > | > | > Does this help? | > | > | > | > Dale | > | > | > | > ----- Original Message ----- | > | > | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | > | > | To: "Pharo Development" <Pharo-project@lists.gforge.inria.fr> | > | > | Sent: Saturday, November 17, 2012 11:28:03 AM | > | > | Subject: [Pharo-project] Metacello loading semantics | > | > | | > | > | Hi dale | > | > | | > | > | I implemented a little tool that | > | > | - create a kind of specification (version, package, repo) | > | > | for | > | > | all | > | > | the packages reachable from a configurationOf | > | > | (see below method method | > | > | packagesAndAssociatedRepositoriesFromConfigurationRecord ) | > | > | | > | > | It generates a script looking as follows: | > | > | script3 | > | > | ^ #( | > | > | #('ConfigurationOfFame-TudorGirba.5.mcz' | > | > | 'http://www.squeaksource.com/Fame/' | > | > | 'ConfigurationOfFame' ) | > | > | #('ConfigurationOfMooseAlgos-usmanbhatti.27.mcz' | > | > | 'http://www.squeaksource.com/MooseAlgos/' | > | > | 'ConfigurationOfMooseAlgos' ) | > | > | #('ConfigurationOfGlamour-TudorGirba.98.mcz' | > | > | 'http://www.squeaksource.com/Glamour/' | > | > | 'ConfigurationOfGlamour' | > | > | ) | > | > | #('ConfigurationOfDSM-AlexandreBergel.56.mcz' | > | > | 'http://www.squeaksource.com/dsm/' 'ConfigurationOfDSM' | > | > | ) | > | > | #('ConfigurationOfKumpel-AlexandreBergel.3.mcz' | > | > | 'http://www.squeaksource.com/Kumpel/' | > | > | 'ConfigurationOfKumpel' ) | > | > | #('ConfigurationOfSmallDud………….) | > | > | | > | > | - I keep the order given by Metacello. I extract the | > | > | information | > | > | as | > | > | follows: | > | > | | > | > | packagesAndAssociatedRepositoriesFromConfigurationRecord: | > | > | aConfigurationRecord | > | > | "| record | | > | > | record := ((Smalltalk globals at: #ConfigurationOfMoose) | > | > | project | > | > | version: 'default') ignoreImage: true; record. | > | > | self | > | > | packagesAndAssociatedRepositoriesFromConfigurationRecord: | > | > | record | > | > | " | > | > | | > | > | | pkgs | | > | > | pkgs := OrderedCollection new. | > | > | aConfigurationRecord loadDirective packageDirectivesDo: | > | > | [:directive | | > | > | pkgs add: | > | > | {directive spec name, '*'. | > | > | (self removeCache: directive spec workingCopy | > | > | repositoryGroup repositories) anyOne | > | > | locationWithTrailingSlash} ]. | > | > | ^ pkgs | > | > | | > | > | | > | > | - then I try to load the package as follows: | > | > | | > | > | flatVersion47: spec | > | > | <version: '4.7-flat'> | > | > | | > | > | spec for: #common do: [ | > | > | self populateSpec: spec with: self script3 ] | > | > | | > | > | I load them doing | > | > | (ConfigurationOfMoose project version: '4.7-flat') load | > | > | Now It does not work because after fetching all the packages, | > | > | the | > | > | system complain that some classes that will be loaded later | > | > | are | > | > | not | > | > | present. | > | > | I imagine that you use Gofer and that Gofer does not use the | > | > | merging | > | > | semantics of MC over multiple projects (that you can have | > | > | multiple | > | > | package in any order and that the classes get loaded - as I | > | > | use | > | > | it | > | > | for the scriptLoader and below). | > | > | | > | > | | > | > | - before I was doing it by doing a diff between the pharo | > | > | package | > | > | and the one loaded and I was loading the specification after | > | > | installing the repository | > | > | and the working copies using the MCVersionLoader and it | > | > | worked I | > | > | could reload completely Moose! | > | > | | > | > | | > | > | load: aCollectionOfPackageString | > | > | "self new | > | > | load: #( 'Famix-C-TudorGirba.15' | > | > | 'Famix-Core-NicolasAnquetil.201'))" | > | > | | > | > | | loader | | > | > | loader := MCVersionLoader new. | > | > | (self newerVersionsIn: aCollectionOfPackageString) | > | > | do: [ :fn | loader addVersion: (self | > | > | loadVersionFromFileNamed: | > | > | fn). | > | > | (Delay forSeconds: 2) wait.] | > | > | displayingProgress: 'Adding versions...'. | > | > | loader hasVersions ifTrue: [loader load]. | > | > | | > | > | | > | > | So my question is | > | > | - Does the packageDirectivesDo: provide an order that I can | > | > | use | > | > | to | > | > | load package with the gofer-load on project at a time? | > | > | May be my code does not keep the same order but I payed | > | > | attention to | > | > | this point. I will check that further. | > | > | | > | > | - may be I could extend Gofer to support multiple project | > | > | merge | > | > | loading. | > | > | | > | > | So right now I'm stuck and this is sad because freezing a | > | > | complex | > | > | configuration is really needed. | > | > | | > | > | Stef | > | > | | > | > | | > | > | | > | > | | > | > | | > | > | | > | > | > | | > | -- | > | www.tudorgirba.com | > | | > | "It's not what we do that matters most, it's how we do it." | > | | > | | > | | > |