I understand that since the whole development will be Git-based, then the configurations will be pretty much obsolete. Alexandre

On Apr 15, 2017, at 6:08 PM, Tudor Girba <tudor@tudorgirba.com> wrote:

Hi,

I do not quite understand the benefit of removing Configurations in Pharo 6. Is there a benefit that I do not see?

Cheers, Doru

...

On Apr 14, 2017, at 2:24 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-04-14 14:20 GMT+02:00 Andrei Chis <chisvasileandrei@gmail.com>:

On Apr 14, 2017 14:13, "Pavel Krivanek" <pavel.krivanek@gmail.com> wrote:

2017-04-14 12:52 GMT+02:00 Andrei Chis <chisvasileandrei@gmail.com>: Isn't it a bit too late for such a change? Might break projects that expect configurations to be present.

I do not think so. We do not have configurations for the system itself (well, we have them in an external repository but they are not update since switch to baselines). They are used only for the actively maintained semi-external projects like GT, QA, Epicea, Zinc etc. However even in that case they are sometimes out-of-sync with the upstream.

Most users of such configurations are using the upstream configurations. If some one is using wrongly the versions in Pharo, he uses reference to the Pharo repository where the configurations will still be present.

For the record, I still think it's not the best idea to do this now :)

Probably :-) In the end, the users that have troubles with it can easily do it during preparations of their images too.

-- Pavel

Cheers, Andrei

Cheers, -- Pavel

Cheers, Andrei

On Fri, Apr 14, 2017 at 1:07 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote: Hi,

in Pharo 7 all configurations will be removed and replaced with the baselines. The bootstrapped image itself does not load the configurations at all. Most of them is now outdated and old versions of configurations in the standard Pharo image can cause problems for users of newer versions semi-internal packages (like Moose-Algos).

So we, me and Cyril think that we should remove all current Configurations from the image just before the Pharo 6 release. Do you see any disadvantages of this step?

Cheers, -- Pavel

-- www.tudorgirba.com www.feenk.com

"Reasonable is what we are accustomed with."

-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

On 15/04/2017 23:08, Tudor Girba wrote:

Hi,

I do not quite understand the benefit of removing Configurations in Pharo 6. Is there a benefit that I do not see?

Cheers, Doru

If you have a project depending on another project that got his configuration in the image, the configuration will not be updated if you do not do it explicitly. We found with Pavel that the version of MooseAlgo used by Moose was outdated because ConfigurationOfMooseAlgos was in the image with 5-6 commits late. Users should not have to update the configuration of the dependencies of their project by hand in their builds because Pharo has old configurations inside the image. -- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques Prévert 01, 59650 Villeneuve d'ascq France

On 15/04/2017 23:16, Tudor Girba wrote:

Hi,

Ok, that is a better reason.

As far as I can tell, the baseline situation is not much better, is it?

Cheers, Doru

Yes. If the process would have been the same for Pharo 7 than for Pharo 6 I would have suggest to unload the configurations after every update of a project using configuration. Here, since the process will change I wait to see how the new process will look like. But I would like the configurations/baselines to be unload of Pharo to avoid this. I would like to unload all the configurations of the image of Pharo 6 to remove a potential bug source of Pharo 6 for future users who would need an update of GlamourCore, a GT tool, Ston, UFFI or any other project having his configuration in the image. -- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques Prévert 01, 59650 Villeneuve d'ascq France

However, Configurations are useful in offering people a way to understand how the code is organized. For example, in Moose we have the inspector extension that shows the dependencies and it is very valuable. The only thing we need is to Metacello to be able to load new versions of the configuration/baseline. Cheers, Doru

On Apr 16, 2017, at 8:36 AM, Tudor Girba <tudor@tudorgirba.com> wrote:

Hi,

For the record, the way we load the latest version of Moose is to explicitly load the configurations we know about as being in the base Pharo image. Like this:

./pharo $JOB_NAME.image eval "Gofer new smalltalkhubUser: 'Moose' project: 'Glamour'; package: 'ConfigurationOfGlamourCore'; load. Gofer new smalltalkhubUser: 'Moose' project: 'GToolkit'; package: 'ConfigurationOfGTInspector'; package: 'ConfigurationOfGTInspectorCore'; package: 'ConfigurationOfGTSpotter'; package: 'ConfigurationOfGTPlayground'; package: 'ConfigurationOfGTPlaygroundCore'; package: 'ConfigurationOfGToolkit'; package: 'ConfigurationOfGToolkitCore'; package: 'ConfigurationOfGTEventRecorder'; load. Smalltalk snapshot: true andQuit: true."

This is a horrible hack, and like any hack it backfires in time. It backfired now because we forgot about MooseAlgos being in the base Pharo image :).

Cheers, Doru

...

On Apr 15, 2017, at 11:23 PM, Cyril Ferlicot D. <cyril.ferlicot@gmail.com> wrote:

On 15/04/2017 23:16, Tudor Girba wrote:

...

Hi,

Ok, that is a better reason.

As far as I can tell, the baseline situation is not much better, is it?

Cheers, Doru

Yes.

If the process would have been the same for Pharo 7 than for Pharo 6 I would have suggest to unload the configurations after every update of a project using configuration.

Here, since the process will change I wait to see how the new process will look like. But I would like the configurations/baselines to be unload of Pharo to avoid this.

I would like to unload all the configurations of the image of Pharo 6 to remove a potential bug source of Pharo 6 for future users who would need an update of GlamourCore, a GT tool, Ston, UFFI or any other project having his configuration in the image.

-- Cyril Ferlicot https://ferlicot.fr

http://www.synectique.eu 2 rue Jacques Prévert 01, 59650 Villeneuve d'ascq France

-- www.tudorgirba.com www.feenk.com

"Problem solving should be focused on describing the problem in a way that makes the solution obvious."

-- www.tudorgirba.com www.feenk.com “The smaller and more pervasive the hardware becomes, the more physical the software gets."

On 16/04/17 16:14, Dimitris Chloupis wrote:

Just for the record the easiest way to load packages in the image the Package Browser relies solely on configurations . Is there a plan to migrate because as much I am vocal supporter of Pharo moving to git it will be a big lose if Package Browser is not ported .

Which browser do you mean? Catalog? That generally does not work well because it cannot deal with combinations of configurations. Stephan

On 16/04/2017 08:54, Tudor Girba wrote:

However, Configurations are useful in offering people a way to understand how the code is organized. For example, in Moose we have the inspector extension that shows the dependencies and it is very valuable.

The only thing we need is to Metacello to be able to load new versions of the configuration/baseline.

Hi, This is already possible via the method #get. But, IMO, the user should not have to do that with a fresh Pharo image.

Cheers, Doru

-- www.tudorgirba.com www.feenk.com

“The smaller and more pervasive the hardware becomes, the more physical the software gets."

-- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques Prévert 01, 59650 Villeneuve d'ascq France

Hi Dale, Le 17/04/2017 à 21:05, Dale Henrichs a écrit :

I would think that a `project list` view that made the Metacello project registration visible would help developers keep things straight.

It seems that the issue here is that developers can't tell what projects are already loaded in the current image and also cannot tell what version of the project is loaded ... if you are using the `Metacello new` to load projects, then Metacello knows what projects and what versions are loaded in the image .... and that informatation really needs to be exposed to the developers ...

If you have a `project list` then you can do things like automatically do a get on a configuration/baseline when a project is loaded via the `project list tool` ... there are additional details that need to be tracked and managed, but without the a basic `project list` the developer is responsible for "knowing what to do" and the first step is to let the developer know exactly which versions of which projects are loaded in the base image ...

I wrote some code to that effect in the AltBrowser (gives access to the project, and all packages loaded via that project, via a Metacello registry tag). It didn't work as well as I expected, in part due to the interaction between the Metacello registry and package loading into the image. Nothing undoable, but due to the total lack of Metacello state in the default Pharo image (nothing is loaded via Metacello in there) and the Catalog browser not using Metacello as well, it was a bit too early to invest into that. In short, the Metacello registry provides a nice entry point for a system browser view of loaded projects and versions, but the Pharo image building process is not yet there. If someone is interested, one need to look into the GT extensions for Metacello objects ... those extensions are a nice source of knowledge about extracting information from the loaded projects. Regards, Thierry

Dale

On 04/16/2017 11:46 AM, Cyril Ferlicot D. wrote:

...

On 16/04/2017 08:54, Tudor Girba wrote:

...

However, Configurations are useful in offering people a way to understand how the code is organized. For example, in Moose we have the inspector extension that shows the dependencies and it is very valuable.

The only thing we need is to Metacello to be able to load new versions of the configuration/baseline.

Hi,

This is already possible via the method #get.

But, IMO, the user should not have to do that with a fresh Pharo image.

...

Cheers, Doru

-- www.tudorgirba.com www.feenk.com

“The smaller and more pervasive the hardware becomes, the more physical the software gets."

Le 17/04/2017 à 22:13, Dale Henrichs a écrit :

On 04/17/2017 12:47 PM, Thierry Goubier wrote:

...

Hi Dale,

Le 17/04/2017 à 21:05, Dale Henrichs a écrit :

...

I would think that a `project list` view that made the Metacello project registration visible would help developers keep things straight.

It seems that the issue here is that developers can't tell what projects are already loaded in the current image and also cannot tell what version of the project is loaded ... if you are using the `Metacello new` to load projects, then Metacello knows what projects and what versions are loaded in the image .... and that informatation really needs to be exposed to the developers ...

If you have a `project list` then you can do things like automatically do a get on a configuration/baseline when a project is loaded via the `project list tool` ... there are additional details that need to be tracked and managed, but without the a basic `project list` the developer is responsible for "knowing what to do" and the first step is to let the developer know exactly which versions of which projects are loaded in the base image ...

I wrote some code to that effect in the AltBrowser (gives access to the project, and all packages loaded via that project, via a Metacello registry tag).

It didn't work as well as I expected, in part due to the interaction between the Metacello registry and package loading into the image. Nothing undoable, but due to the total lack of Metacello state in the default Pharo image (nothing is loaded via Metacello in there) and the Catalog browser not using Metacello as well, it was a bit too early to invest into that.

In short, the Metacello registry provides a nice entry point for a system browser view of loaded projects and versions, but the Pharo image building process is not yet there.

If someone is interested, one need to look into the GT extensions for Metacello objects ... those extensions are a nice source of knowledge about extracting information from the loaded projects. This is unfortunate ... I'm willing to help anyone who wants to tackle this problem as well --- I know a thing or two about Metacello myself and I've had a working `project list` in tODE for 3-4 years now ... oh well ...

The code isn't lost anyway. I've reintegrated it as classifying based on exiting ConfigurationOf / BaselineOf objects, relying on the spec if there is no registering in Metacello (kind of funky in the case of a ConfigurationOf, since you don't know which version was effectively loaded... so you just play guesses with the various versions available in the project). At least, now I can easily see directly in the browser how the Pharo 6 development is moving packages inside configurations and baselines from version to version. I'll tackle next integrating that with Metacello, that is running project-based Metacello commands using that project structure. Thierry

Dale

On 04/24/2017 01:49 PM, Thierry Goubier wrote:

Le 17/04/2017 à 22:13, Dale Henrichs a écrit :

...

On 04/17/2017 12:47 PM, Thierry Goubier wrote:

...

Hi Dale,

Le 17/04/2017 à 21:05, Dale Henrichs a écrit :

...

I would think that a `project list` view that made the Metacello project registration visible would help developers keep things straight.

It seems that the issue here is that developers can't tell what projects are already loaded in the current image and also cannot tell what version of the project is loaded ... if you are using the `Metacello new` to load projects, then Metacello knows what projects and what versions are loaded in the image .... and that informatation really needs to be exposed to the developers ...

If you have a `project list` then you can do things like automatically do a get on a configuration/baseline when a project is loaded via the `project list tool` ... there are additional details that need to be tracked and managed, but without the a basic `project list` the developer is responsible for "knowing what to do" and the first step is to let the developer know exactly which versions of which projects are loaded in the base image ...

I wrote some code to that effect in the AltBrowser (gives access to the project, and all packages loaded via that project, via a Metacello registry tag).

It didn't work as well as I expected, in part due to the interaction between the Metacello registry and package loading into the image. Nothing undoable, but due to the total lack of Metacello state in the default Pharo image (nothing is loaded via Metacello in there) and the Catalog browser not using Metacello as well, it was a bit too early to invest into that.

In short, the Metacello registry provides a nice entry point for a system browser view of loaded projects and versions, but the Pharo image building process is not yet there.

If someone is interested, one need to look into the GT extensions for Metacello objects ... those extensions are a nice source of knowledge about extracting information from the loaded projects. This is unfortunate ... I'm willing to help anyone who wants to tackle this problem as well --- I know a thing or two about Metacello myself and I've had a working `project list` in tODE for 3-4 years now ... oh well ...

The code isn't lost anyway. I've reintegrated it as classifying based on exiting ConfigurationOf / BaselineOf objects, relying on the spec if there is no registering in Metacello (kind of funky in the case of a ConfigurationOf, since you don't know which version was effectively loaded... so you just play guesses with the various versions available in the project). Yeah, bootstrapping the registration is dicey business, because you can fall prey to the very bugs you are trying to avoid --- Metacello does a prime registry as a project load doit when the registration code is initially ... you might look at MetacelloProjectRegistry>>primeRegistryFromImage:baselineClasses:prioritizeConfiguration:, but it sounds like you are doing something very similar ...

At least, now I can easily see directly in the browser how the Pharo 6 development is moving packages inside configurations and baselines from version to version. I'll tackle next integrating that with Metacello, that is running project-based Metacello commands using that project structure. Let me know if there are any operations that you should think could/should be included in Metacello itself ... If you are doing things that are similar to what I'm doing in tODE, then it probably makes sense to move the method into Metacello, so that other tool builders cna share common code ...

Dale

Thierry

...

Dale

On 4/24/17 9:40 PM, Thierry Goubier wrote:

Hi Dale,

Le 25/04/2017 à 00:02, Dale Henrichs a écrit :

...

On 04/24/2017 01:49 PM, Thierry Goubier wrote:

...

Le 17/04/2017 à 22:13, Dale Henrichs a écrit :

...

On 04/17/2017 12:47 PM, Thierry Goubier wrote:

...

Hi Dale,

Le 17/04/2017 à 21:05, Dale Henrichs a écrit :

...

I would think that a `project list` view that made the Metacello project registration visible would help developers keep things straight.

It seems that the issue here is that developers can't tell what projects are already loaded in the current image and also cannot tell what version of the project is loaded ... if you are using the `Metacello new` to load projects, then Metacello knows what projects and what versions are loaded in the image .... and that informatation really needs to be exposed to the developers ...

If you have a `project list` then you can do things like automatically do a get on a configuration/baseline when a project is loaded via the `project list tool` ... there are additional details that need to be tracked and managed, but without the a basic `project list` the developer is responsible for "knowing what to do" and the first step is to let the developer know exactly which versions of which projects are loaded in the base image ...

I wrote some code to that effect in the AltBrowser (gives access to the project, and all packages loaded via that project, via a Metacello registry tag).

It didn't work as well as I expected, in part due to the interaction between the Metacello registry and package loading into the image. Nothing undoable, but due to the total lack of Metacello state in the default Pharo image (nothing is loaded via Metacello in there) and the Catalog browser not using Metacello as well, it was a bit too early to invest into that.

In short, the Metacello registry provides a nice entry point for a system browser view of loaded projects and versions, but the Pharo image building process is not yet there.

If someone is interested, one need to look into the GT extensions for Metacello objects ... those extensions are a nice source of knowledge about extracting information from the loaded projects. This is unfortunate ... I'm willing to help anyone who wants to tackle this problem as well --- I know a thing or two about Metacello myself and I've had a working `project list` in tODE for 3-4 years now ... oh well ...

The code isn't lost anyway. I've reintegrated it as classifying based on exiting ConfigurationOf / BaselineOf objects, relying on the spec if there is no registering in Metacello (kind of funky in the case of a ConfigurationOf, since you don't know which version was effectively loaded... so you just play guesses with the various versions available in the project). Yeah, bootstrapping the registration is dicey business, because you can fall prey to the very bugs you are trying to avoid --- Metacello does a prime registry as a project load doit when the registration code is initially ... you might look at MetacelloProjectRegistry>>primeRegistryFromImage:baselineClasses:prioritizeConfiguration:,

but it sounds like you are doing something very similar ...

Hum. I'm not trying to determine a version for the configurations present in the image... and #primeRegistryFromImage seems to, and then I seem to get errors because it selects a wrong version (at least I got an error doing that with ConfigurationOfEpicea, set at version 7.8.p4 where it should be version 8.1.3, and Metacello fails with a version not found).

I'll look a bit deeper to see what is happening and how to reproduce it. It is very likely that you are running into the very type of bug that caused me to go with the Metacello registry in the first place ... It can be impossible for Metacello to guess the correct version of a loaded configuration ... I'm not sure it is worth much effort to try to characterize the problem ... better would be for Pharo to use "Metacello new" to build the image, so that the loaded versions are registered in the first place ...

If you insist on looking deeper:) a common cause of wrong guesses is that the configuration specifies a project version dependency or a version of package that is later than the versions actually loaded in the image --- this can happen when only a partial load of the configuration is done where those projects or packages are not actually loaded by the configuration itself --- without the actual registration information from the load, Metacello looks at the loaded projects/packages and tries to match it with a version of the configuration and this "false" dependency information causes Metacello to look for an earlier version that matches ... Dale

On 04/25/2017 08:09 AM, Thierry Goubier wrote:

2017-04-25 16:11 GMT+02:00 Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>>:

[ Shortened for brevity ... ]

Hum. I'm not trying to determine a version for the configurations present in the image... and #primeRegistryFromImage seems to, and then I seem to get errors because it selects a wrong version (at least I got an error doing that with ConfigurationOfEpicea, set at version 7.8.p4 where it should be version 8.1.3, and Metacello fails with a version not found).

I'll look a bit deeper to see what is happening and how to reproduce it.

It is very likely that you are running into the very type of bug that caused me to go with the Metacello registry in the first place ... It can be impossible for Metacello to guess the correct version of a loaded configuration ... I'm not sure it is worth much effort to try to characterize the problem ... better would be for Pharo to use "Metacello new" to build the image, so that the loaded versions are registered in the first place ...

Of course...

Now, I just compute the closure of all the configuration version specs over the set of packages present in the image, by name. If a package name is present in one of the version specs, then it is considered as part of the project.

If you insist on looking deeper:) a common cause of wrong guesses is that the configuration specifies a project version dependency or a version of package that is later than the versions actually loaded in the image --- this can happen when only a partial load of the configuration is done where those projects or packages are not actually loaded by the configuration itself --- without the actual registration information from the load, Metacello looks at the loaded projects/packages and tries to match it with a version of the configuration and this "false" dependency information causes Metacello to look for an earlier version that matches ...

Yes, this is the reason. In the case of Epicea in the latest image, the last version is 8.1.3, but ConfigurationOfEpicea>>#currentVersion returns 7.8p4.

7.8p4 is the highest version for which there is a dependency on the STON project, dependency which is respected in the image (version requested is 0.17, version present is 0.23). 7.8p4 and all higher versions also have a dependency on Smark, which isn't loaded.

So Smark is the "culprit". Since Epicea is loaded and functioning, the Smark project is not absolutely required, but since there is a dependency in the Configuration Metacello doesn't know that the project isn't strictly required ... I would be academically interested in the dependency on Smark ... it is possible the Smark is just in the list of projects, which makes it part of the default list ... since old-style Metacello loads do not record the "loads list" for a project, Metacello has to assume that the default list is loaded ... which could make Smark required ...

Would that mean that: - If one of the project in requirements is present in the image, then the version is considered current (here, for 7.8p4, Ston is present, Smark is not) if, for the packages of the project, all are present with versions higher than the current configuration version spec?

Yes. When calculating the current version, Metacello can only assume that the default list is loaded ... I considered "improving" the current version calculation, but I knew that actually recording the exact project specification that was loaded was far superior to any complicated guessing game that I could devise for Metacello and I took that route ... besides the current version calculation can be extremely expensive ...

- If none of the projects in requirements are present in the image, then the version is not considered current (in 8.1.3, Smark is not present but I see no dependency on Ston[*]) ? Yep ... for old-style loading ...

I have a feeling this is a strange result, and I'd prefer 8.1.3 to have the same validity as 7.8p4. ... I prefer to record the exact project spec that is loaded ... I don't think it is possible to create a current version algorithm that is superior to recording the exact version that is loaded ... when it is loaded ...

Is this MetacelloProject>>#currentVersion used to determine if a project should be upgraded? Yes ... Remember, we are talking old-style Metacello loading here ... #currentVersion is only used when one does not use `Metacello new` style loading and old-style Metacello loading has been obsolete for 4 years ...

Another question, related to the original topic: if the current configurations in the image are in this state, is it a good idea to leave them in? Would it have a chance of triggering incorrect updates or suspicious errors[**] if we start using them to load Metacello projects that depends on them? With new style loading, if a project is not registered, then Metacello assumes that the dependent project is not loaded at all and will proceed to load the project version as requested ... so I don't think that any incorrect updates should occur, but if folks have been playing fast and loose with configurations (loading individual packages from other projects without using a configuration or ???) then it is possible the new style loading will load a different set of projects and packages ... but it will do so consistently ... whereas the old-style loading could lead to inconsistent behavior ... presumably some developers have tried to work around the old-style loading errors by explicitly loading the package that they want without using a configuration ... so when using new-style loading, developers may have to "fix there configurations" --- but then that is probably a good thing ...

Dale