Hi Pablo. Dev branch approach not really works. Because any merge into master will break master baseline. (notice that baseline is in same repo). And managing merges by hand all the time is not a solution. 2018-03-05 15:57 GMT+01:00 tesonep@gmail.com <tesonep@gmail.com>:

Yes, I understand for the developer version, you should have a developer branch and use that. The problem is for the Pharo image only.

On Mon, Mar 5, 2018 at 3:39 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

...

Hi,

it is an issue of Calypso dependencies. I discussed it with Denis before Calypso integration and he would like to change it but, for now, it would make the development for him much harder because he needs a way how to load the latest bleeding edge versions of the packages and no one was able to tell him the proper way how to support both scenarios (loading of a specific release and loading of the development versions) with baselines without needing of duplicate them.

Anyway, I agree that the current state must be fixed as soon as possible.

Cheers, -- Pavel

2018-03-05 15:30 GMT+01:00 tesonep@gmail.com <tesonep@gmail.com>:

...

Hello, i have seen in the latest version of Pharo baselines pointing to "floating" versions. A version that is not fixed. I want to know why this is like that?

To me this is basically wrong, because basically this goes against the idea of having repeatable builds. Also using semantic versions with tags in Git is wrong, as tags should be immutable (that is why you have to use --force to update them, in Guille's words "If you have to force it, it is not good").

Cheers,

-- Pablo Tesone. tesonep@gmail.com

-- Pablo Tesone. tesonep@gmail.com

2018-03-05 16:13 GMT+01:00 Cyril Ferlicot <cyril.ferlicot@gmail.com>:

On Mon, Mar 5, 2018 at 4:10 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

Hi Pablo.

Dev branch approach not really works. Because any merge into master will break master baseline. (notice that baseline is in same repo). And managing merges by hand all the time is not a solution.

Hi! If you don't want to manage the merges by hand you can maybe have two bsaelines? BaselineOfCalypso and BaselineOfCalypsoDev?

It should work. But is it right way that everybody should follow? With configurations it was easy to do in single class.

-- Cyril Ferlicot https://ferlicot.fr

In Configuration I defined two baselines: dev and stable. In dev baseline I referenced dev version of dependencies. In stable I referenced stable version of dependencies. 2018-03-05 16:38 GMT+01:00 Guillermo Polito <guillermopolito@gmail.com>:

But, "one single class" does not mean anything. Because it depends from which branch/commitish you are loading it from...

Can you explain better what is the problem because I am not getting it...

In any case, independently of where is the burden, I want to veto any new integration that may make future builds non-reproducible. Otherwise this is a source of chaos and dead kittens.

On Mon, Mar 5, 2018 at 4:17 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

2018-03-05 16:13 GMT+01:00 Cyril Ferlicot <cyril.ferlicot@gmail.com>:

...

On Mon, Mar 5, 2018 at 4:10 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

Hi Pablo.

Dev branch approach not really works. Because any merge into master will break master baseline. (notice that baseline is in same repo). And managing merges by hand all the time is not a solution.

Hi! If you don't want to manage the merges by hand you can maybe have two bsaelines? BaselineOfCalypso and BaselineOfCalypsoDev?

It should work. But is it right way that everybody should follow?

With configurations it was easy to do in single class.

...

-- Cyril Ferlicot https://ferlicot.fr

--

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>*

*Web:* *http://guillep.github.io* <http://guillep.github.io>

*Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

On Mon, Mar 5, 2018 at 4:45 PM, Cyril Ferlicot <cyril.ferlicot@gmail.com>
wrote:

>
> On Mon, Mar 5, 2018 at 4:38 PM, Guillermo Polito <
> guillermopolito@gmail.com> wrote:
>
>> But, "one single class" does not mean anything. Because it depends from
>> which branch/commitish you are loading it from...
>>
>> Can you explain better what is the problem because I am not getting it...
>>
>> In any case, independently of where is the burden, I want to veto any new
>> integration that may make future builds non-reproducible. Otherwise this is
>> a source of chaos and dead kittens.
>>
>> The problem is that actually if you have a git project with a stable and
> dev branch you will need:
> - One baseline on the stable branch with determined dependencies versions
> - One baseline on the dev branch with less restricted versions (for
> example 3.x.x)
> Then, when you release a new stable version by merging the development
> branch you need to edit the merge by hand to merge everything except the
> baseline.
> I think this is what Denis wants to avoid.
>
But in general, such merges need to be done only for major and minor
> releases. Not for a patch because you can do a hotfix. You create a new
> branch from master, then merge in in master and development when it's done.
>

I still do not understand... Also I do not understand your usage of the
term "the merge". Can somebody give a **concrete** scenario?

When you release a version, please do not move that version. You should
then create new versions, with new numbers and new code. But never touch
old versions with old numbers and old code. Like that I can download the
same old code using the same old number to get the old version whenever I
want :).

You can try to do it with branches, tags, different repositories, or even
with zipfiles in mails. I don't care. Just don't modify releases and I'm
happy with it.




>
> --
> Cyril Ferlicot
> https://ferlicot.fr
>



-- 



Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - *http://www.cnrs.fr
<http://www.cnrs.fr>*


*Web:* *http://guillep.github.io* <http://guillep.github.io>

*Phone: *+33 06 52 70 66 13

2018-03-05 17:02 GMT+01:00 Cyril Ferlicot <cyril.ferlicot@gmail.com>:

On Mon, Mar 5, 2018 at 4:51 PM, Guillermo Polito < guillermopolito@gmail.com> wrote:

...

I still do not understand... Also I do not understand your usage of the term "the merge". Can somebody give a **concrete** scenario?

I'll try

For example the project MaterialDesignLite have two branches that are always here: - master - development

Each commit on master is a stable release and end up with a tag as

"v1.2.2".

I do not agree with this sentence:

Since it's stable, BaselineOfMaterialDesignLite should depend only on fixed version of the dependencies. For example it should depend on MaterialDesignColor "v1.0.0".

Maybe I am wrong of course. But this is what I expect from semantic versioning: If I made project which uses some libraries I want to get latest fixes even in my stable version. This is what semantics versioning is about. Minor releases are supposed to be safe for dependent projects. It is of course question of trust to third parties. But if I do not trust them I should fork dependency and use it instead of original one. On the branch dev, we want to get the patches and possibly the minor

versions of the dependencies automatically. In the baseline we then want to depend on MaterialDesignColor "v1.x.x". Thus, we follow the changes of MaterialDesignColor while it's not a major release. Because of this situation, BaselineOfMaterialDesignLite is different on the two branches. Later, if I want to merge development into master in order to release a new version, master will get the BaselineOfMaterialDesignLite with semantic versionning dependencies instead of the fixed dependencies. Before the release I'll need to change the Baseline to get fix dependencies once again. I hope I was clearer. :)

...

When you release a version, please do not move that version. You should then create new versions, with new numbers and new code. But never touch old versions with old numbers and old code. Like that I can download the same old code using the same old number to get the old version whenever I want :).

You can try to do it with branches, tags, different repositories, or even with zipfiles in mails. I don't care. Just don't modify releases and I'm happy with it.

-- Cyril Ferlicot https://ferlicot.fr

Yes and no. I think I kind of get it now from all comments... There are two problems together, one is how to reproduce the loading of a project (or getting a working version), and the other is how to use metacello also to manage the development of the project when the dependencies feel more like subprojects that may be edited altogether... On one side there is the reproducibility problem. I understand Stephan's and Nicolas' points. Using exact versions may block development. However, I see also that people in general have problems using semantic versioning and getting working versions done. This morning the image was broken because of wrong but "not fixed" dependencies... This afternoon Pablo was blocked executing the bootstrap because iceberg's and the image version did not match and Metacello was not the best friend when resolving a conflict on a package that did not exist on one of the two conflicting versions... So yes, it may block upgrades, but until we have tools that allow us to cope with the complexity, I prefer to have reproducible versions where I can reproduce bugs in a reproducible way than an unpredictable version where I cannot grasp the cause of a problem :/. On the other side, there is the fact that Metacello baselines are so far nice to describe release project dependencies, but they are not so nice to describe subprojects/development dependencies that may get edited along with the parent project. Kind of what we used to do with #bleedingEdge. I feel this is a complex problem, that not even SBT or maven that are there since years are capable of solving nicely... Tode and Iceberg metacello integration try to solve this problem by "ignoring the dependency and using the loaded repository" but this may not be successful either... Now, pushing the complexity to how we manage the Pharo repository is not the solution either :) On Mon, Mar 5, 2018 at 5:51 PM, Nicolas Cellier < nicolas.cellier.aka.nice@gmail.com> wrote:

The well known problem with fixed configurations is dependencies:

My project A version 1.2.3 depends on project C version 4.3 (semantic versioning). I have tested it with 4.3.35, it works well... If semantic versioning is correctly used, it should work with any 4.x.y where x>=3. There is another project B version 2.1.4 which depends on project C version 4.4 (still semantic).

Loading latest patch 4.4.20 should work for both A & B. And even 4.6.11 which is the latest compatible version in 4.x serie

IMO, the package ConfigurationOfPackageA and ConfigurationOfPackageB should NOT specify the exact version of package C, but only the minimal compatible version. Then, if you want a reproducible image, that is up to the specific assembly of package A and package B (let's call it ConfigurationOfPackageAandBandC) that you should force the specific C version.

Isn't it the problem?

2018-03-05 17:17 GMT+01:00 Denis Kudriashov <dionisiydk@gmail.com>:

...

2018-03-05 17:02 GMT+01:00 Cyril Ferlicot <cyril.ferlicot@gmail.com>:

...

On Mon, Mar 5, 2018 at 4:51 PM, Guillermo Polito < guillermopolito@gmail.com> wrote:

...

I still do not understand... Also I do not understand your usage of the term "the merge". Can somebody give a **concrete** scenario?

I'll try

For example the project MaterialDesignLite have two branches that are always here: - master - development

Each commit on master is a stable release and end up with a tag as

...

"v1.2.2".

I do not agree with this sentence:

...

Since it's stable, BaselineOfMaterialDesignLite should depend only on fixed version of the dependencies. For example it should depend on MaterialDesignColor "v1.0.0".

Maybe I am wrong of course. But this is what I expect from semantic versioning:

If I made project which uses some libraries I want to get latest fixes even in my stable version. This is what semantics versioning is about. Minor releases are supposed to be safe for dependent projects. It is of course question of trust to third parties. But if I do not trust them I should fork dependency and use it instead of original one.

On the branch dev, we want to get the patches and possibly the minor

...

versions of the dependencies automatically. In the baseline we then want to depend on MaterialDesignColor "v1.x.x". Thus, we follow the changes of MaterialDesignColor while it's not a major release. Because of this situation, BaselineOfMaterialDesignLite is different on the two branches. Later, if I want to merge development into master in order to release a new version, master will get the BaselineOfMaterialDesignLite with semantic versionning dependencies instead of the fixed dependencies. Before the release I'll need to change the Baseline to get fix dependencies once again. I hope I was clearer. :)

...

When you release a version, please do not move that version. You should then create new versions, with new numbers and new code. But never touch old versions with old numbers and old code. Like that I can download the same old code using the same old number to get the old version whenever I want :).

You can try to do it with branches, tags, different repositories, or even with zipfiles in mails. I don't care. Just don't modify releases and I'm happy with it.

-- Cyril Ferlicot https://ferlicot.fr

-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Guillermo Polito <guillermopolito@gmail.com> wrote:

...

So yes, it may block upgrades, but until we have tools that allow us to cope with the complexity, I prefer to have reproducible versions where I can reproduce bugs in a reproducible way than an unpredictable version where I cannot grasp the cause of a problem :/.

I strongly advice against using Baselines for that. They are supposed to be edited from hand, and using them for reproducible versions will break other projects, as we've seen earlier with Configurations. AFAIK Metacello can register which versions were loaded in which order, so that information should be recorded for reproducibility. Stephan

2018-03-05 18:23 GMT+01:00 Stephan Eggermont <stephan@stack.nl>:

Guillermo Polito <guillermopolito@gmail.com> wrote:

...

...

So yes, it may block upgrades, but until we have tools that allow us to cope with the complexity, I prefer to have reproducible versions where I can reproduce bugs in a reproducible way than an unpredictable version where I cannot grasp the cause of a problem :/.

I strongly advice against using Baselines for that. They are supposed to be edited from hand, and using them for reproducible versions will break other projects, as we've seen earlier with Configurations. AFAIK Metacello can register which versions were loaded in which order, so that information should be recorded for reproducibility.

It is clear that we need both baselines for that case. But because we will move Calypso to Pharo repo I think it is easy to manage strong versions directly in Pharo loading script for now (in #loadCalypso method).

Stephan

On 03/05/2018 09:07 AM, Guillermo Polito wrote:

On the other side, there is the fact that Metacello baselines are so far nice to describe release project dependencies, but they are not so nice to describe subprojects/development dependencies that may get edited along with the parent project. Kind of what we used to do with #bleedingEdge. I feel this is a complex problem, that not even SBT or maven that are there since years are capable of solving nicely... Tode and Iceberg metacello integration try to solve this problem by "ignoring the dependency and using the loaded repository" but this may not be successful either...

I have not been following this thread in detail, but this comment leads me to believe that the issue that you guys are running into has to do with trying to ongoing distributed development across multiple projects and multiple platform versions ... If so then I think that the moral equivalent of #bleedingEdge should be a branch structure along the lines of:   master branch --- the project release that is known to work on all supported platform versions.   dev_pharo6      --- the current #bleedingEdge for Pharo6.0   dev_pharo7      --- the current #bleedingEdge for Pharo7.0 In an image where you want to use the dev_pharo7 for a project, you do a Metacello lock on github://xxxx/xxx:dev_pharo7/src BEFORE loading the project ... if there are multiple projects that need to all coordinate development then you follow the same convention and use a Metacello lock for each of the projects ... Executing expressions to lock projects is tedious to use and manage. In tODE I started using project entries[1] that are downloaded to disk and shared by multiple images that allow a developer to specify what branch/tag/commit they are interested in using ... each image that is started up in a "cluster" uses the project entry to do a Metacello lock on the project "automagically"... if there is a series of projects where the developer needs to use dev_pharo6 they can arrange to edit the project entry to reflect the proper branches for the set of projects that need special _development_ treatment ... these .ston files can be shared amongst groups of developers ... of course once the local clones of these projects are on the local disk, then it is up to the developer (or a set of scripts) to periodically update to the latest development version of each of the projects ... To share a specific configuration of development git repos, you need to know that SHA anyway, so the image can produce a collection of project entries with the SHA of the commit and those project entries can be included in a bug report ... For Rowan[2], I am using a "load specification"[3], which is a second generation project entry .... the Rowan work is still evolving ... Anyway, I think that putting the control of what is being used for development completely in the developers hands and avoiding the need to edit baselines is a good thing ... and I think that the idea of having specifications that are shared by a cluster of images is a good way to  go ... Dale [1] https://github.com/GsDevKit/GsDevKit_home/blob/gh-pages/Seaside32.ston [2] https://github.com/dalehenrich/Rowan [3] https://github.com/dalehenrich/Rowan/blob/master/specs/Rowan.ston

On 03/05/2018 11:22 AM, Stephane Ducasse wrote:

...

In an image where you want to use the dev_pharo7 for a project, you do a Metacello lock on github://xxxx/xxx:dev_pharo7/src BEFORE loading the project ... if there are multiple projects that need to all coordinate development then you follow the same convention and use a Metacello lock for each of the projects ... Executing expressions to lock projects is tedious to use and manage. Do you have a text explaining what is a lock? I do not understand the problems locks solve. The "Lock Command Reference" hopefully does the job[1].

Dale [1] https://github.com/Metacello/metacello/blob/master/docs/LockCommandReference...

Hi Dale! On Mon, Mar 5, 2018 at 8:07 PM, Dale Henrichs < dale.henrichs@gemtalksystems.com> wrote:

On 03/05/2018 09:07 AM, Guillermo Polito wrote:

...

On the other side, there is the fact that Metacello baselines are so far nice to describe release project dependencies, but they are not so nice to describe subprojects/development dependencies that may get edited along with the parent project. Kind of what we used to do with #bleedingEdge. I feel this is a complex problem, that not even SBT or maven that are there since years are capable of solving nicely... Tode and Iceberg metacello integration try to solve this problem by "ignoring the dependency and using the loaded repository" but this may not be successful either...

I have not been following this thread in detail, but this comment leads me to believe that the issue that you guys are running into has to do with trying to ongoing distributed development across multiple projects and multiple platform versions ...

I don't know exactly the case of Calypso, but I don't think it is being maintained for other than latest Pharo 7. The Calypso case as I understand it is that it has a couple of dependencies Calypso | - commander | - class annotations The three projects are hosted in different repositories, but Denis wants to work on "bleeding edge" of the three of them at the same time. Then when he loads calypso in a new image, if he just uses a baseline he will load specific versions of the dependencies and not the latest ones... I think this is the particular use case for locking, isn't it?

If so then I think that the moral equivalent of #bleedingEdge should be a branch structure along the lines of:

master branch --- the project release that is known to work on all supported platform versions. dev_pharo6 --- the current #bleedingEdge for Pharo6.0 dev_pharo7 --- the current #bleedingEdge for Pharo7.0

In an image where you want to use the dev_pharo7 for a project, you do a Metacello lock on github://xxxx/xxx:dev_pharo7/src BEFORE loading the project ... if there are multiple projects that need to all coordinate development then you follow the same convention and use a Metacello lock for each of the projects ... Executing expressions to lock projects is tedious to use and manage.

It is not that complicated I think ^^.

In tODE I started using project entries[1] that are downloaded to disk and shared by multiple images that allow a developer to specify what branch/tag/commit they are interested in using ... each image that is started up in a "cluster" uses the project entry to do a Metacello lock on the project "automagically"... if there is a series of projects where the developer needs to use dev_pharo6 they can arrange to edit the project entry to reflect the proper branches for the set of projects that need special _development_ treatment ... these .ston files can be shared amongst groups of developers ... of course once the local clones of these projects are on the local disk, then it is up to the developer (or a set of scripts) to periodically update to the latest development version of each of the projects ...

YES. This gets closer to what I want. What I don't like from locking is that I have to: - know in advance all the projects (+ dependencies) I want to develop (specially if I'm not the main project developer) - know where they are stored in the disk - do an explicit locking on each of them There is a missing abstraction that probably you get in Rowan or tODE with that project entry specification? Because the reality is that personally I'm starting a new image every a couple of days, and having a specification for it would be much easier than locking here and there... I want just to specify "My project X at development time requires Y, Z, H and Z and H are subprojects". Then, he should realize he should lock Z and H for me. Now, If subprojects Z and H are in the same repository as X, that should not be complicated :). In Denis' case it could be more complicated...

To share a specific configuration of development git repos, you need to know that SHA anyway, so the image can produce a collection of project entries with the SHA of the commit and those project entries can be included in a bug report ...

For Rowan[2], I am using a "load specification"[3], which is a second generation project entry .... the Rowan work is still evolving ...

Anyway, I think that putting the control of what is being used for development completely in the developers hands and avoiding the need to edit baselines is a good thing ... and I think that the idea of having specifications that are shared by a cluster of images is a good way to go ...

Dale

[1] https://github.com/GsDevKit/GsDevKit_home/blob/gh-pages/Seaside32.ston [2] https://github.com/dalehenrich/Rowan [3] https://github.com/dalehenrich/Rowan/blob/master/specs/Rowan.ston

-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Thanks dale I will watch and read and sync with guys when I'm back to civilisation :) On Tue, Mar 6, 2018 at 6:35 PM, Dale Henrichs < dale.henrichs@gemtalksystems.com> wrote:

On 03/06/2018 06:21 AM, Guillermo Polito wrote:

Hi Dale!

On Mon, Mar 5, 2018 at 8:07 PM, Dale Henrichs < dale.henrichs@gemtalksystems.com> wrote:

...

On 03/05/2018 09:07 AM, Guillermo Polito wrote:

...

On the other side, there is the fact that Metacello baselines are so far nice to describe release project dependencies, but they are not so nice to describe subprojects/development dependencies that may get edited along with the parent project. Kind of what we used to do with #bleedingEdge. I feel this is a complex problem, that not even SBT or maven that are there since years are capable of solving nicely... Tode and Iceberg metacello integration try to solve this problem by "ignoring the dependency and using the loaded repository" but this may not be successful either...

I have not been following this thread in detail, but this comment leads me to believe that the issue that you guys are running into has to do with trying to ongoing distributed development across multiple projects and multiple platform versions ...

I don't know exactly the case of Calypso, but I don't think it is being maintained for other than latest Pharo 7.

The Calypso case as I understand it is that it has a couple of dependencies

Calypso | - commander | - class annotations

The three projects are hosted in different repositories, but Denis wants to work on "bleeding edge" of the three of them at the same time. Then when he loads calypso in a new image, if he just uses a baseline he will load specific versions of the dependencies and not the latest ones...

I think this is the particular use case for locking, isn't it?

Yes this is exactly the use case for locking.

...

If so then I think that the moral equivalent of #bleedingEdge should be a branch structure along the lines of:

master branch --- the project release that is known to work on all supported platform versions. dev_pharo6 --- the current #bleedingEdge for Pharo6.0 dev_pharo7 --- the current #bleedingEdge for Pharo7.0

In an image where you want to use the dev_pharo7 for a project, you do a Metacello lock on github://xxxx/xxx:dev_pharo7/src BEFORE loading the project ... if there are multiple projects that need to all coordinate development then you follow the same convention and use a Metacello lock for each of the projects ... Executing expressions to lock projects is tedious to use and manage.

It is not that complicated I think ^^.

I have found that "load specifications" are much easier to work with than "load expressions", similar to the way class definitions are superior to smalltalk code for creating classes, one can reason about a "load specification" for example it is possible to merge two load specifications to produce a single load specification whereas this kind of thing is not possible if you are given two arbitrary Smalltalk expressions to merge ...

...

In tODE I started using project entries[1] that are downloaded to disk and shared by multiple images that allow a developer to specify what branch/tag/commit they are interested in using ... each image that is started up in a "cluster" uses the project entry to do a Metacello lock on the project "automagically"... if there is a series of projects where the developer needs to use dev_pharo6 they can arrange to edit the project entry to reflect the proper branches for the set of projects that need special _development_ treatment ... these .ston files can be shared amongst groups of developers ... of course once the local clones of these projects are on the local disk, then it is up to the developer (or a set of scripts) to periodically update to the latest development version of each of the projects ...

YES. This gets closer to what I want. What I don't like from locking is that I have to: - know in advance all the projects (+ dependencies) I want to develop (specially if I'm not the main project developer) - know where they are stored in the disk - do an explicit locking on each of them

Haha, and that is why I created the project entries ("load specifications") in the first place:)

There is a missing abstraction that probably you get in Rowan or tODE with that project entry specification?

Yes, tODE project entries are an implementation that I have been using for several years now and over time I have found some limitations in the tODE implementation that I am attempting to correct in the Rowan implementation ... The Rowan spec I originally referenced[1] is actually "load expression" that is the equivalent of a Gofer load expression ... I have classes for a Metacello-based "load expression" but I haven't gotten to the point where I have integrated Metacello into Rowan.

[1] https://github.com/dalehenrich/Rowan/blob/master/specs/Rowan.ston

Because the reality is that personally I'm starting a new image every a couple of days, and having a specification for it would be much easier than locking here and there... I want just to specify "My project X at development time requires Y, Z, H and Z and H are subprojects". Then, he should realize he should lock Z and H for me.

Yes this is exactly the problem I was dealing with in the early implementation of tODE and with project entries.

You might want to look again at my FAST talk on "Dangerous Liaisons: Smalltalk, Files, And Git"[2] as I talk about "image clusters" that share a collection of project entries and the fact that each image needs to be able to have a private set of "project entries" ... now that you are experiencing the pain my talk might make more sense:)

... also I am thinking in terms of an "image specification" that is made up of a collection of Rowan "load specifications" ... Rowan has the notion of a Project Set and it is possible to load a set of projects. So it is not that big of a stretch to imagine a .ston image specification (collection load specifications) can be passed around and then customized if needed by individual developers and then project set can be used to create the image or images ....

[2] https://www.youtube.com/watch?v=Ejmqs0xLvSk

Now, If subprojects Z and H are in the same repository as X, that should not be complicated :). In Denis' case it could be more complicated...

Right but once Denis gets his dependent projects cloned to local disk he can update each of the projects on his schedule by pulling later versions of the projects as he goes ... I lean to disk-based clones that can be shared by multiple images (an "image cluster") ... tODE does not have the notion of image clusters, but I do have sets of images that are sharing a common set of projects where a number of those projects are on development-specific branches so Iknow that this concept makes sense ...

...

To share a specific configuration of development git repos, you need to know that SHA anyway, so the image can produce a collection of project entries with the SHA of the commit and those project entries can be included in a bug report ...

For Rowan[2], I am using a "load specification"[3], which is a second generation project entry .... the Rowan work is still evolving ...

Anyway, I think that putting the control of what is being used for development completely in the developers hands and avoiding the need to edit baselines is a good thing ... and I think that the idea of having specifications that are shared by a cluster of images is a good way to go ...

Dale

[1] https://github.com/GsDevKit/GsDevKit_home/blob/gh-pages/Seas ide32.ston [2] https://github.com/dalehenrich/Rowan [3] https://github.com/dalehenrich/Rowan/blob/master/specs/Rowan.ston

--

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>*

*Web:* *http://guillep.github.io* <http://guillep.github.io>

*Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

Hi Dale. Where to read about locking? 2018-03-05 18:57 GMT+01:00 Dale Henrichs <dale.henrichs@gemtalksystems.com>:

Cyril,

For development I tend to use Metacello locking instead of hard-wired dependencies in the BaselineOf and that works very well --- it completely avoids the need to edit a baseline for development purposes and this approach works really well for me ... perhaps we can discuss this in more detail in a separate thread or even private email? This all seems to be more complicated for you guys than has been my experience and of course you guys _appear_ to be completely ignoring the Metacello locking feature so I'm curious why ...

Dale

On 03/05/2018 08:02 AM, Cyril Ferlicot wrote:

On Mon, Mar 5, 2018 at 4:51 PM, Guillermo Polito < guillermopolito@gmail.com> wrote:

...

I still do not understand... Also I do not understand your usage of the term "the merge". Can somebody give a **concrete** scenario?

I'll try

For example the project MaterialDesignLite have two branches that are always here: - master - development Each commit on master is a stable release and end up with a tag as "v1.2.2". Since it's stable, BaselineOfMaterialDesignLite should depend only on fixed version of the dependencies. For example it should depend on MaterialDesignColor "v1.0.0". On the branch dev, we want to get the patches and possibly the minor versions of the dependencies automatically. In the baseline we then want to depend on MaterialDesignColor "v1.x.x". Thus, we follow the changes of MaterialDesignColor while it's not a major release. Because of this situation, BaselineOfMaterialDesignLite is different on the two branches. Later, if I want to merge development into master in order to release a new version, master will get the BaselineOfMaterialDesignLite with semantic versionning dependencies instead of the fixed dependencies. Before the release I'll need to change the Baseline to get fix dependencies once again. I hope I was clearer. :)

...

When you release a version, please do not move that version. You should then create new versions, with new numbers and new code. But never touch old versions with old numbers and old code. Like that I can download the same old code using the same old number to get the old version whenever I want :).

You can try to do it with branches, tags, different repositories, or even with zipfiles in mails. I don't care. Just don't modify releases and I'm happy with it.

-- Cyril Ferlicot https://ferlicot.fr

Dale no private emails please. stef On Mon, Mar 5, 2018 at 6:57 PM, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:

Cyril,

For development I tend to use Metacello locking instead of hard-wired dependencies in the BaselineOf and that works very well --- it completely avoids the need to edit a baseline for development purposes and this approach works really well for me ... perhaps we can discuss this in more detail in a separate thread or even private email? This all seems to be more complicated for you guys than has been my experience and of course you guys _appear_ to be completely ignoring the Metacello locking feature so I'm curious why ...

Dale

On 03/05/2018 08:02 AM, Cyril Ferlicot wrote:

On Mon, Mar 5, 2018 at 4:51 PM, Guillermo Polito <guillermopolito@gmail.com> wrote:

...

I still do not understand... Also I do not understand your usage of the term "the merge". Can somebody give a **concrete** scenario?

I'll try For example the project MaterialDesignLite have two branches that are always here: - master - development Each commit on master is a stable release and end up with a tag as "v1.2.2". Since it's stable, BaselineOfMaterialDesignLite should depend only on fixed version of the dependencies. For example it should depend on MaterialDesignColor "v1.0.0". On the branch dev, we want to get the patches and possibly the minor versions of the dependencies automatically. In the baseline we then want to depend on MaterialDesignColor "v1.x.x". Thus, we follow the changes of MaterialDesignColor while it's not a major release. Because of this situation, BaselineOfMaterialDesignLite is different on the two branches. Later, if I want to merge development into master in order to release a new version, master will get the BaselineOfMaterialDesignLite with semantic versionning dependencies instead of the fixed dependencies. Before the release I'll need to change the Baseline to get fix dependencies once again. I hope I was clearer. :)

...

When you release a version, please do not move that version. You should then create new versions, with new numbers and new code. But never touch old versions with old numbers and old code. Like that I can download the same old code using the same old number to get the old version whenever I want :).

You can try to do it with branches, tags, different repositories, or even with zipfiles in mails. I don't care. Just don't modify releases and I'm happy with it.

-- Cyril Ferlicot https://ferlicot.fr

On 5 March 2018 at 23:51, Guillermo Polito <guillermopolito@gmail.com>
wrote:

>
>
> On Mon, Mar 5, 2018 at 4:45 PM, Cyril Ferlicot <cyril.ferlicot@gmail.com>
> wrote:
>
>>
>> On Mon, Mar 5, 2018 at 4:38 PM, Guillermo Polito <
>> guillermopolito@gmail.com> wrote:
>>
>>> But, "one single class" does not mean anything. Because it depends from
>>> which branch/commitish you are loading it from...
>>>
>>> Can you explain better what is the problem because I am not getting it...
>>>
>>> In any case, independently of where is the burden, I want to veto any
>>> new integration that may make future builds non-reproducible. Otherwise
>>> this is a source of chaos and dead kittens.
>>>
>>> The problem is that actually if you have a git project with a stable and
>> dev branch you will need:
>> - One baseline on the stable branch with determined dependencies versions
>> - One baseline on the dev branch with less restricted versions (for
>> example 3.x.x)
>> Then, when you release a new stable version by merging the development
>> branch you need to edit the merge by hand to merge everything except the
>> baseline.
>> I think this is what Denis wants to avoid.
>>
> But in general, such merges need to be done only for major and minor
>> releases. Not for a patch because you can do a hotfix. You create a new
>> branch from master, then merge in in master and development when it's done.
>>
>
> I still do not understand... Also I do not understand your usage of the
> term "the merge". Can somebody give a **concrete** scenario?
>
> When you release a version, please do not move that version. You should
> then create new versions, with new numbers and new code. But never touch
> old versions with old numbers and old code. Like that I can download the
> same old code using the same old number to get the old version whenever I
> want :).
>
> You can try to do it with branches, tags, different repositories, or even
> with zipfiles in mails. I don't care. Just don't modify releases and I'm
> happy with it.
>

Not branches.  Git has is no real difference between branches and tags,
except that "by definition" branches move and tags don't.
cheers -ben

But can you explain me the scenario? I still don't understand it :( On Mon, Mar 5, 2018 at 4:51 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

Maybe we should just load Calypso dependencies explicitly. In any case they will be used by other projects. Like new Iceberg will use Commander. And ClassAnnotation will be in Kernel. So we will need to load them in another time than Calypso.

Now we can just put them directly into #loadCalypso method. (there are three dependencies) It will fix my problem and make Pharo build reproducible.

2018-03-05 16:38 GMT+01:00 Guillermo Polito <guillermopolito@gmail.com>:

...

But, "one single class" does not mean anything. Because it depends from which branch/commitish you are loading it from...

Can you explain better what is the problem because I am not getting it...

In any case, independently of where is the burden, I want to veto any new integration that may make future builds non-reproducible. Otherwise this is a source of chaos and dead kittens.

On Mon, Mar 5, 2018 at 4:17 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

2018-03-05 16:13 GMT+01:00 Cyril Ferlicot <cyril.ferlicot@gmail.com>:

...

On Mon, Mar 5, 2018 at 4:10 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

Hi Pablo.

Dev branch approach not really works. Because any merge into master will break master baseline. (notice that baseline is in same repo). And managing merges by hand all the time is not a solution.

Hi! If you don't want to manage the merges by hand you can maybe have two bsaelines? BaselineOfCalypso and BaselineOfCalypsoDev?

It should work. But is it right way that everybody should follow?

With configurations it was easy to do in single class.

...

-- Cyril Ferlicot https://ferlicot.fr

--

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>*

*Web:* *http://guillep.github.io* <http://guillep.github.io>

*Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

2018-03-05 16:17 GMT+01:00 Denis Kudriashov <dionisiydk@gmail.com>:

2018-03-05 16:13 GMT+01:00 Cyril Ferlicot <cyril.ferlicot@gmail.com>:

...

On Mon, Mar 5, 2018 at 4:10 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

Hi Pablo.

Dev branch approach not really works. Because any merge into master will break master baseline. (notice that baseline is in same repo). And managing merges by hand all the time is not a solution.

Hi! If you don't want to manage the merges by hand you can maybe have two bsaelines? BaselineOfCalypso and BaselineOfCalypsoDev?

At the end I found that this approach with two baselines is not really working. So I will adopt my projects to the following pattern: Each release will point to static version of dependencies. They all in master branch. Floating version like 0.5.x will be maintained in separate branch instead of floating tag. In this branch baseline will reference dependencies using similar "floating" branch in their repositories. So to get all minor fixes from all dependencies the code should be loaded from such floating branch (0.5.x). The main project can be locked to fix current version (only dependencies will be updated). And all released versions (0.5.3 tag) will be completely reproducible.

It should work. But is it right way that everybody should follow?

With configurations it was easy to do in single class.

...

-- Cyril Ferlicot https://ferlicot.fr

Denis Kudriashov <dionisiydk@gmail.com> wrote:

So to get all minor fixes from all dependencies the code should be loaded from such floating branch (0.5.x). The main project can be locked to fix current version (only dependencies will be updated). And all released versions (0.5.3 tag) will be completely reproducible.

So your goal for released versions is to be only loadable on a specific pharo build? Stephan

Denis Kudriashov <dionisiydk@gmail.com> wrote:

2018-03-17 13:15 GMT+01:00 Stephan Eggermont <stephan@stack.nl>:

...

Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

So to get all minor fixes from all dependencies the code should be loaded from such floating branch (0.5.x). The main project can be locked to fix current version (only dependencies will be updated). And all released versions (0.5.3 tag) will be completely reproducible.

So your goal for released versions is to be only loadable on a specific pharo build?

On any pharo build. I not understand what you mean

Your dependencies will need to be updated to run on newer pharo builds, won’t they? Stephan

Denis Kudriashov <dionisiydk@gmail.com> wrote:

No. Each build just loads specified version. I do not need to create new version for new build. But I think I still not understand you

That specified version will no longer work in a newer pharo build. Pharo changes, making it necessary for your dependencies to change. Will you make a new release each time one of your dependencies needs to change? Stephan

Denis Kudriashov <dionisiydk@gmail.com> wrote:

2018-03-17 17:01 GMT+01:00 Stephan Eggermont <stephan@stack.nl>:

...

Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

No. Each build just loads specified version. I do not need to create new version for new build. But I think I still not understand you

That specified version will no longer work in a newer pharo build. Pharo changes, making it necessary for your dependencies to change. Will you make a new release each time one of your dependencies needs to change?

I do not put dependencies of projects which are part of standard Pharo image. So Pharo updates do not affect my baselines

And your dependencies are not impacted by standard pharo changes? Stephan

Denis Kudriashov <dionisiydk@gmail.com> wrote:

2018-03-19 9:16 GMT+01:00 Stephan Eggermont <stephan@stack.nl>:

...

Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

2018-03-17 17:01 GMT+01:00 Stephan Eggermont <stephan@stack.nl>:

...

Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

No. Each build just loads specified version. I do not need to create

new

...

version for new build. But I think I still not understand you

That specified version will no longer work in a newer pharo build. Pharo changes, making it necessary for your dependencies to change. Will you make a new release each time one of your dependencies needs to change?

I do not put dependencies of projects which are part of standard Pharo image. So Pharo updates do not affect my baselines

And your dependencies are not impacted by standard pharo changes?

It can be impacted of course. But it is normal process for alpha Pharo 7. And Pharo 6 is fixed, so I do not care.

Then why bother releasing? Stephan

Also it is not really development versions. I follow semantics versioning. So I reference 0.3.x versions to get all minor fixes from dependencies. 2018-03-05 15:57 GMT+01:00 tesonep@gmail.com <tesonep@gmail.com>:

Yes, I understand for the developer version, you should have a developer branch and use that. The problem is for the Pharo image only.

On Mon, Mar 5, 2018 at 3:39 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

...

Hi,

it is an issue of Calypso dependencies. I discussed it with Denis before Calypso integration and he would like to change it but, for now, it would make the development for him much harder because he needs a way how to load the latest bleeding edge versions of the packages and no one was able to tell him the proper way how to support both scenarios (loading of a specific release and loading of the development versions) with baselines without needing of duplicate them.

Anyway, I agree that the current state must be fixed as soon as possible.

Cheers, -- Pavel

2018-03-05 15:30 GMT+01:00 tesonep@gmail.com <tesonep@gmail.com>:

...

Hello, i have seen in the latest version of Pharo baselines pointing to "floating" versions. A version that is not fixed. I want to know why this is like that?

To me this is basically wrong, because basically this goes against the idea of having repeatable builds. Also using semantic versions with tags in Git is wrong, as tags should be immutable (that is why you have to use --force to update them, in Guille's words "If you have to force it, it is not good").

Cheers,

-- Pablo Tesone. tesonep@gmail.com

-- Pablo Tesone. tesonep@gmail.com

On 03/05/2018 08:48 AM, Stephan Eggermont wrote:

tesonep@gmail.com <tesonep@gmail.com> wrote:

...

Hello, i have seen in the latest version of Pharo baselines pointing to "floating" versions. A version that is not fixed. I want to know why this is like that? Because that is the way it should be? It is hand-edited, so repeatable builds are not the goal, just working software. Repeatable builds come from recording what you loaded in what order. Repeatable builds with fixed dependency versions don't help you with working software, as they block your dependencies from getting patched. Never depend on hard-coded versions of anything you don't control yourself. See the 5D paper

Stephan Eggermont Stephan,

For this approach to work it is important that people not make a practice of committing api-breaking changes and by definition each new release of pharo breaks the fundamental api ... I've been able to survive in this universe with Metacello over the years, because I make a point of ensuring that the latest version of Metacello works on all supported platform versions and my observation has been that most Pharo projects ignore older versions of pharo for as long as they can get away with it and this becomes the source of so many issues I've seen ... With github and travis-ci it is practical to support projects on multiple platform versions ... if you want to ... this approach means that when you move to a new platform version you have to use packaging to isolate your version-specific changes and you probably have to refactor your implementation to be able to manage these version-specific changes ... only when you make api-breaking changes is it necessary to start new branches or use a series of tags ... Dale