I am quite sure that you already know all of this, but I just have urge to make some noice. :) In Java world Maven handles this situation using term snapshot dependency. Non-snapshot release can/should not contain any snapshot dependencies. Contents of snapshot version is expected to change and non-snapshot releases are supposed to remain static. To me it looks like that these stable & co symbolic versions are quite the same than Mavens snapshot versions. Thus I think it would make sense have a functionality in metacello to "release" a version which would use the exact versions instead of symbolic ones. At least there there should be validation agains these issues. Possibly it should lock the monticello packages used to load the configurations also. IMHO it breaks the whole if so called stable version of project A depends on some other project which uses some symbolic version of some third project. When this third project advances it will eventually break the project A and thus the B in between does not seem so stable anymore. 2011/3/23 Dale Henrichs <dhenrich@vmware.com>:

Miguel,

Two things:

 1) there are certain conditions where #stable is the correct version to use  2) I thought the point of running hudson-based tests is to identify these types      of problems as soon as they occur so that they can be fixed earlier.

With that said, I'd be inclined to create some scripts that recorded the "configuration signature" of an image, something like the following:

 - Nile 1.2 (GuillermoPolito.19)  - OCompletion 1.2.2 (GuillermePolito.39)  - Pharo 1.1.1 (LaurentLaffonte.148)  - ....

where the parenthesized info is the author and version of the mcz file. So if an error shows up out of the blue, the first thing would be to check for changes in the "configuration signature" and go from there ...

We're already doing this kind of thing for the MetacelloBrowser, so it would be very easy to whip up a script to build the signature ... I guess each image could keep/generate it's signature....

I think the "configuration signature" is very useful ... if the "configuration signature" doesn't change the exact same code should be loaded ...

Dale

On Mar 23, 2011, at 8:40 AM, Miguel Cobá wrote:

...

Maybe we should ask Dale when time permits to add functionality to Metacello so that can check all the dependencies for a given configuration and make sure (or raise some warning in the validation) that some dependency is refered by a symbolic non numeric version (like #stable, #latestVersion, #bleedingEdge) so that this validation can be run by hudson also and verify that all the configurations are frozen to a given number and no dependent on a symbolic version that can change anytime after release of Pharo when a package get a new (#stable for example) version.

Cheers

El mié, 23-03-2011 a las 13:26 +0100, Marcus Denker escribió:

...

On Mar 23, 2011, at 12:52 PM, Torsten Bergmann wrote:

...

Pharo 1.2 is red again!

DuplicatedVariableError: stylingActive is already defined in Workspace

https://pharo-ic.lille.inria.fr/hudson/view/Pharo/job/Pharo%201.2/195/

We really need to make sure that the config does not load code if we don't explicitly say so.

So what code was changed? Why was it loaded?

-- Marcus Denker  -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.

-- Miguel Cobá http://twitter.com/MiguelCobaMtz http://miguel.leugim.com.mx

-- Panu

2011/3/23 Dale Henrichs <dhenrich@vmware.com>:

Panu,

I agree almost completely most of the project dependencies in configuration _should_ use a literal version ... I still maintain that there are use cases where using symbolic version like #stable is correct.

Your example was good. However in the long run when those #stable dependencies develop, old versions that depend on them became broken because of changed API/protocol and not because of a introduced bug. But using spesific versions would not solve this problem either just introduces new ones. Using version ranges could help a little but one could not know the range before hand. Of course if suggestion of semver would be used one could guess that next major release will probable broke things.

I have proposed the "configuration signature" because I have noticed that some developers will change existing/released configurations (which could cause problems) and I think that Pharo-1.2 just might be using several versions with a #development blessing (where it is an accepted practice to change the specification for a version) so any change in version of a configuration mcz file is suspect ...

I hope when things mature people start to obey these "rules". This same problems exist with maven also but people maybe know the consequences and don't go against the odds. On mission critical cases one is supposed to store all the required dependencies to avoid third party to fiddle with builds. Which is the case with Metacello too.

The problem with locking down the versions of everything is very similar to the problem of static definitions of classes (in java) ... one change to a class definition and there is a cascade of changes that must be made to the rest of the system before you can get the whole thing to compile again.... There _is_ tension between the desire to be able to guarantee reproducable results and providing enough wiggle room to avoid putting development into version grid lock ...

I think these approaches should coexists. Some projects are very strict about repeatable builds. In some projects this will just create uneccessary overhead.

But I think the hudson server is actually doing it's job by loading the configurations that a developer would be loading and finding problems that a developer would otherwise find on his or her own ... the fact that the build failed is a good thing and fixing the problem whether it be in a literal version or a symbolic version is the important thing...

Yep. IMHO sensible intergration builds are easier to do with Metacello than Maven + some scm. -- Panu

Miguel, Well put ... I think that both options have their place during the whole development cycle ... there are points in time where it is appropriate to "checkpoint the configurations and packages" for reproducible results as well as to distribute the source so that a single server outage doesn't shut everything down ... In the interim it makes sense to use hudson to validate verify the ongoing development... Dale On Mar 23, 2011, at 2:46 PM, Miguel Cobá wrote:

I agree with the points you note. I think that are reasonable. And as you say, maybe the rigid scheme of specific versions is not the best solution.

There are then a couple options:

1. Accept that there is no control on the packages and versions used to build a Pharo release, and then fix the problems as soon as possible when they show up in the hudson reports. This acknowledges the fact that those packages are *important* for Pharo, so a level of care on them must be put for some time after each release of Pharo.

2. When releasing a Pharo version, use the script to copy the configuration and the packages loaded by them to a know blessed, read only repository on squeaksource.com. This allows to reliably repeat builds of any pharo core for auditing or traceability. Of course each user in its own image can use the configuration pointing to the upstream/original repositories, so that they can get the new developments. This only get us a repeteable (but not very useful and maybe not so necessary for other than auditing) build anytime in the future.

Of course this is not a problem of Metacello or any given packaging system. It is a matter of policies of the project so I think that for Pharo a policy can be stated and then applied. IMHO a mixture of the two options before is the best for Pharo. To copy the packages with Dale's script to a know location as part of the release process and to keep the builds green for a given number of release as they appear on the Hudson reports.

What do you think?

El mié, 23-03-2011 a las 13:05 -0700, Dale Henrichs escribió:

...

Panu,

I agree almost completely most of the project dependencies in configuration _should_ use a literal version ... I still maintain that there are use cases where using symbolic version like #stable is correct.

For example, I have a project called GemTools that depends upon OB being present ... I actually do not care which version of OB is loaded, since I am _not_ tightly coupled to OB ... I just happen to have several windows that are implemented as subclasses of OBBrowser - nothing really fancy, but I want GemTools to be usable in Squeak, Pharo1.0, Pharo1.1 and Pharo1.2, etc., etc.

If I am required to specify a specific version of OB to use, then I must become an expert about which versions of OB to use on which platform of Pharo or Squeak and if a new version of OB is released I have to update _my_ configuration to reflect the newly released version ... oh and I have to do this whenever a new ob is released for any one of the (5 or 6) different platforms that OB runs on ...

If I specify that I want the #stable version of OB, I don't have to worry any more and I don't have to keep track of the ins and outs of OB ...

If GemTools _is_ referencing a #stable version and the hudson server is loading GemTools and the tests break because the #stable version of OB has changed and introduced a bug, I say ... Hurray for hudson, since a bug has been found with the #stable version of OB .

At least the poor developer who loads the #stable version OB won't be the first (or only) person to find the bug...

I have proposed the "configuration signature" because I have noticed that some developers will change existing/released configurations (which could cause problems) and I think that Pharo-1.2 just might be using several versions with a #development blessing (where it is an accepted practice to change the specification for a version) so any change in version of a configuration mcz file is suspect ...

I _have_ been toying with the idea that a Metacello version could include the signature of the mcz file, so that one could specify something like the following:

spec project: 'OmniBrowser' with: '1.2.3:dkh.345'.

or

spec project: 'OmniBrowser' with: #'stable:dkh.345'

If this is done, then the specification is completely locked down ... of course the ':' notion would be optional ...

The problem with locking down the versions of everything is very similar to the problem of static definitions of classes (in java) ... one change to a class definition and there is a cascade of changes that must be made to the rest of the system before you can get the whole thing to compile again....

There _is_ tension between the desire to be able to guarantee reproducable results and providing enough wiggle room to avoid putting development into version grid lock ...

With all of that said, there is nothing to prevent a validator rule from being written that checks (recursively) for the use of symbolic versions.

But I think the hudson server is actually doing it's job by loading the configurations that a developer would be loading and finding problems that a developer would otherwise find on his or her own ... the fact that the build failed is a good thing and fixing the problem whether it be in a literal version or a symbolic version is the important thing...

This _is_ an interesting discussion and I am sure that more can be said (I know that _I_ have more to say:) and I hope that it continues...my hope is that we find a balance between rigidity and flexibility ... too rigid and progress becomes tedious (too many things have to change) too flexible and everything spins out of control ...

Having lots of good tests and running them regularly is one of the things that allows for more flexibility and keeps things from spinning out of control, so if you are running lots of tests on a complex system that is changing regularly ... expect the tests to fail and make sure that we have the tools that help us identify and resolve the problems as they appear...

Dale

On Mar 23, 2011, at 12:23 PM, Panu Suominen wrote:

...

I am quite sure that you already know all of this, but I just have urge to make some noice. :)

In Java world Maven handles this situation using term snapshot dependency. Non-snapshot release can/should not contain any snapshot dependencies. Contents of snapshot version is expected to change and non-snapshot releases are supposed to remain static. To me it looks like that these stable & co symbolic versions are quite the same than Mavens snapshot versions.

Thus I think it would make sense have a functionality in metacello to "release" a version which would use the exact versions instead of symbolic ones. At least there there should be validation agains these issues. Possibly it should lock the monticello packages used to load the configurations also. IMHO it breaks the whole if so called stable version of project A depends on some other project which uses some symbolic version of some third project. When this third project advances it will eventually break the project A and thus the B in between does not seem so stable anymore.

2011/3/23 Dale Henrichs <dhenrich@vmware.com>:

...

Miguel,

Two things:

1) there are certain conditions where #stable is the correct version to use 2) I thought the point of running hudson-based tests is to identify these types of problems as soon as they occur so that they can be fixed earlier.

With that said, I'd be inclined to create some scripts that recorded the "configuration signature" of an image, something like the following:

- Nile 1.2 (GuillermoPolito.19) - OCompletion 1.2.2 (GuillermePolito.39) - Pharo 1.1.1 (LaurentLaffonte.148) - ....

where the parenthesized info is the author and version of the mcz file. So if an error shows up out of the blue, the first thing would be to check for changes in the "configuration signature" and go from there ...

We're already doing this kind of thing for the MetacelloBrowser, so it would be very easy to whip up a script to build the signature ... I guess each image could keep/generate it's signature....

I think the "configuration signature" is very useful ... if the "configuration signature" doesn't change the exact same code should be loaded ...

Dale

On Mar 23, 2011, at 8:40 AM, Miguel Cobá wrote:

...

Maybe we should ask Dale when time permits to add functionality to Metacello so that can check all the dependencies for a given configuration and make sure (or raise some warning in the validation) that some dependency is refered by a symbolic non numeric version (like #stable, #latestVersion, #bleedingEdge) so that this validation can be run by hudson also and verify that all the configurations are frozen to a given number and no dependent on a symbolic version that can change anytime after release of Pharo when a package get a new (#stable for example) version.

Cheers

El mié, 23-03-2011 a las 13:26 +0100, Marcus Denker escribió:

...

On Mar 23, 2011, at 12:52 PM, Torsten Bergmann wrote:

...

Pharo 1.2 is red again!

DuplicatedVariableError: stylingActive is already defined in Workspace

https://pharo-ic.lille.inria.fr/hudson/view/Pharo/job/Pharo%201.2/195/

We really need to make sure that the config does not load code if we don't explicitly say so.

So what code was changed? Why was it loaded?

-- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.

-- Miguel Cobá http://twitter.com/MiguelCobaMtz http://miguel.leugim.com.mx

-- Panu

-- Miguel Cobá http://twitter.com/MiguelCobaMtz http://miguel.leugim.com.mx

Stef, Yep ... all the pieces are there ... it's just a matter of putting them together in the right order..this is what we do for GemStone GLASS builds... Dale On Mar 24, 2011, at 1:21 AM, Stéphane Ducasse wrote:

henrichs

do you know if you have now the pieces in place to be able to freeze a configuration and all its recursive sets in a dedicated repository?

Stef

Sounds great. So, what needs to be done then? I would be interested to use this in the context of Moose as well. Cheers, Doru On 24 Mar 2011, at 21:36, Stéphane Ducasse wrote:

this is a good news

Stef

On Mar 24, 2011, at 4:59 PM, Dale Henrichs wrote:

...

Stef,

Yep ... all the pieces are there ... it's just a matter of putting them together in the right order..this is what we do for GemStone GLASS builds...

Dale

On Mar 24, 2011, at 1:21 AM, Stéphane Ducasse wrote:

...

henrichs

do you know if you have now the pieces in place to be able to freeze a configuration and all its recursive sets in a dedicated repository?

Stef

-- www.tudorgirba.com "Every successful trip needs a suitable vehicle."

Am 23.03.2011 um 22:46 schrieb Miguel Cobá:

I agree with the points you note. I think that are reasonable. And as you say, maybe the rigid scheme of specific versions is not the best solution.

There are then a couple options:

1. Accept that there is no control on the packages and versions used to build a Pharo release, and then fix the problems as soon as possible when they show up in the hudson reports. This acknowledges the fact that those packages are *important* for Pharo, so a level of care on them must be put for some time after each release of Pharo.

2. When releasing a Pharo version, use the script to copy the configuration and the packages loaded by them to a know blessed, read only repository on squeaksource.com. This allows to reliably repeat builds of any pharo core for auditing or traceability. Of course each user in its own image can use the configuration pointing to the upstream/original repositories, so that they can get the new developments. This only get us a repeteable (but not very useful and maybe not so necessary for other than auditing) build anytime in the future.

Of course this is not a problem of Metacello or any given packaging system. It is a matter of policies of the project so I think that for Pharo a policy can be stated and then applied. IMHO a mixture of the two options before is the best for Pharo. To copy the packages with Dale's script to a know location as part of the release process and to keep the builds green for a given number of release as they appear on the Hudson reports.

What do you think?

Very well put. To me a release is just (as Igor said) any succesful build on hudson. It is just a freeze in development time on which you can base documentation, books like pharo by example etc. A repeatable release build enables you to produce the smallest possible delta if you need to make a minor/patch release. Norbert

El mié, 23-03-2011 a las 13:05 -0700, Dale Henrichs escribió:

...

Panu,

I agree almost completely most of the project dependencies in configuration _should_ use a literal version ... I still maintain that there are use cases where using symbolic version like #stable is correct.

For example, I have a project called GemTools that depends upon OB being present ... I actually do not care which version of OB is loaded, since I am _not_ tightly coupled to OB ... I just happen to have several windows that are implemented as subclasses of OBBrowser - nothing really fancy, but I want GemTools to be usable in Squeak, Pharo1.0, Pharo1.1 and Pharo1.2, etc., etc.

If I am required to specify a specific version of OB to use, then I must become an expert about which versions of OB to use on which platform of Pharo or Squeak and if a new version of OB is released I have to update _my_ configuration to reflect the newly released version ... oh and I have to do this whenever a new ob is released for any one of the (5 or 6) different platforms that OB runs on ...

If I specify that I want the #stable version of OB, I don't have to worry any more and I don't have to keep track of the ins and outs of OB ...

If GemTools _is_ referencing a #stable version and the hudson server is loading GemTools and the tests break because the #stable version of OB has changed and introduced a bug, I say ... Hurray for hudson, since a bug has been found with the #stable version of OB .

At least the poor developer who loads the #stable version OB won't be the first (or only) person to find the bug...

I have proposed the "configuration signature" because I have noticed that some developers will change existing/released configurations (which could cause problems) and I think that Pharo-1.2 just might be using several versions with a #development blessing (where it is an accepted practice to change the specification for a version) so any change in version of a configuration mcz file is suspect ...

I _have_ been toying with the idea that a Metacello version could include the signature of the mcz file, so that one could specify something like the following:

spec project: 'OmniBrowser' with: '1.2.3:dkh.345'.

or

spec project: 'OmniBrowser' with: #'stable:dkh.345'

If this is done, then the specification is completely locked down ... of course the ':' notion would be optional ...

The problem with locking down the versions of everything is very similar to the problem of static definitions of classes (in java) ... one change to a class definition and there is a cascade of changes that must be made to the rest of the system before you can get the whole thing to compile again....

There _is_ tension between the desire to be able to guarantee reproducable results and providing enough wiggle room to avoid putting development into version grid lock ...

With all of that said, there is nothing to prevent a validator rule from being written that checks (recursively) for the use of symbolic versions.

But I think the hudson server is actually doing it's job by loading the configurations that a developer would be loading and finding problems that a developer would otherwise find on his or her own ... the fact that the build failed is a good thing and fixing the problem whether it be in a literal version or a symbolic version is the important thing...

This _is_ an interesting discussion and I am sure that more can be said (I know that _I_ have more to say:) and I hope that it continues...my hope is that we find a balance between rigidity and flexibility ... too rigid and progress becomes tedious (too many things have to change) too flexible and everything spins out of control ...

Having lots of good tests and running them regularly is one of the things that allows for more flexibility and keeps things from spinning out of control, so if you are running lots of tests on a complex system that is changing regularly ... expect the tests to fail and make sure that we have the tools that help us identify and resolve the problems as they appear...

Dale

On Mar 23, 2011, at 12:23 PM, Panu Suominen wrote:

...

I am quite sure that you already know all of this, but I just have urge to make some noice. :)

In Java world Maven handles this situation using term snapshot dependency. Non-snapshot release can/should not contain any snapshot dependencies. Contents of snapshot version is expected to change and non-snapshot releases are supposed to remain static. To me it looks like that these stable & co symbolic versions are quite the same than Mavens snapshot versions.

Thus I think it would make sense have a functionality in metacello to "release" a version which would use the exact versions instead of symbolic ones. At least there there should be validation agains these issues. Possibly it should lock the monticello packages used to load the configurations also. IMHO it breaks the whole if so called stable version of project A depends on some other project which uses some symbolic version of some third project. When this third project advances it will eventually break the project A and thus the B in between does not seem so stable anymore.

2011/3/23 Dale Henrichs <dhenrich@vmware.com>:

...

Miguel,

Two things:

1) there are certain conditions where #stable is the correct version to use 2) I thought the point of running hudson-based tests is to identify these types of problems as soon as they occur so that they can be fixed earlier.

With that said, I'd be inclined to create some scripts that recorded the "configuration signature" of an image, something like the following:

- Nile 1.2 (GuillermoPolito.19) - OCompletion 1.2.2 (GuillermePolito.39) - Pharo 1.1.1 (LaurentLaffonte.148) - ....

where the parenthesized info is the author and version of the mcz file. So if an error shows up out of the blue, the first thing would be to check for changes in the "configuration signature" and go from there ...

We're already doing this kind of thing for the MetacelloBrowser, so it would be very easy to whip up a script to build the signature ... I guess each image could keep/generate it's signature....

I think the "configuration signature" is very useful ... if the "configuration signature" doesn't change the exact same code should be loaded ...

Dale

On Mar 23, 2011, at 8:40 AM, Miguel Cobá wrote:

...

Maybe we should ask Dale when time permits to add functionality to Metacello so that can check all the dependencies for a given configuration and make sure (or raise some warning in the validation) that some dependency is refered by a symbolic non numeric version (like #stable, #latestVersion, #bleedingEdge) so that this validation can be run by hudson also and verify that all the configurations are frozen to a given number and no dependent on a symbolic version that can change anytime after release of Pharo when a package get a new (#stable for example) version.

Cheers

El mié, 23-03-2011 a las 13:26 +0100, Marcus Denker escribió:

...

On Mar 23, 2011, at 12:52 PM, Torsten Bergmann wrote:

...

Pharo 1.2 is red again!

DuplicatedVariableError: stylingActive is already defined in Workspace

https://pharo-ic.lille.inria.fr/hudson/view/Pharo/job/Pharo%201.2/195/

We really need to make sure that the config does not load code if we don't explicitly say so.

So what code was changed? Why was it loaded?

-- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.

-- Miguel Cobá http://twitter.com/MiguelCobaMtz http://miguel.leugim.com.mx

-- Panu

-- Miguel Cobá http://twitter.com/MiguelCobaMtz http://miguel.leugim.com.mx