Hi guys How can I get Magritte seaside (and a working version not one breaking on XML I do not what)? I loaded the published stable version of Magritte into Pharo and there is no seaside version. I checked the configuration and it is so complex.... I have the impression that we need to distinguish Magritte from Magritte seaside. So I will do a separate configuration of magritte seaside. Stef
On Oct 6, 2015, at 11:11 AM, stepharo <stepharo@free.fr> wrote:
Hi guys
How can I get Magritte seaside (and a working version not one breaking on XML I do not what)? I loaded the published stable version of Magritte into Pharo and there is no seaside version. I checked the configuration and it is so complexâ¦.
I think you have to load the âSeaside" group of ConfigurationOfMagritte3, but I am not sure.. A month ago I played with Magritte and Seaside and I basically loaded a bunch of configurations until it worked. As a new user I loaded âMagritteMagicâ without knowing better. This one is broken if you load it on Pharo 4. Later I found out that MagritteMagic was just for demonstration purposes. So maybe it should get an âexampleâ or âdemoâ tag in the Project Catalog.
On 06-10-15 11:11, stepharo wrote:
I will do a separate configuration of magritte seaside.
You need to load the correct groups. Separate configurations are a bad practice and should be eliminated, not encouraged. The configuration has a lot of essential complexity, as many different combinations of components are needed in different situations. We need tools that recognize that essential complexity and support it, not tools that try to hide it and then have no practical usability. Stephan
Le 6/10/15 11:40, Stephan Eggermont a écrit :
On 06-10-15 11:11, stepharo wrote:
I will do a separate configuration of magritte seaside.
You need to load the correct groups. Separate configurations are a bad practice and should be eliminated, not encouraged.
I totally disagree. And groups are not supported by the configurationBrowser
The configuration has a lot of essential complexity, as many different combinations of components are needed in different situations. We need tools that recognize that essential complexity and support it, not tools that try to hide it and then have no practical usability.
Well I disagree. And all the effort of christophe for the new dependencies declarations at the package level just goes in that direction. I will do a MagritteSeaside Configuration that I will maintain and I will also do a MagritteTutorial one.
Stephan
The many configurations of GT/Glamour have not exactly convinced me. For a configuration that works when just loading #'stable' from the configuration browser, I'd suggest trying with the configuration of BootstrapMagritte. That only runs into the #subStrings: vs #substrings:. Stephan
On 10/6/15 5:10 AM, stepharo wrote:
Le 6/10/15 11:40, Stephan Eggermont a écrit :
On 06-10-15 11:11, stepharo wrote:
I will do a separate configuration of magritte seaside.
You need to load the correct groups. Separate configurations are a bad practice and should be eliminated, not encouraged.
I totally disagree. And groups are not supported by the configurationBrowser
The configuration has a lot of essential complexity, as many different combinations of components are needed in different situations. We need tools that recognize that essential complexity and support it, not tools that try to hide it and then have no practical usability.
Well I disagree. And all the effort of christophe for the new dependencies declarations at the package level just goes in that direction. I will do a MagritteSeaside Configuration that I will maintain and I will also do a MagritteTutorial one.
Stef, I hope you and Christophe are considering cross platform considerations when implementing "new dependencies declarations at the package level" ... MonticelloConfigurations had "package level dependencies" and the implementation was extremely rigid to the point where it wasn't possible to substitute an alternate (platform-speicific) package in the package graph without creating new package versions of all packages in the graph ... Of course I am not aware of the approach that Christophe, but be aware that "there be dragons there":) Dale
Stef,
I hope you and Christophe are considering cross platform considerations when implementing "new dependencies declarations at the package level" ... MonticelloConfigurations had "package level dependencies" and the implementation was extremely rigid to the point where it wasn't possible to substitute an alternate (platform-speicific) package in the package graph without creating new package versions of all packages in the graph ... Of course I am not aware of the approach that Christophe, but be aware that "there be dragons there":)
Hi dale. I will let christophe answers because I do not know the details. I know that he is working hard on it. Ideally I would like to have a ConfigurationOf for each package because it would reduce the bloat of the actual configurationOf and also be closer to the package. But if I say that people will start shouting that I'm mad :) (I'm not fool and I'm right :). so this is why we are doing it inside the package but we will see. First the core bootstrap based on dependencies ;) Stef
On 10/6/15 11:43 AM, stepharo wrote:
Stef,
I hope you and Christophe are considering cross platform considerations when implementing "new dependencies declarations at the package level" ... MonticelloConfigurations had "package level dependencies" and the implementation was extremely rigid to the point where it wasn't possible to substitute an alternate (platform-speicific) package in the package graph without creating new package versions of all packages in the graph ... Of course I am not aware of the approach that Christophe, but be aware that "there be dragons there":)
Hi dale.
I will let christophe answers because I do not know the details. I know that he is working hard on it. Ideally I would like to have a ConfigurationOf for each package because it would reduce the bloat of the actual configurationOf and also be closer to the package. But if I say that people will start shouting that I'm mad :) (I'm not fool and I'm right :). so this is why we are doing it inside the package but we will see. First the core bootstrap based on dependencies ;)
Stef
Stef, Basing things on a "configuration" external to the package structure itself will avoid the MetacelloConfiguration limitations. From your description, it sounds like Christophe might be "deconstructing" the project configuration in some way --- I would be interested in more details ... Thanks, Dale
Hi Dale, Le 6 oct. 2015 à 20:23, Dale Henrichs a écrit :
On 10/6/15 5:10 AM, stepharo wrote:
Le 6/10/15 11:40, Stephan Eggermont a écrit :
On 06-10-15 11:11, stepharo wrote:
I will do a separate configuration of magritte seaside.
You need to load the correct groups. Separate configurations are a bad practice and should be eliminated, not encouraged.
I totally disagree. And groups are not supported by the configurationBrowser
The configuration has a lot of essential complexity, as many different combinations of components are needed in different situations. We need tools that recognize that essential complexity and support it, not tools that try to hide it and then have no practical usability.
Well I disagree. And all the effort of christophe for the new dependencies declarations at the package level just goes in that direction. I will do a MagritteSeaside Configuration that I will maintain and I will also do a MagritteTutorial one.
Stef,
I hope you and Christophe are considering cross platform considerations when implementing "new dependencies declarations at the package level" ... MonticelloConfigurations had "package level dependencies" and the implementation was extremely rigid to the point where it wasn't possible to substitute an alternate (platform-speicific) package in the package graph without creating new package versions of all packages in the graph ... Of course I am not aware of the approach that Christophe, but be aware that "there be dragons there":)
Thanks for the warning. cross-platform packages are integrated into this new approach by allowing to express dependencies to virtual packages (idea from debian OS: http://www.debian.org/doc/debian-policy/ch-binary.html#s-virtual_pkg). It allows low-coupling and is very flexible. You will find more information in the paper published at IWST'14: https://hal.inria.fr/hal-01086083/document (by the way, you should already have it, I sent it to you last year but you were very busy at this time ;) ) To summarize in a few words, we want dependencies expressed at the package level (and we want more meta-data on packages), we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). We also want to detect dependencies automatically as much as possible to avoid missing or not-used dependencies (dependency analyser tool). All together, it will allow to define architectural rules and layers in Pharo. I now experimenting all that with the Pharo bootstrap. If you have any question or remark, do not hesitate to ask or to give feedback. Christophe.
On 07/10/15 08:47, Christophe Demarey wrote:
If you have any question or remark, do not hesitate to ask or to give feedback.
How do you express the migration of functionality from one package to another? How does that affect users of those packages? How do you intend to deal with variants (as in the 5D model)? Stephan
Hi Stephan, Le 7 oct. 2015 à 10:22, Stephan Eggermont a écrit :
On 07/10/15 08:47, Christophe Demarey wrote:
If you have any question or remark, do not hesitate to ask or to give feedback.
How do you express the migration of functionality from one package to another? How does that affect users of those packages?
I don't get what you mean. Could you describe a use case?
How do you intend to deal with variants (as in the 5D model)
You can group commonalities in package assemblies. For specifities, you could use virtual packages to decrease the number of assemblies to write. In a common assembly, you could describe a dependency to a virtual package. Then you will have concrete packages providing this virtual package, each of them having its own dependencies and working on a specific platform. Christophe
Christophe, I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, but I am curious how you think "package dependencies" will play with git-based projects? In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together. In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together .... I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand.
Dale On 10/6/15 11:47 PM, Christophe Demarey wrote:
Hi Dale,
Le 6 oct. 2015 à 20:23, Dale Henrichs a écrit :
On 10/6/15 5:10 AM, stepharo wrote:
Le 6/10/15 11:40, Stephan Eggermont a écrit :
On 06-10-15 11:11, stepharo wrote:
I will do a separate configuration of magritte seaside. You need to load the correct groups. Separate configurations are a bad practice and should be eliminated, not encouraged. I totally disagree. And groups are not supported by the configurationBrowser The configuration has a lot of essential complexity, as many different combinations of components are needed in different situations. We need tools that recognize that essential complexity and support it, not tools that try to hide it and then have no practical usability. Well I disagree. And all the effort of christophe for the new dependencies declarations at the package level just goes in that direction. I will do a MagritteSeaside Configuration that I will maintain and I will also do a MagritteTutorial one.
Stef,
I hope you and Christophe are considering cross platform considerations when implementing "new dependencies declarations at the package level" ... MonticelloConfigurations had "package level dependencies" and the implementation was extremely rigid to the point where it wasn't possible to substitute an alternate (platform-speicific) package in the package graph without creating new package versions of all packages in the graph ... Of course I am not aware of the approach that Christophe, but be aware that "there be dragons there":)
Thanks for the warning. cross-platform packages are integrated into this new approach by allowing to express dependencies to virtual packages (idea from debian OS: http://www.debian.org/doc/debian-policy/ch-binary.html#s-virtual_pkg). It allows low-coupling and is very flexible. You will find more information in the paper published at IWST'14: https://hal.inria.fr/hal-01086083/document (by the way, you should already have it, I sent it to you last year but you were very busy at this time ;) )
To summarize in a few words, we want dependencies expressed at the package level (and we want more meta-data on packages), we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). We also want to detect dependencies automatically as much as possible to avoid missing or not-used dependencies (dependency analyser tool). All together, it will allow to define architectural rules and layers in Pharo. I now experimenting all that with the Pharo bootstrap.
If you have any question or remark, do not hesitate to ask or to give feedback.
Christophe.
Le 11/10/15 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, you should read it. :) We wrote it for that and it is not boring nor long.
but I am curious how you think "package dependencies" will play with git-based projects?
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together.
I think that this is only true for packages committed within the same repo. Now between porjects published in different repo you have to express them. I do not think that we all want to publish in the same repo and clone out everything.
In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand.
We want to be able to have a package market place where tools can grab dependencies without to load code and can compute the set of packages that should be loaded. When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution. Stef
Le 6 oct. 2015 à 20:23, Dale Henrichs a écrit :
On 10/6/15 5:10 AM, stepharo wrote:
Le 6/10/15 11:40, Stephan Eggermont a écrit :
On 06-10-15 11:11, stepharo wrote:
I will do a separate configuration of magritte seaside. You need to load the correct groups. Separate configurations are a bad practice and should be eliminated, not encouraged. I totally disagree. And groups are not supported by the configurationBrowser The configuration has a lot of essential complexity, as many different combinations of components are needed in different situations. We need tools that recognize that essential complexity and support it, not tools that try to hide it and then have no practical usability. Well I disagree. And all the effort of christophe for the new dependencies declarations at the package level just goes in that direction. I will do a MagritteSeaside Configuration that I will maintain and I will also do a MagritteTutorial one.
Stef,
I hope you and Christophe are considering cross platform considerations when implementing "new dependencies declarations at the package level" ... MonticelloConfigurations had "package level dependencies" and the implementation was extremely rigid to the point where it wasn't possible to substitute an alternate (platform-speicific) package in the package graph without creating new package versions of all packages in the graph ... Of course I am not aware of the approach that Christophe, but be aware that "there be dragons there":)
Thanks for the warning. cross-platform packages are integrated into this new approach by allowing to express dependencies to virtual packages (idea from debian OS: http://www.debian.org/doc/debian-policy/ch-binary.html#s-virtual_pkg). It allows low-coupling and is very flexible. You will find more information in the paper published at IWST'14: https://hal.inria.fr/hal-01086083/document (by the way, you should already have it, I sent it to you last year but you were very busy at this time ;) )
To summarize in a few words, we want dependencies expressed at the package level (and we want more meta-data on packages), we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). We also want to detect dependencies automatically as much as possible to avoid missing or not-used dependencies (dependency analyser tool). All together, it will allow to define architectural rules and layers in Pharo. I now experimenting all that with the Pharo bootstrap.
If you have any question or remark, do not hesitate to ask or to give feedback.
Christophe.
On 10/11/15 12:19 AM, stepharo wrote:
Le 11/10/15 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, you should read it. :) We wrote it for that and it is not boring nor long.
I scanned through it at the time and as I recall, I thought that the functionality described was already covered by git and BaselineOf ... but I did not read it in great detail and I did not (and still don't) have the time to compose a long-winded response:)
but I am curious how you think "package dependencies" will play with git-based projects?
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together.
I think that this is only true for packages committed within the same repo. Now between porjects published in different repo you have to express them. I do not think that we all want to publish in the same repo and clone out everything.
... and inter-project dependencies is what a BaselineOf does .... which brings me back to the conclusion that I reached when I scanned the paper:)
In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand.
We want to be able to have a package market place where tools can grab dependencies without to load code and can compute the set of packages that should be loaded.
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution. Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand.
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ... Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) Dale
I think that this is only true for packages committed within the same repo. Now between porjects published in different repo you have to express them. I do not think that we all want to publish in the same repo and clone out everything. ... and inter-project dependencies is what a BaselineOf does .... which brings me back to the conclusion that I reached when I scanned the paper:)
Yes probably but you have to have a BaselineOf for every package. Because we want to have the dependency expressed in every package and not in a gigantic "project" configuration of.
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution.
Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand. But you have to load the configuration and somehow execute it so that it computes the dependencies and we do not want.
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ... Yes we do. First we wanted to make it work. Then we should have kind of virtual packages that act as platforms or project level
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) I do not know where christophe save his code but it is be public.
Dale
On 10/11/15 11:40 AM, stepharo wrote:
I think that this is only true for packages committed within the same repo. Now between porjects published in different repo you have to express them. I do not think that we all want to publish in the same repo and clone out everything. ... and inter-project dependencies is what a BaselineOf does .... which brings me back to the conclusion that I reached when I scanned the paper:)
Yes probably but you have to have a BaselineOf for every package. Because we want to have the dependency expressed in every package and not in a gigantic "project" configuration of.
If you are talking about ConfigurationOf, then I agree that they can be gigantic, but 99% of the junk in a configuration is completely eliminated when you use a BaselineOf ... 99% of the junk in a configuration is dedicated to plugging the gap between monticello as an SCM and built-in capabilities for git ... A BaselineOf has a single baseline method and only the "pure" dependencies amongst the packages and projects needs to be expressed ... and these dependency specs are about "as small" as you can get ...
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution.
Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand. But you have to load the configuration and somehow execute it so that it computes the dependencies and we do not want.
Do you recall that I have been reminding folks that the specifications for ConfigurationOf and BaselineOf will someday become an XML file? ... I have taken great pains over the years to preserve this "expectation" ... Remember the choice to use classes was made as a bootstrap convenience to avoid having to create tools ... 6 years later and there is still a dearth of tools:) When you "execute" the code in a ConfigurationOf or BaselineOf, a completely separate data structure is constructed using a well-defined API and it is this "data structure" that does the real work ... it would be very easy to convert a BaselineOf into an XML/JSON/STON file without losing any information, unless you've have disregarding my warnings:) There is nothing in Metacello "project specifications" that _depends_ upon executing code ...
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ... Yes we do. First we wanted to make it work. Then we should have kind of virtual packages that act as platforms or project level
When you talk about "virtual packages", I would say that a BaselineOf is pretty much a "virtual package" already. When it comes to cross-platform support there are several factors that were built into Metacello from the very beginning: - it should be possible to USE a package on a different platform than it was originally written for without modifying the package itself - it should be possible to SUBSTITUTE a platform-specific package with another platform-specific package without modifying either of the packages - it should be possible to CHANGE the package dependencies based on platform-dependent requirements without modifying any of the packages The key here is that if one needs to change a package to use it on a different platform then you lose the ability to share source code. The BaselineOf satisfies these requirements, because the "project meta data" is separate from the package/source code and the BaselineOf itself is cross-platform. If you haven't thought about these issues from the very beginning, then it may not be easy to shoe-horn support in after the fact ... With all of that said, I really do love the idea of not having to support Metacello anymore:) So I would like to see you succeed with your effort!
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) I do not know where christophe save his code but it is be public.
I may not have time to read a paper ... but I would have time to read code:) Dale
A BaselineOf has a single baseline method and only the "pure" dependencies amongst the packages and projects needs to be expressed ... and these dependency specs are about "as small" as you can get ...
Yes but we want them per package. We do not work on that since more than one year by accident.
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution.
Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand. But you have to load the configuration and somehow execute it so that it computes the dependencies and we do not want. Do you recall that I have been reminding folks that the specifications for ConfigurationOf and BaselineOf will someday become an XML file? ... I have taken great pains over the years to preserve this "expectation" ... Remember the choice to use classes was made as a bootstrap convenience to avoid having to create tools ... 6 years later and there is still a dearth of tools:)
When you "execute" the code in a ConfigurationOf or BaselineOf, a completely separate data structure is constructed using a well-defined API and it is this "data structure" that does the real work ... it would be very easy to convert a BaselineOf into an XML/JSON/STON file without losing any information, unless you've have disregarding my warnings:)
There is nothing in Metacello "project specifications" that _depends_ upon executing code ... I know what I meant was use of pragmas, chaining pragma, loading all the configuration information (different baselines....).
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ... Yes we do. First we wanted to make it work. Then we should have kind of virtual packages that act as platforms or project level When you talk about "virtual packages", I would say that a BaselineOf is pretty much a "virtual package" already.
When it comes to cross-platform support there are several factors that were built into Metacello from the very beginning:
- it should be possible to USE a package on a different platform than it was originally written for without modifying the package itself - it should be possible to SUBSTITUTE a platform-specific package with another platform-specific package without modifying either of the packages - it should be possible to CHANGE the package dependencies based on platform-dependent requirements without modifying any of the packages
The key here is that if one needs to change a package to use it on a different platform then you lose the ability to share source code.
I see well the reason. :) The last point is the most tricky one when we have per package dependencies. But I will let christophe answers.
The BaselineOf satisfies these requirements, because the "project meta data" is separate from the package/source code and the BaselineOf itself is cross-platform.
If you haven't thought about these issues from the very beginning, then it may not be easy to shoe-horn support in after the fact ...
I will let christophe answer.
With all of that said, I really do love the idea of not having to support Metacello anymore:) So I would like to see you succeed with your effort!
Thanks. We will need help definitively.
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) I do not know where christophe save his code but it is be public. I may not have time to read a paper ... but I would have time to read code:)
:)
Dale
On 10/11/2015 11:10 PM, stepharo wrote:
A BaselineOf has a single baseline method and only the "pure" dependencies amongst the packages and projects needs to be expressed ... and these dependency specs are about "as small" as you can get ...
Yes but we want them per package. We do not work on that since more than one year by accident.
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution.
Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand. But you have to load the configuration and somehow execute it so that it computes the dependencies and we do not want. Do you recall that I have been reminding folks that the specifications for ConfigurationOf and BaselineOf will someday become an XML file? ... I have taken great pains over the years to preserve this "expectation" ... Remember the choice to use classes was made as a bootstrap convenience to avoid having to create tools ... 6 years later and there is still a dearth of tools:)
When you "execute" the code in a ConfigurationOf or BaselineOf, a completely separate data structure is constructed using a well-defined API and it is this "data structure" that does the real work ... it would be very easy to convert a BaselineOf into an XML/JSON/STON file without losing any information, unless you've have disregarding my warnings:)
There is nothing in Metacello "project specifications" that _depends_ upon executing code ... I know what I meant was use of pragmas, chaining pragma, loading all the configuration information (different baselines....).
Yeah, I meant that too ... I believe that the entire Metacello specification (ConfigurationOf and BaselineOf) can be represented in an XML/JSON/STON file modulo all of the extra methods that folks have tacked onto the ConfigurationOf for conevenience... but I don't consider that extra stuff part of Metacello ... while chained pragmas, etc. can be represented in XML/JSON/STON files, the primary reason that they are present is to support the ConfigurationOf ... a BaselineOf is typically a single method and doesn't use all of that extra stuff ...
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ... Yes we do. First we wanted to make it work. Then we should have kind of virtual packages that act as platforms or project level When you talk about "virtual packages", I would say that a BaselineOf is pretty much a "virtual package" already.
When it comes to cross-platform support there are several factors that were built into Metacello from the very beginning:
- it should be possible to USE a package on a different platform than it was originally written for without modifying the package itself - it should be possible to SUBSTITUTE a platform-specific package with another platform-specific package without modifying either of the packages - it should be possible to CHANGE the package dependencies based on platform-dependent requirements without modifying any of the packages
The key here is that if one needs to change a package to use it on a different platform then you lose the ability to share source code.
I see well the reason. :) The last point is the most tricky one when we have per package dependencies. But I will let christophe answers. It's the reason that I think per package dependencies (at least embedded in the package structure itself) is a bad idea ... it not only gets in the way of cross platform usage, but for the more complicated projects you end up having to touch a bunch of different files to make a "simple dependency change" not to mention having to edit different versions of the same package that may participate in different/conflicting dependency chains ...
The BaselineOf satisfies these requirements, because the "project meta data" is separate from the package/source code and the BaselineOf itself is cross-platform.
If you haven't thought about these issues from the very beginning, then it may not be easy to shoe-horn support in after the fact ...
I will let christophe answer.
With all of that said, I really do love the idea of not having to support Metacello anymore:) So I would like to see you succeed with your effort!
Thanks. We will need help definitively. I have been looking at what would need to be done to "port Metacello" to the base GemStone product so that the GLASS/GSDevKit packages could be loaded "directly" into the system, as opposed to the current route, which has a fair amount complicated bootstrapping involved ... I would be using only BaselineOf so the it would not be necessary to port all of Metacello to the base GemStone product ... While I think that I can carve out a "minimal Metacello" it wouldn't be completely free of complication, since I have a commitment to keep the old Metacello functioning and I cannot afford to support two independent versions of Metacello ...
If instead I worked with Christophe on this project (and it would meet my needs) then I would save myself a fair amount of work:) And part of that savings would be invested in building a Metacello-bridge to the new packages so that there would be a smooth transition ...
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) I do not know where christophe save his code but it is be public. I may not have time to read a paper ... but I would have time to read code:)
:)
It's the same reason I don't read comments in the code .... when you read the code, you get to see what is actually being done as opposed to what the developer (or comment writer) wants you to think is being done:) Often there is a disparity between what the comment says and what the code does and in my experience the code always seems to win:) Dale
Le 11 oct. 2015 à 22:32, Dale Henrichs a écrit :
When you talk about "virtual packages", I would say that a BaselineOf is pretty much a "virtual package" already.
I don't think BaselineOf could be seen as a virtual package. In package A, you tell that A depends on V. In package B, you simply tell that B provides V. Many other packages could also provide V. Then, it is the solver's job to take the most relevant package providing V.
When it comes to cross-platform support there are several factors that were built into Metacello from the very beginning:
- it should be possible to USE a package on a different platform than it was originally written for without modifying the package itself
+1 TBH, it is not yet implement in Cargo because I have a dilemnia: once you release a version, you should not edit it anymore. On another hand, you would like to say that this package released and validated on Pharo 4 is also valid for Pharo5. A good compromise would be to be able to edit only some metadata of a released package.
- it should be possible to SUBSTITUTE a platform-specific package with another platform-specific package without modifying either of the packages
with virtual packages, it becomes easy
- it should be possible to CHANGE the package dependencies based on platform-dependent requirements without modifying any of the packages
The key here is that if one needs to change a package to use it on a different platform then you lose the ability to share source code.
The BaselineOf satisfies these requirements, because the "project meta data" is separate from the package/source code and the BaselineOf itself is cross-platform.
If you haven't thought about these issues from the very beginning, then it may not be easy to shoe-horn support in after the fact ...
With our approach, metadata lies in the package for the current version and it is on the package repository when the version is released. Platform-specific packages could have their own dependencies. What will be possible is to create a new package (not a source code package but a package that is used by a package manager, i.e. metadata) to support a new platform and reusing existing code (by pointing to the package source code). Do you have an example to provide to me?
With all of that said, I really do love the idea of not having to support Metacello anymore:) So I would like to see you succeed with your effort!
It will take time ... I would really like to get feedbacks on Cargo because the knowledge you got with Metacello is invaluable.
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) I do not know where christophe save his code but it is be public. I may not have time to read a paper ... but I would have time to read code:)
http://smalltalkhub.com/#!/~demarey/CargoPackageManager Regards, Christophe
On 10/12/2015 02:58 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 22:32, Dale Henrichs a écrit :
When you talk about "virtual packages", I would say that a BaselineOf is pretty much a "virtual package" already.
I don't think BaselineOf could be seen as a virtual package. In package A, you tell that A depends on V. In package B, you simply tell that B provides V. Many other packages could also provide V. Then, it is the solver's job to take the most relevant package providing V.
ah ... I had a different definition of "virtual package" in mind:)
When it comes to cross-platform support there are several factors that were built into Metacello from the very beginning:
- it should be possible to USE a package on a different platform than it was originally written for without modifying the package itself
+1 TBH, it is not yet implement in Cargo because I have a dilemnia: once you release a version, you should not edit it anymore. On another hand, you would like to say that this package released and validated on Pharo 4 is also valid for Pharo5. A good compromise would be to be able to edit *only *some metadata of a released package.
This is another reason why I think that embedded package dependency is a bad idea .... who controls what meta data gets modified and how do you guarantee integrity. As I consider my response I think I am beginning to see where the difference in our thought processes lie ... For example my "virtual package" is defined as a specific commit in a git repository (or any other disk-based SCM) ... the version of the release is defined as the SHA of the commit, or a tag, or a branch name ... The components of this virtual package are one or more monticello packages and a BaselineOf.. One uses the virtual package by cloning the git repository to the local disk or by using a `github://` description ... I choose this approach, because in Smalltalk we typically deal with source code that has to be compiled into our working image ... I think that with your "virtual package", you are taking the approach where you are treating the Smalltalk source code as if it were a pre-compiled entity (like linux packages) and the dependencies between projects is more difficult to deal with because you have to worry about the compatibility exeutable binaries ... In the linux world these binary packages need their own dependency mechanisms independent of a development environment so indeed there are very different requirements involved here ... including the need to have dependency specifications independent of the dependency specifications used in the development environment ... If this is closer to the truth, then I will have to say that my primary concern today is for development time dependencies and the types of packages that you are talking about with the Cargo Package Manager are aimed at a completely different problem space altogether ... BTW, I will read your source code so that I can confirm my supposition ... Dale
Le 12 oct. 2015 à 19:53, Dale Henrichs a écrit :
On 10/12/2015 02:58 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 22:32, Dale Henrichs a écrit :
When you talk about "virtual packages", I would say that a BaselineOf is pretty much a "virtual package" already.
I don't think BaselineOf could be seen as a virtual package. In package A, you tell that A depends on V. In package B, you simply tell that B provides V. Many other packages could also provide V. Then, it is the solver's job to take the most relevant package providing V.
ah ... I had a different definition of "virtual package" in mind:)
When it comes to cross-platform support there are several factors that were built into Metacello from the very beginning:
- it should be possible to USE a package on a different platform than it was originally written for without modifying the package itself
+1 TBH, it is not yet implement in Cargo because I have a dilemnia: once you release a version, you should not edit it anymore. On another hand, you would like to say that this package released and validated on Pharo 4 is also valid for Pharo5. A good compromise would be to be able to edit only some metadata of a released package.
This is another reason why I think that embedded package dependency is a bad idea .... who controls what meta data gets modified and how do you guarantee integrity.
The same concern applies for non-embedded package dependency. You need to control who can modify and who cannot. I do not see your point.
As I consider my response I think I am beginning to see where the difference in our thought processes lie ...
For example my "virtual package" is defined as a specific commit in a git repository (or any other disk-based SCM) ... the version of the release is defined as the SHA of the commit, or a tag, or a branch name ...
in Cargo, I call that an Assembly. It is a set of package units working together.
The components of this virtual package are one or more monticello packages and a BaselineOf..
these components in cargo are package units.
One uses the virtual package by cloning the git repository to the local disk or by using a `github://` description ...
I choose this approach, because in Smalltalk we typically deal with source code that has to be compiled into our working image ...
I think that with your "virtual package", you are taking the approach where you are treating the Smalltalk source code as if it were a pre-compiled entity (like linux packages) and the dependencies between projects is more difficult to deal with because you have to worry about the compatibility exeutable binaries ... In the linux world these binary packages need their own dependency mechanisms independent of a development environment so indeed there are very different requirements involved here ... including the need to have dependency specifications independent of the dependency specifications used in the development environment ...
If this is closer to the truth, then I will have to say that my primary concern today is for development time dependencies and the types of packages that you are talking about with the Cargo Package Manager are aimed at a completely different problem space altogether ...
BTW, I will read your source code so that I can confirm my supposition ...
I think we talk about the same packages but with different words. Virtual packages are just there to allow low-coupling between some packages. I think this could simplify a lot some project dependency definition and save time when you need to update it. Maybe for now, you could just see Cargo managing packages. There package unit and assembly (composite pattern).
On 10/14/2015 01:00 AM, Christophe Demarey wrote:
Le 12 oct. 2015 à 19:53, Dale Henrichs a écrit :
On 10/12/2015 02:58 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 22:32, Dale Henrichs a écrit :
When you talk about "virtual packages", I would say that a BaselineOf is pretty much a "virtual package" already.
I don't think BaselineOf could be seen as a virtual package. In package A, you tell that A depends on V. In package B, you simply tell that B provides V. Many other packages could also provide V. Then, it is the solver's job to take the most relevant package providing V.
ah ... I had a different definition of "virtual package" in mind:)
When it comes to cross-platform support there are several factors that were built into Metacello from the very beginning:
- it should be possible to USE a package on a different platform than it was originally written for without modifying the package itself
+1 TBH, it is not yet implement in Cargo because I have a dilemnia: once you release a version, you should not edit it anymore. On another hand, you would like to say that this package released and validated on Pharo 4 is also valid for Pharo5. A good compromise would be to be able to edit *only *some metadata of a released package.
This is another reason why I think that embedded package dependency is a bad idea .... who controls what meta data gets modified and how do you guarantee integrity.
The same concern applies for non-embedded package dependency. You need to control who can modify and who cannot. I do not see your point.
As I consider my response I think I am beginning to see where the difference in our thought processes lie ...
For example my "virtual package" is defined as a specific commit in a git repository (or any other disk-based SCM) ... the version of the release is defined as the SHA of the commit, or a tag, or a branch name ...
in Cargo, I call that an Assembly. It is a set of package units working together.
The components of this virtual package are one or more monticello packages and a BaselineOf..
these components in cargo are package units.
One uses the virtual package by cloning the git repository to the local disk or by using a `github://` description ...
I choose this approach, because in Smalltalk we typically deal with source code that has to be compiled into our working image ...
I think that with your "virtual package", you are taking the approach where you are treating the Smalltalk source code as if it were a pre-compiled entity (like linux packages) and the dependencies between projects is more difficult to deal with because you have to worry about the compatibility exeutable binaries ... In the linux world these binary packages need their own dependency mechanisms independent of a development environment so indeed there are very different requirements involved here ... including the need to have dependency specifications independent of the dependency specifications used in the development environment ...
If this is closer to the truth, then I will have to say that my primary concern today is for development time dependencies and the types of packages that you are talking about with the Cargo Package Manager are aimed at a completely different problem space altogether ...
BTW, I will read your source code so that I can confirm my supposition ...
I think we talk about the same packages but with different words. Virtual packages are just there to allow low-coupling between some packages. I think this could simplify a lot some project dependency definition and save time when you need to update it. Maybe for now, you could just see Cargo managing packages. There package unit and assembly (composite pattern).
I agree that at this point in time we have to move beyond descriptions and start looking at source and detailed functionality, because that is where the differences between Cargo and Metacello will manifest themselves ... they are both package managers after all:) Dale
On 10/12/2015 02:58 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 22:32, Dale Henrichs a écrit :
With our approach, metadata lies in the package for the current version and it is on the package repository when the version is released. Platform-specific packages could have their own dependencies. What will be possible is to create a new package (not a source code package but a package that is used by a package manager, i.e. metadata) to support a new platform and reusing existing code (by pointing to the package source code). Do you have an example to provide to me?
With all of that said, I really do love the idea of not having to support Metacello anymore:) So I would like to see you succeed with your effort!
It will take time ... I would really like to get feedbacks on Cargo because the knowledge you got with Metacello is invaluable.
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) I do not know where christophe save his code but it is be public. I may not have time to read a paper ... but I would have time to read code:)
http://smalltalkhub.com/#!/~demarey/CargoPackageManager <http://smalltalkhub.com/#%21/%7Edemarey/CargoPackageManager>
While I think we might be trying to solve different problems ... I will take a close look at the code in CargoPackageMananger ... too bad, the code isn't on github, because it would be much easier for me to read, comment and contribute:) Dale
Le 12 oct. 2015 à 21:17, Dale Henrichs a écrit :
On 10/12/2015 02:58 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 22:32, Dale Henrichs a écrit :
With our approach, metadata lies in the package for the current version and it is on the package repository when the version is released. Platform-specific packages could have their own dependencies. What will be possible is to create a new package (not a source code package but a package that is used by a package manager, i.e. metadata) to support a new platform and reusing existing code (by pointing to the package source code). Do you have an example to provide to me?
With all of that said, I really do love the idea of not having to support Metacello anymore:) So I would like to see you succeed with your effort!
It will take time ... I would really like to get feedbacks on Cargo because the knowledge you got with Metacello is invaluable.
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) I do not know where christophe save his code but it is be public. I may not have time to read a paper ... but I would have time to read code:)
http://smalltalkhub.com/#!/~demarey/CargoPackageManager <http://smalltalkhub.com/#%21/%7Edemarey/CargoPackageManager>
While I think we might be trying to solve different problems ... I will take a close look at the code in CargoPackageMananger ... too bad, the code isn't on github, because it would be much easier for me to read, comment and contribute:)
I have github in mind but did not yet take the plunge. I will see if can do it next days.
On 10/14/2015 12:48 AM, Christophe Demarey wrote:
Le 12 oct. 2015 à 21:17, Dale Henrichs a écrit :
On 10/12/2015 02:58 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 22:32, Dale Henrichs a écrit :
With our approach, metadata lies in the package for the current version and it is on the package repository when the version is released. Platform-specific packages could have their own dependencies. What will be possible is to create a new package (not a source code package but a package that is used by a package manager, i.e. metadata) to support a new platform and reusing existing code (by pointing to the package source code). Do you have an example to provide to me?
With all of that said, I really do love the idea of not having to support Metacello anymore:) So I would like to see you succeed with your effort! It will take time ... I would really like to get feedbacks on Cargo because the knowledge you got with Metacello is invaluable.
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) I do not know where christophe save his code but it is be public. I may not have time to read a paper ... but I would have time to read code:) http://smalltalkhub.com/#!/~demarey/CargoPackageManager <http://smalltalkhub.com/#%21/%7Edemarey/CargoPackageManager>
While I think we might be trying to solve different problems ... I will take a close look at the code in CargoPackageMananger ... too bad, the code isn't on github, because it would be much easier for me to read, comment and contribute:)
I have github in mind but did not yet take the plunge. I will see if can do it next days. That would be awesome as it will make collaboration so much easier!
Dale
Le 11 oct. 2015 à 18:42, Dale Henrichs a écrit :
On 10/11/15 12:19 AM, stepharo wrote:
Le 11/10/15 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, you should read it. :) We wrote it for that and it is not boring nor long.
I scanned through it at the time and as I recall, I thought that the functionality described was already covered by git and BaselineOf ... but I did not read it in great detail and I did not (and still don't) have the time to compose a long-winded response:)
but I am curious how you think "package dependencies" will play with git-based projects?
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together.
I think that this is only true for packages committed within the same repo. Now between porjects published in different repo you have to express them. I do not think that we all want to publish in the same repo and clone out everything.
... and inter-project dependencies is what a BaselineOf does .... which brings me back to the conclusion that I reached when I scanned the paper:)
In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand.
We want to be able to have a package market place where tools can grab dependencies without to load code and can compute the set of packages that should be loaded.
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution. Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand.
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ...
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:)
Well, the point is not to replace metacello but to go towards a per package metadata description allowing some flexibility with the introduction of virtual packages. This will allow, in a first time, to set up a package repository and more important, a web site on top of it. In a second time, I also want to enable more flexibility in expressing dependencies constraints (eg. > 2.0, 3.*, etc.). To achieve that, you need a very performant dependency solver and I would like to reuse linux ones (it has be done for ocaml by example) through CUDF (check http://mancoosi.org/). For now, I implemented a simple solver for static dependency constraints (=1.2). I checked Metacello implementation and it looked to me that approaches are a bit too far to be able to reuse the whole code. For sure, it is not as robust as Metacello is, because you enhanced it for years. What I want now, is to experiment (let's name it) Cargo Package Manager to see if it fits the needs. Pharo bootstrap is the first use case. What I would like, is that, at the end, the dependency solver get out of Pharo and that we use already existing linux ones. What will stay into Pharo are packages metadata and the way to load a list of packages.
2015-10-12 11:20 GMT+02:00 Christophe Demarey <Christophe.Demarey@inria.fr>:
Well, the point is not to replace metacello but to go towards a per package metadata description allowing some flexibility with the introduction of virtual packages. This will allow, in a first time, to set up a package repository and more important, a web site on top of it. In a second time, I also want to enable more flexibility in expressing dependencies constraints (eg. > 2.0, 3.*, etc.). To achieve that, you need a very performant dependency solver and I would like to reuse linux ones (it has be done for ocaml by example) through CUDF (check http://mancoosi.org/).
For now, I implemented a simple solver for static dependency constraints (=1.2). I checked Metacello implementation and it looked to me that approaches are a bit too far to be able to reuse the whole code. For sure, it is not as robust as Metacello is, because you enhanced it for years. What I want now, is to experiment (let's name it) Cargo Package Manager to see if it fits the needs. Pharo bootstrap is the first use case.
What I would like, is that, at the end, the dependency solver get out of Pharo and that we use already existing linux ones. What will stay into Pharo are packages metadata and the way to load a list of packages.
This is something I don't understand: why this focus on making this out of Pharo? What CUDF shows is that, overall, a package dependency problem can be expressed as a SAT problem which is then solvable by the appropriate solver... For example, in my ubuntu, I see that I have three (!) CUDF dependency solvers (mccf, packup, aspdcup), where mccf can use the following underlying solvers: lpsolve, cbc, Cplex, Gurobi, Glpk, SCIP or WBO (!). More interesting is packup, which rely on minisat+. Minisat, with a short search, shows up as 600 lignes of C++ (https://github.com/niklasso/minisat or http://minisat.se/). Why not porting Minisat in Pharo then? You would simplify your building your dependency issues (no need to link Pharo to yet another external tool which only works on Mac OS X / Linux :( )... And that would make such a solver available for other uses. (I think Metacello could switch to using Minisat if a Smalltalk port was available... ) In the mean-time, the number of available CUDF tools in Linux shows probably nice integration issues with dependency loading depending, on some cases, on the solver selected... cool :( Thierry
Le 12 oct. 2015 à 14:03, Thierry Goubier a écrit :
2015-10-12 11:20 GMT+02:00 Christophe Demarey <Christophe.Demarey@inria.fr>:
Well, the point is not to replace metacello but to go towards a per package metadata description allowing some flexibility with the introduction of virtual packages. This will allow, in a first time, to set up a package repository and more important, a web site on top of it. In a second time, I also want to enable more flexibility in expressing dependencies constraints (eg. > 2.0, 3.*, etc.). To achieve that, you need a very performant dependency solver and I would like to reuse linux ones (it has be done for ocaml by example) through CUDF (check http://mancoosi.org/).
For now, I implemented a simple solver for static dependency constraints (=1.2). I checked Metacello implementation and it looked to me that approaches are a bit too far to be able to reuse the whole code. For sure, it is not as robust as Metacello is, because you enhanced it for years. What I want now, is to experiment (let's name it) Cargo Package Manager to see if it fits the needs. Pharo bootstrap is the first use case.
What I would like, is that, at the end, the dependency solver get out of Pharo and that we use already existing linux ones. What will stay into Pharo are packages metadata and the way to load a list of packages.
This is something I don't understand: why this focus on making this out of Pharo?
The focus is not there but I'm not a specialist of Constraint Satisfaction Problems (CSP), I'm not aware of CSP implementation in Pharo and so, I just want to reuse something working fine for years. We are not so much in the community and we should take care to do not have to maintain too many things, especially when it has little added value.
What CUDF shows is that, overall, a package dependency problem can be expressed as a SAT problem which is then solvable by the appropriate solver... For example, in my ubuntu, I see that I have three (!) CUDF dependency solvers (mccf, packup, aspdcup), where mccf can use the following underlying solvers: lpsolve, cbc, Cplex, Gurobi, Glpk, SCIP or WBO (!).
right
More interesting is packup, which rely on minisat+. Minisat, with a short search, shows up as 600 lignes of C++ (https://github.com/niklasso/minisat or http://minisat.se/). Why not porting Minisat in Pharo then?
could be an idea. Also good to know, all CUDF backends does not have the same speed. I keep that in mind. It would be good to have a CSP solver available in Pharo.
You would simplify your building your dependency issues (no need to link Pharo to yet another external tool which only works on Mac OS X / Linux :( )... And that would make such a solver available for other uses.
My idea was to provide the solving facility as a Rest service and so, do not have to care about OS portability.
(I think Metacello could switch to using Minisat if a Smalltalk port was available... )
In the mean-time, the number of available CUDF tools in Linux shows probably nice integration issues with dependency loading depending, on some cases, on the solver selected... cool :(
oh ... Thanks for this information, Christophe
On 10/12/2015 02:20 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 18:42, Dale Henrichs a écrit :
On 10/11/15 12:19 AM, stepharo wrote:
Le 11/10/15 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, you should read it. :) We wrote it for that and it is not boring nor long.
I scanned through it at the time and as I recall, I thought that the functionality described was already covered by git and BaselineOf ... but I did not read it in great detail and I did not (and still don't) have the time to compose a long-winded response:)
but I am curious how you think "package dependencies" will play with git-based projects?
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together. I think that this is only true for packages committed within the same repo. Now between porjects published in different repo you have to express them. I do not think that we all want to publish in the same repo and clone out everything. ... and inter-project dependencies is what a BaselineOf does .... which brings me back to the conclusion that I reached when I scanned the paper:) In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand. We want to be able to have a package market place where tools can grab dependencies without to load code and can compute the set of packages that should be loaded.
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution. Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand.
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ...
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:)
Well, the point is not to replace metacello but to go towards a per package metadata description allowing some flexibility with the introduction of virtual packages. Oh darn, you mean I have to continue to support Metacello:) This will allow, in a first time, to set up a package repository and more important, a web site on top of it. In a second time, I also want to enable more flexibility in expressing dependencies constraints (eg. > 2.0, 3.*, etc.). To achieve that, you need a very performant dependency solver and I would like to reuse linux ones (it has be done for ocaml by example) through CUDF (check http://mancoosi.org/ There are a couple of different schemes for expressing "version ranges" in Metacello. Have you looked at them?
By reuse, do you mean that you will reimplement the algorithms in Smalltalk or are you suggesting that depency resolution will only work on linux? From a practical perspective I am curious what problems will be solved by having "dependency constraints" expressed by anything more complicated than what I use for github: github://dalehenrich/tode:v0.0.?/repository Which translates to load the latest patch version of the tode project (where major.minor.patch) ... I know that in theory more complicated schemes are intersting, but that assumes that the developers assigning version numbers are actually adhering to a rational version numbering scheme and from a practical matter even the above pattern is risky business:)
For now, I implemented a simple solver for static dependency constraints (=1.2). I checked Metacello implementation and it looked to me that approaches are a bit too far to be able to reuse the whole code. For sure, it is not as robust as Metacello is, because you enhanced it for years. What I want now, is to experiment (let's name it) Cargo Package Manager to see if it fits the needs. Pharo bootstrap is the first use case.
okay, so this is just in the experimental phases .... so for the time being I should be focussing my efforts on the minimal Metacello approach rather than get involved in the Cargo Package Manager? Dale
Le 12 oct. 2015 à 19:22, Dale Henrichs a écrit :
On 10/12/2015 02:20 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 18:42, Dale Henrichs a écrit :
On 10/11/15 12:19 AM, stepharo wrote:
Le 11/10/15 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, you should read it. :) We wrote it for that and it is not boring nor long.
I scanned through it at the time and as I recall, I thought that the functionality described was already covered by git and BaselineOf ... but I did not read it in great detail and I did not (and still don't) have the time to compose a long-winded response:)
but I am curious how you think "package dependencies" will play with git-based projects?
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together. I think that this is only true for packages committed within the same repo. Now between porjects published in different repo you have to express them. I do not think that we all want to publish in the same repo and clone out everything. ... and inter-project dependencies is what a BaselineOf does .... which brings me back to the conclusion that I reached when I scanned the paper:) In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand. We want to be able to have a package market place where tools can grab dependencies without to load code and can compute the set of packages that should be loaded.
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution. Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand.
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ...
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:)
Well, the point is not to replace metacello but to go towards a per package metadata description allowing some flexibility with the introduction of virtual packages. Oh darn, you mean I have to continue to support Metacello:) This will allow, in a first time, to set up a package repository and more important, a web site on top of it. In a second time, I also want to enable more flexibility in expressing dependencies constraints (eg. > 2.0, 3.*, etc.). To achieve that, you need a very performant dependency solver and I would like to reuse linux ones (it has be done for ocaml by example) through CUDF (check http://mancoosi.org/ There are a couple of different schemes for expressing "version ranges" in Metacello. Have you looked at them?
No, I did not check on latest Metacello versions. What is supported?
By reuse, do you mean that you will reimplement the algorithms in Smalltalk or are you suggesting that depency resolution will only work on linux?
I would like to define the dependency solving as a rest service, so no need to support multiple platforms. the dependency solving can be hosted on linux.
From a practical perspective I am curious what problems will be solved by having "dependency constraints" expressed by anything more complicated than what I use for github:
github://dalehenrich/tode:v0.0.?/repository
Which translates to load the latest patch version of the tode project (where major.minor.patch) ... I know that in theory more complicated schemes are intersting, but that assumes that the developers assigning version numbers are actually adhering to a rational version numbering scheme and from a practical matter even the above pattern is risky business:)
you're right but we should encourage people to adopt semantic versionning. It's very powerful when semantic versioning policy is applied.
For now, I implemented a simple solver for static dependency constraints (=1.2). I checked Metacello implementation and it looked to me that approaches are a bit too far to be able to reuse the whole code. For sure, it is not as robust as Metacello is, because you enhanced it for years. What I want now, is to experiment (let's name it) Cargo Package Manager to see if it fits the needs. Pharo bootstrap is the first use case.
okay, so this is just in the experimental phases .... so for the time being I should be focussing my efforts on the minimal Metacello approach rather than get involved in the Cargo Package Manager?
What I wanted to say is that is still experimental for me because it does not have yet users. Maybe we could take Gemstone and GLASS/GSDevKit package as a use case and see if Cargo fits your needs. I would really enjoy to join forces and have a better solution. What are your requirements for having GLASS/GSDevKit loadable into GemStone. Do you have a deadline? I will have more time starting November (for now, I work on the bootstrap with Max for 2 weeks). Christophe
On 10/14/2015 12:39 AM, Christophe Demarey wrote:
Le 12 oct. 2015 à 19:22, Dale Henrichs a écrit :
On 10/12/2015 02:20 AM, Christophe Demarey wrote:
Le 11 oct. 2015 à 18:42, Dale Henrichs a écrit :
On 10/11/15 12:19 AM, stepharo wrote:
Le 11/10/15 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, you should read it. :) We wrote it for that and it is not boring nor long. I scanned through it at the time and as I recall, I thought that the functionality described was already covered by git and BaselineOf ... but I did not read it in great detail and I did not (and still don't) have the time to compose a long-winded response:) but I am curious how you think "package dependencies" will play with git-based projects?
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together. I think that this is only true for packages committed within the same repo. Now between porjects published in different repo you have to express them. I do not think that we all want to publish in the same repo and clone out everything. ... and inter-project dependencies is what a BaselineOf does .... which brings me back to the conclusion that I reached when I scanned the paper:) In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand. We want to be able to have a package market place where tools can grab dependencies without to load code and can compute the set of packages that should be loaded.
When a package is released into the market: then it externalise its metadata so that a crawler can automatically build dependency graph and create specific distribution. Okay. This is a problem .... but it happens to be a problem that Metacello "can solve/does solve" - so there must be something else (a deeper problem?) that I don't quite understand.
With that said, if you are planning to replace Metacello, then I am excited:) But I will repeat that I hope that you are considering cross platform issues ...
Perhaps at this point in time, I'd like to read some code. Then I can skip reading the paper and get a feel for how hard it will be to port to GemStone:) Well, the point is not to replace metacello but to go towards a per package metadata description allowing some flexibility with the introduction of virtual packages. Oh darn, you mean I have to continue to support Metacello:) This will allow, in a first time, to set up a package repository and more important, a web site on top of it. In a second time, I also want to enable more flexibility in expressing dependencies constraints (eg. > 2.0, 3.*, etc.). To achieve that, you need a very performant dependency solver and I would like to reuse linux ones (it has be done for ocaml by example) through CUDF (check http://mancoosi.org/ There are a couple of different schemes for expressing "version ranges" in Metacello. Have you looked at them?
No, I did not check on latest Metacello versions. What is supported? There are a number of operators that can be specified on a project by project basis including ~>[1] ... there never seemed to be a big demand for these operations, presumably because so few developers adhere to a semantic versioning scheme (which makes trying to reason about version numbers very hard) ... and I think that the symbolic version naming convention invented by the Seaside guys: #release3, #release3.1, #release3.2, etc. is superior (from a practical perspective) ...
[1] https://robots.thoughtbot.com/rubys-pessimistic-operator
By reuse, do you mean that you will reimplement the algorithms in Smalltalk or are you suggesting that depency resolution will only work on linux? I would like to define the dependency solving as a rest service, so no need to support multiple platforms. the dependency solving can be hosted on linux.
So you can't do a build using CargoPackageManager without being connected to the internet ... As I said in another message, I think that you are trying to solve a different problem than Metacello ... there are a number of similarities in the contents of the meta data that is being used (and common specification classes at this level would probably be beneficial), but Metacello is primarily aimed at developers building and using packages produced by others ... I am taking the approach that for developers an SCM (I talk about git all the time, but any disk-based SCM can be used) is part an parcel of of not only developing your own projects but when using other projects, you inevitably have to a) customize them for your own use or b) contribute bugfixes back to the community and an SCM is required for that ...
From a practical perspective I am curious what problems will be solved by having "dependency constraints" expressed by anything more complicated than what I use for github:
github://dalehenrich/tode:v0.0.?/repository
Which translates to load the latest patch version of the tode project (where major.minor.patch) ... I know that in theory more complicated schemes are intersting, but that assumes that the developers assigning version numbers are actually adhering to a rational version numbering scheme and from a practical matter even the above pattern is risky business:) you're right but we should encourage people to adopt semantic versionning. It's very powerful when semantic versioning policy is applied.
For now, I implemented a simple solver for static dependency constraints (=1.2). I checked Metacello implementation and it looked to me that approaches are a bit too far to be able to reuse the whole code. For sure, it is not as robust as Metacello is, because you enhanced it for years. What I want now, is to experiment (let's name it) Cargo Package Manager to see if it fits the needs. Pharo bootstrap is the first use case. okay, so this is just in the experimental phases .... so for the time being I should be focussing my efforts on the minimal Metacello approach rather than get involved in the Cargo Package Manager? What I wanted to say is that is still experimental for me because it does not have yet users. Maybe we could take Gemstone and GLASS/GSDevKit package as a use case and see if Cargo fits your needs. I would really enjoy to join forces and have a better solution. What are your requirements for having GLASS/GSDevKit loadable into GemStone. Do you have a deadline? I will have more time starting November (for now, I work on the bootstrap with Max for 2 weeks).
post-November will work best for me as I am targeting a talk on GsDevKit_home and tODE for Smalltalks in November ... I don't have a deadline, but the need to have FileTree and Metacello-lite integrated in the base GemStone is bubbling close to the top ... and post-November would be a good time ... Dale
Hi Dale, Le 11 oct. 2015 à 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, but I am curious how you think "package dependencies" will play with git-based projects?
Dependencies are not tied to a Version Control System (monticello, git or whatever). Dependencies are a package concern. With a released version, at the end, we need to fech source code from a VCS (as we do not have a shared binary format): these steps are already done by MC*Repository classes, including git.
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together.
Git allows you to easily reference a set of packages working together. It works fine for packages of the same repository but you get back the problem since you deal with packages of other repositories.
In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
For the first step of the bootstrap, it will work but not for next steps where we will split the Pharo image into different projects.
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand.
When you develop, you have a working copy of a package meta-data, including dependencies. Actually, there are current dependencies of the package. You could avoid to refer to specific versions and just point to the package name as your working image should already have packages loaded. (kind of configurationOf baseline) When you release a version (strong act), then you "freeze" the current working version of the package meta-data and you publish it somewhere (a package repository) so that it becomes available to others. This metadata is not source cod, is easily accessible by tools and it becomes easy to build a web site on top of this to search / promote packages. So, the problems I'm trying to solve there are: do not be tied to a VCS and do not need to load code to "play" with packages metadata offer a central place to easily find the package you need (for example, java has http://central.sonatype.org/, php has https://packagist.org/, etc.) do not mix preoccupations: I do not want to have metadata of all released versions + working copy of a package at the same place Christophe
On 12-10-15 10:42, Christophe Demarey wrote:
When you develop, you have a working copy of a package meta-data, including dependencies. Actually, there are current dependencies of the package. You could avoid to refer to specific versions and just point to the package name as your working image should already have packages loaded. (kind of configurationOf baseline) When you release a version (strong act), then you "freeze" the current working version of the package meta-data and you publish it somewhere (a package repository) so that it becomes available to others. This metadata is not source cod, is easily accessible by tools and it becomes easy to build a web site on top of this to search / promote packages.
What does releasing mean here? Could you frame it in terms of the 5D paper and the problems they describe? Stephna
Le 12 oct. 2015 à 12:53, Stephan Eggermont a écrit :
On 12-10-15 10:42, Christophe Demarey wrote:
When you develop, you have a working copy of a package meta-data, including dependencies. Actually, there are current dependencies of the package. You could avoid to refer to specific versions and just point to the package name as your working image should already have packages loaded. (kind of configurationOf baseline) When you release a version (strong act), then you "freeze" the current working version of the package meta-data and you publish it somewhere (a package repository) so that it becomes available to others. This metadata is not source cod, is easily accessible by tools and it becomes easy to build a web site on top of this to search / promote packages.
What does releasing mean here? Could you frame it in terms of the 5D paper and the problems they describe?
Regarding the 5D paper, what I call release is the version dimension, where you edit design information. "the designers typically work on a nonversioned copy of the data files which is kept separate from the versioned copies stored in the archive" the nonversioned copy (in our case is versionned as source code but not as a release) is what I called the working copy of package metadata. A released version is a versioned copy of package metadata published in the package repository. It means a release version has reach some level of maturity and is ready to use. Christophe
On 14-10-15 08:25, Christophe Demarey wrote:
What does releasing mean here? Could you frame it in terms of the 5D paper and the problems they describe?
Regarding the 5D paper, what I call release is the version dimension, where you edit design information. "the designers typically work on a nonversioned copy of the data files which is kept separate from the versioned copies stored in the archive"
the nonversioned copy (in our case is versionned as source code but not as a release) is what I called the working copy of package metadata. A released version is a versioned copy of package metadata published in the package repository. It means a release version has reach some level of maturity and is ready to use.
If I try applying the 5D model, if releasing says something about the maturity level it should be mapped to status. Platform maps to variant, and the package maps to hierarchy. Further in the paper they talk about the interactions between the different dimensions. A conclusion in the paper is that if you want to manage different dimensions, you actually describe a development proces. The needed artifacts and structure follows directly from describing the process(es) you want to support. We want tooling that can support different workflows, as different projects have different needs. That is why I suggested starting from personas and exactly describing how they solve the issues they run into and how they want to work. Stephan
On 10/12/2015 01:42 AM, Christophe Demarey wrote:
Hi Dale,
Le 11 oct. 2015 à 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, but I am curious how you think "package dependencies" will play with git-based projects?
Dependencies are not tied to a Version Control System (monticello, git or whatever). Dependencies are a package concern. With a released version, at the end, we need to fech source code from a VCS (as we do not have a shared binary format): these steps are already done by MC*Repository classes, including git. Yes but I am asking specifically how this link is specified ... in Metacello, you use repository descriptions to identify the source repository. I am curious how you specify these cross repository dependencies ... or do you even support cross-repository dependencies?
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together.
Git allows you to easily reference a set of packages working together. It works fine for packages of the same repository but you get back the problem since you deal with packages of other repositories. right and in Metacello the cross-repository specifications is handled by a project spec:
spec baselineOf: 'Seaside' repository: 'github://SeasideSt/Seaside:v3.2.0/repository' In this case the git version is specified as part of the repository spec (v3.2.0 is a git tag) ... it gets back to how do you handle cross repository references?
In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
For the first step of the bootstrap, it will work but not for next steps where we will split the Pharo image into different projects.
and how are project dependencies handled? It seems that the project dependencies are intimately tied to package dependencies as often the project dependency itself comes from a specific set of packages and needs to be specified at the package level ...
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand.
When you develop, you have a working copy of a package meta-data, including dependencies. Actually, there are current dependencies of the package. You could avoid to refer to specific versions and just point to the package name as your working image should already have packages loaded. (kind of configurationOf baseline)
This is a BaselineOf in Metacello ... and does exactly what you talk about here ...
When you release a version (strong act), then you "freeze" the current working version of the package meta-data and you publish it somewhere (a package repository) so that it becomes available to others. This metadata is not source cod, is easily accessible by tools and it becomes easy to build a web site on top of this to search / promote packages. So, the problems I'm trying to solve there are:
* do not be tied to a VCS and do not need to load code to "play" with packages metadata
As I've mentioned in another message, there is no reason other than a lack of tools that the Metacello specifications (ConfigurationOf and BaselineOf) are not stored in XML/JSON/STON files ... I never liked the idea that code had to be loaded, but it was expedient at the time ...
* offer a central place to easily find the package you need (for example, java has http://central.sonatype.org/, php has https://packagist.org/, etc.)
I am curious about your use of the term "package" here ... is this "package" as in Monticello package, or "package" as in a collection of packages and project dependencies ... or ConfigurationOf or BaselineOf?
* do not mix preoccupations: I do not want to have metadata of all released versions + working copy of a package at the same place
Have you looked at a BaselineOf? I also don't like the fact that ConfigurationOf has release and version information embedded in it ... but the ConfigurationOf was invented to plug the gap between what was available in the Monticello eco-system and what is available in a full-featured scm like git. The BaselineOf was invented because git was able to take care of all of version relationships and Metacello no longer needed all of that junk. If you look at a BaselineOf you will see that it is reduced to a single baseline method with package dependencies specified by name (the entire BaselineOf applies to all of the packages in a directory on disk managed by git or whatever) very simple very compact and very easy to maintain ... the only reason one touches a BaselineOf is to change a dependency.... So I think that for what you seem to need all that needs to be done is to define an XML/JSON/STON representation for the data in a BaselineOf ... a generator for a different format could easily be written and could take existing BaselineOf and spit out the data ... Dale
Le 12 oct. 2015 à 19:01, Dale Henrichs a écrit :
On 10/12/2015 01:42 AM, Christophe Demarey wrote:
Hi Dale,
Le 11 oct. 2015 à 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, but I am curious how you think "package dependencies" will play with git-based projects?
Dependencies are not tied to a Version Control System (monticello, git or whatever). Dependencies are a package concern. With a released version, at the end, we need to fech source code from a VCS (as we do not have a shared binary format): these steps are already done by MC*Repository classes, including git. Yes but I am asking specifically how this link is specified ... in Metacello, you use repository descriptions to identify the source repository. I am curious how you specify these cross repository dependencies ... or do you even support cross-repository dependencies?
I also use repository descriptions as in Metacello. Here is an example of serialized metadata of a package with cargo: CGOPackageUnit { #package : #Grease-Core, #description : ''Core package of the Grease library.'', #version : ''1.1.13'', #timestamp : DateAndTime [ ''2015-04-14T14:51:50.116+00:00'' ], #dependencies : [ ], #repositories : [ ''http://smalltalkhub.foo/mc/Seaside/Grease11/main/'' ], #file : ''Grease-Core-JohanBrichau.94'' }
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together.
Git allows you to easily reference a set of packages working together. It works fine for packages of the same repository but you get back the problem since you deal with packages of other repositories.
right and in Metacello the cross-repository specifications is handled by a project spec:
spec baselineOf: 'Seaside' repository: 'github://SeasideSt/Seaside:v3.2.0/repository'
In this case the git version is specified as part of the repository spec (v3.2.0 is a git tag) ... it gets back to how do you handle cross repository references?
I get metadata on packages at the same central place: the package repository (You could have more that one if needed, e.g. a private package repository). In the package metadata, I have the information to get back the source code artifact (repositories and file fields).
In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
For the first step of the bootstrap, it will work but not for next steps where we will split the Pharo image into different projects. and how are project dependencies handled? It seems that the project dependencies are intimately tied to package dependencies as often the project dependency itself comes from a specific set of packages and needs to be specified at the package level ...
in the current implementation, there is no distinction between a project dependency (an external dependency) and a package dependency (an internal dependency, i.e. a dependency to a package of the same project).
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand.
When you develop, you have a working copy of a package meta-data, including dependencies. Actually, there are current dependencies of the package. You could avoid to refer to specific versions and just point to the package name as your working image should already have packages loaded. (kind of configurationOf baseline)
This is a BaselineOf in Metacello ... and does exactly what you talk about here ...
yes
When you release a version (strong act), then you "freeze" the current working version of the package meta-data and you publish it somewhere (a package repository) so that it becomes available to others. This metadata is not source cod, is easily accessible by tools and it becomes easy to build a web site on top of this to search / promote packages. So, the problems I'm trying to solve there are: do not be tied to a VCS and do not need to load code to "play" with packages metadata As I've mentioned in another message, there is no reason other than a lack of tools that the Metacello specifications (ConfigurationOf and BaselineOf) are not stored in XML/JSON/STON files ... I never liked the idea that code had to be loaded, but it was expedient at the time ... I understand
offer a central place to easily find the package you need (for example, java has http://central.sonatype.org/, php has https://packagist.org/, etc.) I am curious about your use of the term "package" here ... is this "package" as in Monticello package, no or "package" as in a collection of packages and project dependencies ... or ConfigurationOf or BaselineOf? package : the unit you want to deliver. It may be a package unit refering to one Monticello package or a package assembly, i.e. a set of package units and so, a set of Monticello packages.
do not mix preoccupations: I do not want to have metadata of all released versions + working copy of a package at the same place Have you looked at a BaselineOf?
Yes, I like it but it is not available for Monticello. If we did not have Monticello but git instead, life would be easier.
I also don't like the fact that ConfigurationOf has release and version information embedded in it ... but the ConfigurationOf was invented to plug the gap between what was available in the Monticello eco-system and what is available in a full-featured scm like git.
good to know historical reasons ;)
The BaselineOf was invented because git was able to take care of all of version relationships and Metacello no longer needed all of that junk.
If you look at a BaselineOf you will see that it is reduced to a single baseline method with package dependencies specified by name (the entire BaselineOf applies to all of the packages in a directory on disk managed by git or whatever) very simple very compact and very easy to maintain ... the only reason one touches a BaselineOf is to change a dependency....
So I think that for what you seem to need all that needs to be done is to define an XML/JSON/STON representation for the data in a BaselineOf ... a generator for a different format could easily be written and could take existing BaselineOf and spit out the data ...
But how do you handle dependencies of projects using Monticello?
On 10/14/2015 12:15 AM, Christophe Demarey wrote:
Le 12 oct. 2015 à 19:01, Dale Henrichs a écrit :
On 10/12/2015 01:42 AM, Christophe Demarey wrote:
Hi Dale,
Le 11 oct. 2015 à 00:40, Dale Henrichs a écrit :
Christophe,
I still don't have a lot of time to read the paper and try to understand what you are trying to accomplish, but I am curious how you think "package dependencies" will play with git-based projects?
Dependencies are not tied to a Version Control System (monticello, git or whatever). Dependencies are a package concern. With a released version, at the end, we need to fech source code from a VCS (as we do not have a shared binary format): these steps are already done by MC*Repository classes, including git. Yes but I am asking specifically how this link is specified ... in Metacello, you use repository descriptions to identify the source repository. I am curious how you specify these cross repository dependencies ... or do you even support cross-repository dependencies?
I also use repository descriptions as in Metacello. Here is an example of serialized metadata of a package with cargo:
CGOPackageUnit { #package : #Grease-Core, #description : ''Core package of the Grease library.'', #version : ''1.1.13'', #timestamp : DateAndTime [ ''2015-04-14T14:51:50.116+00:00'' ], #dependencies : [ ], #repositories : [ ''http://smalltalkhub.foo/mc/Seaside/Grease11/main/'' ], #file : ''Grease-Core-JohanBrichau.94'' }
I was asking about cross repository dependencies and in this example your #dependencies array is empty ... What repository does the CGOPackageUnit reside in?
In git-based repositories I don't think you have the same type of dependency issues that one might have with monticello repositories --- In a monticello repository you have a whole range of possible package versions to pick from, but in a git-based repository the range of package versions is fixed to those that are committed together so the packages that are meant to work together are committed together.
Git allows you to easily reference a set of packages working together. It works fine for packages of the same repository but you get back the problem since you deal with packages of other repositories.
right and in Metacello the cross-repository specifications is handled by a project spec:
spec baselineOf: 'Seaside' repository: 'github://SeasideSt/Seaside:v3.2.0/repository'
In this case the git version is specified as part of the repository spec (v3.2.0 is a git tag) ... it gets back to how do you handle cross repository references?
I get metadata on packages at the same central place: the package repository (You could have more that one if needed, e.g. a private package repository). So you have a completely separate repository of packages....
On a sort of related note ... you are talking about packages here as if they were a complete unit, but my experience with packages is that they are very rarely completely standlone entities ... Most of my projects involve multiple packages that are tightly coupled to each other ... Am I expected to first register each package in the central repository (or private repository) before I can even think of composing a multi-package project ... I assume that I am missing something here ...
In the package metadata, I have the information to get back the source code artifact (repositories and file fields).
In the bootstrap scenario, you would only have one version per package to choose from, so the packages that are meant to work together are committed together ....
For the first step of the bootstrap, it will work but not for next steps where we will split the Pharo image into different projects. and how are project dependencies handled? It seems that the project dependencies are intimately tied to package dependencies as often the project dependency itself comes from a specific set of packages and needs to be specified at the package level ...
in the current implementation, there is no distinction between a project dependency (an external dependency) and a package dependency (an internal dependency, i.e. a dependency to a package of the same project). except that they are presumably specified differently? an example specification would be useful here ...
I guess I don't know what you mean when you say:
we want to decouple a released version of a package from the working copy of the package version description (implies the creation of a package repository + a web interface on top of it to promote/search packages). Perhaps a description of the problem being solved would help me understand.
When you develop, you have a working copy of a package meta-data, including dependencies. Actually, there are current dependencies of the package. You could avoid to refer to specific versions and just point to the package name as your working image should already have packages loaded. (kind of configurationOf baseline)
This is a BaselineOf in Metacello ... and does exactly what you talk about here ...
yes
When you release a version (strong act), then you "freeze" the current working version of the package meta-data and you publish it somewhere (a package repository) so that it becomes available to others. This metadata is not source cod, is easily accessible by tools and it becomes easy to build a web site on top of this to search / promote packages. So, the problems I'm trying to solve there are:
* do not be tied to a VCS and do not need to load code to "play" with packages metadata
As I've mentioned in another message, there is no reason other than a lack of tools that the Metacello specifications (ConfigurationOf and BaselineOf) are not stored in XML/JSON/STON files ... I never liked the idea that code had to be loaded, but it was expedient at the time ... I understand
* offer a central place to easily find the package you need (for example, java has http://central.sonatype.org/, php has https://packagist.org/, etc.)
I am curious about your use of the term "package" here ... is this "package" as in Monticello package, no or "package" as in a collection of packages and project dependencies ... or ConfigurationOf or BaselineOf? package : the unit you want to deliver. It may be a package unit refering to one Monticello package or a package assembly, i.e. a set of package units and so, a set of Monticello packages. okay and an assembly of packages looks like?
* do not mix preoccupations: I do not want to have metadata of all released versions + working copy of a package at the same place
Have you looked at a BaselineOf?
Yes, I like it but it is not available for Monticello. If we did not have Monticello but git instead, life would be easier. A BaselineOf can be used with a Monticello repository but it devolves to a #bleedingEdge specification which is a nightmare ...
Life should be easy ... I guess if your main focus is providing an alternate solution for Monticello repositories then I wish you luck... I don't have the cycles available to try to bridge the gap between Monticello repositories and git repositories ... I prefer to spend my time making git repositories usable with Smalltalk:)
I also don't like the fact that ConfigurationOf has release and version information embedded in it ... but the ConfigurationOf was invented to plug the gap between what was available in the Monticello eco-system and what is available in a full-featured scm like git.
good to know historical reasons ;)
The BaselineOf was invented because git was able to take care of all of version relationships and Metacello no longer needed all of that junk.
If you look at a BaselineOf you will see that it is reduced to a single baseline method with package dependencies specified by name (the entire BaselineOf applies to all of the packages in a directory on disk managed by git or whatever) very simple very compact and very easy to maintain ... the only reason one touches a BaselineOf is to change a dependency....
So I think that for what you seem to need all that needs to be done is to define an XML/JSON/STON representation for the data in a BaselineOf ... a generator for a different format could easily be written and could take existing BaselineOf and spit out the data ...
But how do you handle dependencies of projects using Monticello?
A BaselineOf can specify a dependency on another project and that project may be a BaselineOf or a ConfigurationOf ... A ConfigurationOf can be serialized to STON just like a BaselineOf ... it will just have a lot more junk it ... junk that is managed by git in a git repository ... Dale
participants (6)
-
Christophe Demarey -
Dale Henrichs -
Skip Lentz -
Stephan Eggermont -
stepharo -
Thierry Goubier