I am afraid this mail is going to be too long, but I feel it needs to be written. I will start with the conclusion, so you can skip the mail if you feel it does not interest you. Please keep the discussion on the Moose list, as there the discussion was started. I am very happy that versioner is being built. We need more and better software configuration tools, and versioner will make our life easier. That being said, I am getting a bit nervous about how versioner is released. It should aim to improve the way people can work, and not to create a single workflow for all people working on pharo projects. I hope my fears are unfounded. Software configuration management (SCM) is in my opinion an undervalued topic within the software world. One cannot find very many good books on them, and I am afraid this lack of knowledge shows in the way people go around with their SCM. So I would like to write down some observations of the SCM processes in Pharo that I believe we should keep in mind. The main target of SCM is to keep the impact of changes as minimal as possible. Good SCM allows all people working on a certain project, to make their changes, without causing trouble for other project members and people using this project. Open source projects are (in SCM terms) rather complex, as they involve a lot of people, including some people we do not know. 1. One of the best things of Metacello is that it does not require you to follow a certain workflow. It is very flexible and supports a large range of possible workflows. Within the Pharo community we have at least 3 different workflows: A. For Pharo core, we have a pharo inbox, where suggested fixes for reported bugs are put. These fixes are reviewed and, if accepted, integrated into Pharo. This ensures that the changes in Pharo hold (most of the time) to quite some quality standards. B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all suggested fixes are put in the main repository on smalltalkhub. The fixes are, after review and tests, released by putting them in a âreleasedâ version of the configuration. Since not all fixes should be in all versions, and sometimes only concern a certain platform, the process for Pharo core would not work: releasing fixes are more complex because of the cross-platform issues. C. For the âmooseâ projects all suggested fixes are put in the main repository and are accepted by default. Since most projects evolve very fast, quality assurance is done afterwards, by submitting bug fixes. I can say a lot about why I believe these workflows are appropriate for their different projects, but the point I am trying to make here, is that these workflows have come to existence because of the different demands on them. And to stress my point about the flexibility (thank you Dale): When there was a problem with the workflow in the cross platform projects, I could find a solution (using releases) that Metacello supported. 2. Unfortunately Metacello does not have a good UI. And for large configurations like Seaside, it is a lot of work to keep them maintained. I dream of a tool that acts as a UI for Metacello: this tool is flexible as Metacello, while making the simple things real easy (hitting a button and done). Does versioner aim to do this? 3. I always get very nervous when someone says: They do it like this, and that works very well for them. So we should do that too. I do not believe that is always a good idea. The conclusion should be a question: Can we use that too? And now I cannot find the mail about the java SCM practice, but I do not think we can apply it to all our processes ... 4. Modularity is very important. Also for performing good SCM. And yes, we lack sufficient modularity in Moose. It should be better. I.e. because of the lack of modularity, it is sometimes hard to distinguish between where the change needs to be put in. So there is no clear group to point out who should accept the changes. So this is also an SCM problem. Cleaning up the configuration will help here very much. Thank you Steff for the good work here. And maybe you also have a point that the people working on Moose should have more focus on keeping things modular, because it makes the system better maintainable in a lot of ways. 5. Forking things will only make SCM worse, as it adds a complexity. So I very much hope we can come to an agreement how the process works, without resorting to drastic measures. Regards, Diego
Hi Diego, Le 30 juin 2014 à 09:23, Diego Lont a écrit :
I am very happy that versioner is being built. We need more and better software configuration tools, and versioner will make our life easier. That being said, I am getting a bit nervous about how versioner is released. It should aim to improve the way people can work, and not to create a single workflow for all people working on pharo projects. I hope my fears are unfounded.
You're right, Versionner enforces a single workflow. We tried to do a simple tool that can fit most people and very easy to use. Versionner is far from perfect but allows to easily manage project dependencies and release for a "classical" workflow. There are also some known limitations (e.g., no support of platform-specific sections). Versionner does not aim that ALL people use a single workfow but tries to provide guidelines. If one needs another workflow, it is possible by using Metacello directly. If needed, we can also add features to Versionner. It may be done from feedbacks. To summarize, it is not "One Ring to rule them all" but rather one tool to fit more needs and make configuration management easy.
Software configuration management (SCM) is in my opinion an undervalued topic within the software world. One cannot find very many good books on them, and I am afraid this lack of knowledge shows in the way people go around with their SCM. So I would like to write down some observations of the SCM processes in Pharo that I believe we should keep in mind.
The main target of SCM is to keep the impact of changes as minimal as possible. Good SCM allows all people working on a certain project, to make their changes, without causing trouble for other project members and people using this project. Open source projects are (in SCM terms) rather complex, as they involve a lot of people, including some people we do not know.
1. One of the best things of Metacello is that it does not require you to follow a certain workflow. It is very flexible and supports a large range of possible workflows.
True, it's really valuable. In the same time, it's difficult for people with few notions on SCM to know what to do.
Within the Pharo community we have at least 3 different workflows: A. For Pharo core, we have a pharo inbox, where suggested fixes for reported bugs are put. These fixes are reviewed and, if accepted, integrated into Pharo. This ensures that the changes in Pharo hold (most of the time) to quite some quality standards. It is the same workflow that GitHub favors: you fork the code, do your changes and provide a pull request that will be accepted or not. It works quite well. B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all suggested fixes are put in the main repository on smalltalkhub. The fixes are, after review and tests, released by putting them in a âreleasedâ version of the configuration. Since not all fixes should be in all versions, and sometimes only concern a certain platform, the process for Pharo core would not work: releasing fixes are more complex because of the cross-platform issues. I don't see your point here. What is really different except that you don't have an inbox (some kind of dedicated branch)? Instead, you publish in the same repository, creating new branches. C. For the âmooseâ projects all suggested fixes are put in the main repository and are accepted by default. Since most projects evolve very fast, quality assurance is done afterwards, by submitting bug fixes. Here, there is no review, ok.
I can say a lot about why I believe these workflows are appropriate for their different projects, but the point I am trying to make here, is that these workflows have come to existence because of the different demands on them. And to stress my point about the flexibility (thank you Dale): When there was a problem with the workflow in the cross platform projects, I could find a solution (using releases) that Metacello supported.
I can only agree. There is not one language for everything, one process for everything, one tool for everything => they are languages, processes, tools. Some of them more adapted to a given situation.
2. Unfortunately Metacello does not have a good UI. And for large configurations like Seaside, it is a lot of work to keep them maintained. I dream of a tool that acts as a UI for Metacello: this tool is flexible as Metacello, while making the simple things real easy (hitting a button and done). Does versioner aim to do this?
Clearly not. Versionner was not thought to open all doors Metacello provides but rather provide a tool to manage easily configurations with a classical workflow. That said, Versionner is not against any improvement / feature. If there are need uncovered, we should think on how to cover them.
3. I always get very nervous when someone says: They do it like this, and that works very well for them. So we should do that too. I do not believe that is always a good idea. The conclusion should be a question: Can we use that too?
+1
And now I cannot find the mail about the java SCM practice, but I do not think we can apply it to all our processes ...
4. Modularity is very important.
+100
Also for performing good SCM. And yes, we lack sufficient modularity in Moose. It should be better. I.e. because of the lack of modularity, it is sometimes hard to distinguish between where the change needs to be put in. So there is no clear group to point out who should accept the changes. So this is also an SCM problem.
Cleaning up the configuration will help here very much. Thank you Steff for the good work here. And maybe you also have a point that the people working on Moose should have more focus on keeping things modular, because it makes the system better maintainable in a lot of ways.
5. Forking things will only make SCM worse, as it adds a complexity. So I very much hope we can come to an agreement how the process works, without resorting to drastic measures.
To summarize, Versionner (may) lacks functionalities and there is place to enhancement. The first thing is to identify them clearly (I don't want to bloat Versionner with unused functionalities) and then to implement it. If some of you wish to join efforts, they are welcomed :) Regards, Christophe.
Hi Christophe,
Hi Diego,
Le 30 juin 2014 à 09:23, Diego Lont a écrit :
I am very happy that versioner is being built. We need more and better software configuration tools, and versioner will make our life easier. That being said, I am getting a bit nervous about how versioner is released. It should aim to improve the way people can work, and not to create a single workflow for all people working on pharo projects. I hope my fears are unfounded.
You're right, Versionner enforces a single workflow. We tried to do a simple tool that can fit most people and very easy to use. Versionner is far from perfect but allows to easily manage project dependencies and release for a "classical" workflow. There are also some known limitations (e.g., no support of platform-specific sections). Versionner does not aim that ALL people use a single workfow but tries to provide guidelines. If one needs another workflow, it is possible by using Metacello directly. If needed, we can also add features to Versionner. It may be done from feedbacks. To summarize, it is not "One Ring to rule them all" but rather one tool to fit more needs and make configuration management easy.
That is exactly my point. But Stef currently tries to enforce this workflow of working to Moose. And since this workflow will not work well for Moose (because of the demands on the process), I object to this.
Within the Pharo community we have at least 3 different workflows: A. For Pharo core, we have a pharo inbox, where suggested fixes for reported bugs are put. These fixes are reviewed and, if accepted, integrated into Pharo. This ensures that the changes in Pharo hold (most of the time) to quite some quality standards. It is the same workflow that GitHub favors: you fork the code, do your changes and provide a pull request that will be accepted or not. It works quite well. B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all suggested fixes are put in the main repository on smalltalkhub. The fixes are, after review and tests, released by putting them in a âreleasedâ version of the configuration. Since not all fixes should be in all versions, and sometimes only concern a certain platform, the process for Pharo core would not work: releasing fixes are more complex because of the cross-platform issues. I don't see your point here. What is really different except that you don't have an inbox (some kind of dedicated branch)? Instead, you publish in the same repository, creating new branches. C. For the âmooseâ projects all suggested fixes are put in the main repository and are accepted by default. Since most projects evolve very fast, quality assurance is done afterwards, by submitting bug fixes. Here, there is no review, ok.
Your response shows that I have not made my point well. I wanted to point out that we need different workflows because of the different demands. A workflow is more than only a tool, but starts with people building something and ends with people using something. And you point out that versioner has one workflow implemented, and therefor will not be applicable to all projects. A. For pharo core all projects are included in pharo. The number of external influences (VM, plugins) is very limited. It works here to have no configuration at all, since there is only 1 release-branch: the current one. And since the file server creates a version of each build, we can always go back to a specific build. Quality is very important to pharo as a lot of users depend on a good working pharo. B. For the âcross-platformâ projects this workflow is not sufficient, because there are much more external influences. So here we use the configurations to manage the releases. And the complexity of the configuration of Seaside shows this is a very complex thing to do. This complexity is not from Metacello. Metacello allows us to do a decent job here for the supported platforms. For the unsupported platforms there is some additional manual work involved, with all risks to this. The complexity here stems from the Software Configuration challenges Seaside faces. A simple inbox wonât do the job for us. Quality here is also very important as a lot of users depend on a good working Seaside. Also there are more demands on reproducibility, as there are quite some systems depending on Seaside that should function 24/7. C. For the âmooseâ projects, we have a few advantages above the cross-platform projects, and a few disadvantages: + Moose only needs to work for pharo. So we can cut out a lot of platform specific junk. Some projects are suited for multiple platforms, but not for the amount of platform Seaside should work on. + There are far less projects depending on Moose. Most of them are integrated, so tests will automatically be fired. And the projects that do depend on Moose usually do not have run 24/7. - Moose depends on more projects then Seaside. To create an exact version is therefor much harder. - A large part of the Moose bugs comes from the change rate of the projects Moose depends on. Pharo has a very drastic strategy to improve its architecture. This causes quite some maintenance work. So that there is no review is not the point. The point is that because of these different demands we need to accept a lower quality standard for moose. This lower quality standard is maybe undesirable, but is not easily improved. Having no review is a consequence of this, not the cause. If we want to improve this quality standard, better SCM tools will not suffice. On the other hand, this lower quality standard has so far worked for Moose, as there are less people who depend on Moose. Regards, Diego
Le 30 juin 2014 à 15:30, Diego Lont a écrit :
Hi Christophe,
Hi Diego,
Le 30 juin 2014 à 09:23, Diego Lont a écrit :
I am very happy that versioner is being built. We need more and better software configuration tools, and versioner will make our life easier. That being said, I am getting a bit nervous about how versioner is released. It should aim to improve the way people can work, and not to create a single workflow for all people working on pharo projects. I hope my fears are unfounded.
You're right, Versionner enforces a single workflow. We tried to do a simple tool that can fit most people and very easy to use. Versionner is far from perfect but allows to easily manage project dependencies and release for a "classical" workflow. There are also some known limitations (e.g., no support of platform-specific sections). Versionner does not aim that ALL people use a single workfow but tries to provide guidelines. If one needs another workflow, it is possible by using Metacello directly. If needed, we can also add features to Versionner. It may be done from feedbacks. To summarize, it is not "One Ring to rule them all" but rather one tool to fit more needs and make configuration management easy.
That is exactly my point. But Stef currently tries to enforce this workflow of working to Moose. And since this workflow will not work well for Moose (because of the demands on the process), I object to this.
I don't think Stef wants to impose the workflow. Stef wants to improve code modularity in Moose and avoid broken / bad configs. Moose is a big project and one solution to ease the maintenance is to use a tool (Versionner for example). Now it is clear that Versionner does not have all features to support the Moose process and I will take some time to see how to implement a smart deep release. With that, it should be enough to support Moose workflow.
Within the Pharo community we have at least 3 different workflows: A. For Pharo core, we have a pharo inbox, where suggested fixes for reported bugs are put. These fixes are reviewed and, if accepted, integrated into Pharo. This ensures that the changes in Pharo hold (most of the time) to quite some quality standards. It is the same workflow that GitHub favors: you fork the code, do your changes and provide a pull request that will be accepted or not. It works quite well. B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all suggested fixes are put in the main repository on smalltalkhub. The fixes are, after review and tests, released by putting them in a âreleasedâ version of the configuration. Since not all fixes should be in all versions, and sometimes only concern a certain platform, the process for Pharo core would not work: releasing fixes are more complex because of the cross-platform issues. I don't see your point here. What is really different except that you don't have an inbox (some kind of dedicated branch)? Instead, you publish in the same repository, creating new branches. C. For the âmooseâ projects all suggested fixes are put in the main repository and are accepted by default. Since most projects evolve very fast, quality assurance is done afterwards, by submitting bug fixes. Here, there is no review, ok.
Your response shows that I have not made my point well. I wanted to point out that we need different workflows because of the different demands. A workflow is more than only a tool, but starts with people building something and ends with people using something. And you point out that versioner has one workflow implemented, and therefor will not be applicable to all projects.
A. For pharo core all projects are included in pharo. The number of external influences (VM, plugins) is very limited. It works here to have no configuration at all, since there is only 1 release-branch: the current one. And since the file server creates a version of each build, we can always go back to a specific build. Quality is very important to pharo as a lot of users depend on a good working pharo.
B. For the âcross-platformâ projects this workflow is not sufficient, because there are much more external influences. So here we use the configurations to manage the releases. And the complexity of the configuration of Seaside shows this is a very complex thing to do. This complexity is not from Metacello. Metacello allows us to do a decent job here for the supported platforms. For the unsupported platforms there is some additional manual work involved, with all risks to this. The complexity here stems from the Software Configuration challenges Seaside faces. A simple inbox wonât do the job for us. Quality here is also very important as a lot of users depend on a good working Seaside. Also there are more demands on reproducibility, as there are quite some systems depending on Seaside that should function 24/7.
C. For the âmooseâ projects, we have a few advantages above the cross-platform projects, and a few disadvantages: + Moose only needs to work for pharo. So we can cut out a lot of platform specific junk. Some projects are suited for multiple platforms, but not for the amount of platform Seaside should work on. + There are far less projects depending on Moose. Most of them are integrated, so tests will automatically be fired. And the projects that do depend on Moose usually do not have run 24/7. - Moose depends on more projects then Seaside. To create an exact version is therefor much harder. - A large part of the Moose bugs comes from the change rate of the projects Moose depends on. Pharo has a very drastic strategy to improve its architecture. This causes quite some maintenance work. So that there is no review is not the point. The point is that because of these different demands we need to accept a lower quality standard for moose. This lower quality standard is maybe undesirable, but is not easily improved. Having no review is a consequence of this, not the cause. If we want to improve this quality standard, better SCM tools will not suffice. On the other hand, this lower quality standard has so far worked for Moose, as there are less people who depend on Moose.
Ok, I got it. Just to let you know, we want to move the process you describe for Pharo to build it from a seed + configurations. The Pharo process will change too and we will see if our tools are enough powerful to achieve that.
That is exactly my point. But Stef currently tries to enforce this workflow of working to Moose. And since this workflow will not work well for Moose (because of the demands on the process), I object to this.
I don't think Stef wants to impose the workflow. Stef wants to improve code modularity in Moose and avoid broken / bad configs. Moose is a big project and one solution to ease the maintenance is to use a tool (Versionner for example). Now it is clear that Versionner does not have all features to support the Moose process and I will take some time to see how to implement a smart deep release. With that, it should be enough to support Moose workflow.
We want to have a working workflow :) And the point is to make one becoming true. After we can discuss. Stef
Your response shows that I have not made my point well. I wanted to point out that we need different workflows because of the different demands. A workflow is more than only a tool, but starts with people building something and ends with people using something. And you point out that versioner has one workflow implemented, and therefor will not be applicable to all projects.
A. For pharo core all projects are included in pharo. The number of external influences (VM, plugins) is very limited. It works here to have no configuration at all, since there is only 1 release-branch: the current one. And since the file server creates a version of each build, we can always go back to a specific build. Quality is very important to pharo as a lot of users depend on a good working pharo.
But it should not be like that! Pharo should be modular too. And we need to have configurations. Why we cannot have better tools based on Roassal because we do not want to have a monolithic solution. So we should have a modular image and tools and process to get there.
B. For the âcross-platformâ projects this workflow is not sufficient, because there are much more external influences. So here we use the configurations to manage the releases. And the complexity of the configuration of Seaside shows this is a very complex thing to do. This complexity is not from Metacello. Metacello allows us to do a decent job here for the supported platforms. For the unsupported platforms there is some additional manual work involved, with all risks to this. The complexity here stems from the Software Configuration challenges Seaside faces. A simple inbox wonât do the job for us. Quality here is also very important as a lot of users depend on a good working Seaside. Also there are more demands on reproducibility, as there are quite some systems depending on Seaside that should function 24/7.
C. For the âmooseâ projects, we have a few advantages above the cross-platform projects, and a few disadvantages: + Moose only needs to work for pharo. So we can cut out a lot of platform specific junk. Some projects are suited for multiple platforms, but not for the amount of platform Seaside should work on. + There are far less projects depending on Moose. Most of them are integrated, so tests will automatically be fired. And the projects that do depend on Moose usually do not have run 24/7. - Moose depends on more projects then Seaside. To create an exact version is therefor much harder. - A large part of the Moose bugs comes from the change rate of the projects Moose depends on. Pharo has a very drastic strategy to improve its architecture. This causes quite some maintenance work. So that there is no review is not the point. The point is that because of these different demands we need to accept a lower quality standard for moose. This lower quality standard is maybe undesirable, but is not easily improved. Having no review is a consequence of this, not the cause. If we want to improve this quality standard, better SCM tools will not suffice. On the other hand, this lower quality standard has so far worked for Moose, as there are less people who depend on Moose.
Regards, Diego
I just don't see it the way you do. For me Versioner is a tool to make life a bit easier with Configurations, its not "the one ring to rule them all" , far from it. What stops anyone from implementing his own configuration tool ? On Mon, Jun 30, 2014 at 10:23 AM, Diego Lont <diego.lont@delware.nl> wrote:
I am afraid this mail is going to be too long, but I feel it needs to be written. I will start with the conclusion, so you can skip the mail if you feel it does not interest you. Please keep the discussion on the Moose list, as there the discussion was started.
I am very happy that versioner is being built. We need more and better software configuration tools, and versioner will make our life easier. That being said, I am getting a bit nervous about how versioner is released. It should aim to improve the way people can work, and *not* to create a single workflow for all people working on pharo projects. I hope my fears are unfounded.
Software configuration management (SCM) is in my opinion an undervalued topic within the software world. One cannot find very many good books on them, and I am afraid this lack of knowledge shows in the way people go around with their SCM. So I would like to write down some observations of the SCM processes in Pharo that I believe we should keep in mind.
The main target of SCM is to keep the impact of changes as minimal as possible. Good SCM allows all people working on a certain project, to make their changes, without causing trouble for other project members and people using this project. Open source projects are (in SCM terms) rather complex, as they involve a lot of people, including some people we do not know.
1. One of the best things of Metacello is that it does not require you to follow a certain workflow. It is very flexible and supports a large range of possible workflows. Within the Pharo community we have at least 3 different workflows: A. For Pharo core, we have a pharo inbox, where suggested fixes for reported bugs are put. These fixes are reviewed and, if accepted, integrated into Pharo. This ensures that the changes in Pharo hold (most of the time) to quite some quality standards. B. For the âcross-platformâ projects (Grease, Seaside, Magritte, etc.) all suggested fixes are put in the main repository on smalltalkhub. The fixes are, after review and tests, released by putting them in a âreleasedâ version of the configuration. Since not all fixes should be in all versions, and sometimes only concern a certain platform, the process for Pharo core would not work: releasing fixes are more complex because of the cross-platform issues. C. For the âmooseâ projects all suggested fixes are put in the main repository and are accepted by default. Since most projects evolve very fast, quality assurance is done afterwards, by submitting bug fixes.
I can say a lot about why I believe these workflows are appropriate for their different projects, but the point I am trying to make here, is that these workflows have come to existence because of the different demands on them. And to stress my point about the flexibility (thank you Dale): When there was a problem with the workflow in the cross platform projects, I could find a solution (using releases) that Metacello supported.
2. Unfortunately Metacello does not have a good UI. And for large configurations like Seaside, it is a lot of work to keep them maintained. I dream of a tool that acts as a UI for Metacello: this tool is flexible as Metacello, while making the simple things real easy (hitting a button and done). Does versioner aim to do this?
3. I always get very nervous when someone says: They do it like this, and that works very well for them. So we should do that too. I do not believe that is always a good idea. The conclusion should be a question: Can we use that too? And now I cannot find the mail about the java SCM practice, but I do not think we can apply it to all our processes ...
4. Modularity is very important. Also for performing good SCM. And yes, we lack sufficient modularity in Moose. It should be better. I.e. because of the lack of modularity, it is sometimes hard to distinguish between where the change needs to be put in. So there is no clear group to point out who should accept the changes. So this is also an SCM problem.
Cleaning up the configuration will help here very much. Thank you Steff for the good work here. And maybe you also have a point that the people working on Moose should have more focus on keeping things modular, because it makes the system better maintainable in a lot of ways.
5. Forking things will only make SCM worse, as it adds a complexity. So I very much hope we can come to an agreement how the process works, without resorting to drastic measures.
Regards, Diego
participants (4)
-
Christophe Demarey -
Diego Lont -
kilon alios -
stepharo