On 15 Jan 2016, at 21:01, stepharo <stepharo@free.fr> wrote:

Le 15/1/16 20:45, David Allouche a écrit :

...

it does not show the package that contains my fix (Spec-Tools) This is strange. And MC shows the package as dirty? Then this is a bug.

I do not know how MC shows a package as dirty. And anyway, I ended up fiddling my way into uploading the version to smalltalkhub (I think I just clicked Save in Monticello Browser). So it no longer shows as dirty. But apparently that did not provide the fogzbugz issue metadata… So it might be not dirty anymore. But then I have no idea what to do now to upload it correctly.

...

So I think it would be good that someone updates the documentation. Or fix the slice maker. Or both. Can you reproduce the bug?

I cannot not reproduce it. I have no idea at all how people actually manage to use Slice Maker in a recent image. Have you tried with a recent

BTW it would be nice that you print and sign the pharo license, then you send us a scanned or surface mail version and we will archive it.

Surface mail, is that a kind of new social network thing? I'm afraid I got a bit behind the times those past few years.

We ask that to every contributors.

No problem, I'll do that. Copyright paperwork is important.

2016-01-15 23:23 GMT+01:00 Nicolai Hess <nicolaihess@gmail.com>:

2016-01-15 23:16 GMT+01:00 David Allouche <david@allouche.net>:

...

On 15 Jan 2016, at 21:01, stepharo <stepharo@free.fr> wrote:

Le 15/1/16 20:45, David Allouche a écrit :

1. it does not show the package that contains my fix (Spec-Tools)

This is strange. And MC shows the package as dirty? Then this is a bug.

I do not know how MC shows a package as dirty.

This is how it should look in slice maker (left side) and monticello browser (right side). The " * " marks dirty packages.

Maybe, we should include some screenshots on the "Contribute to Pharo" page.

On 15 Jan 2016, at 23:23, Nicolai Hess <nicolaihess@gmail.com> wrote: 2016-01-15 23:16 GMT+01:00 David Allouche <david@allouche.net <mailto:david@allouche.net>>:

...

On 15 Jan 2016, at 21:01, stepharo <stepharo@free.fr <mailto:stepharo@free.fr>> wrote: Le 15/1/16 20:45, David Allouche a écrit :

...

it does not show the package that contains my fix (Spec-Tools) This is strange. And MC shows the package as dirty? Then this is a bug.

I do not know how MC shows a package as dirty.

This is how it should look in slice maker (left side) and monticello browser (right side). The " * " marks dirty packages.

Okay, got that. So yes, it's broken, but maybe it's something else entirely, I get new reproducible errors every few minutes: errors when activating pretty printing in the diff viewer in Monticello, unexpected contextual menu showing up when I do Cmd-T in the debugger, infinite recursions when trying to debug through the MonticelloRepositoryBrowser... All things that are so obvious that I have trouble believing they are actual bugs. Maybe there was something wrong in how my image was generated. I will try building another image. I also wonder… so this "list of dirty packages" is actually a list of every package where some of them are marked dirty? I mean, why is it not just the list of dirty package, as the name implies?

2016-01-15 23:48 GMT+01:00 David Allouche <david@allouche.net>:

...

On 15 Jan 2016, at 23:33, David Allouche <david@allouche.net> wrote:

So yes, it's broken, but maybe it's something else entirely, I get new reproducible errors every few minutes: errors when activating pretty printing in the diff viewer in Monticello, unexpected contextual menu showing up when I do Cmd-T in the debugger, infinite recursions when trying to debug through the MonticelloRepositoryBrowser... All things that are so obvious that I have trouble believing they are actual bugs. Maybe there was something wrong in how my image was generated. I will try building another image.

Built a new latest image. Now I no longer have any syntax highlighting in the debuggers or class browser…

Now, people should use the latest image for dev. But apparently the latest image is more often broken than not. What is the trick?

Ah, that's not true. I work this way every day. And yes it can happen that the latest image is broken but, not that often.

Sorry for the unpleasant messages, but I am getting incredibly frustrated.

I am now (at this moment) working with the image version 50528.

On 15 Jan 2016, at 23:58, David Allouche <david@allouche.net> wrote:

...

I am now (at this moment) working with the image version 50528.

Yes, that's the one where syntax highlighting appears broken.

Nevermind, it works… I must have imagined it did not, or a transient bout of colour-blindness…

2016-01-15 23:33 GMT+01:00 David Allouche <david@allouche.net>:

On 15 Jan 2016, at 23:23, Nicolai Hess <nicolaihess@gmail.com> wrote: 2016-01-15 23:16 GMT+01:00 David Allouche <david@allouche.net>:

...

On 15 Jan 2016, at 21:01, stepharo <stepharo@free.fr> wrote: Le 15/1/16 20:45, David Allouche a écrit :

1. it does not show the package that contains my fix (Spec-Tools)

This is strange. And MC shows the package as dirty? Then this is a bug.

I do not know how MC shows a package as dirty.

This is how it should look in slice maker (left side) and monticello browser (right side). The " * " marks dirty packages.

Okay, got that.

So yes, it's broken, but maybe it's something else entirely, I get new reproducible errors every few minutes: errors when activating pretty printing in the diff viewer in Monticello,

this is a known error from the "recent" change for rubric text component

unexpected contextual menu showing up when I do Cmd-T in the debugger,

known ( from the recent changes for debugger :) )

infinite recursions when trying to debug through the MonticelloRepositoryBrowser...

I don't know this error. But I don't think "this" MonticelloRepositoryBrowser is actually used.

All things that are so obvious that I have trouble believing they are actual bugs. Maybe there was something wrong in how my image was generated. I will try building another image.

I also wonder… so this "list of dirty packages" is actually a list of every package where some of them are marked dirty? I mean, why is it not just the list of dirty package, as the name implies?

Or, try a stable release. Whenever I try developing on the latest version, I end up getting disrupted by bugs in the core tools 70-80% of the time. For released versions, in 1.4 it was maybe 5%, in 2.0 40%, in 3.0 20%, in 4.0 10% So at least it's been trending downwards, though I expect a totally new debugger, image format, RBParser unification and new FFI in 5.0 will keep causing problems for a quite a while. Cheers, Henry On Fri, Jan 15, 2016 at 11:57 PM, David Allouche <david@allouche.net> wrote:

On 15 Jan 2016, at 23:49, Nicolai Hess <nicolaihess@gmail.com> wrote:

2016-01-15 23:33 GMT+01:00 David Allouche <david@allouche.net>:

...

So yes, it's broken, but maybe it's something else entirely, I get new reproducible errors every few minutes: errors when activating pretty printing in the diff viewer in Monticello,

this is a known error from the "recent" change for rubric text component

...

unexpected contextual menu showing up when I do Cmd-T in the debugger,

known ( from the recent changes for debugger :) )

...

infinite recursions when trying to debug through the MonticelloRepositoryBrowser...

I don't know this error. But I don't think "this" MonticelloRepositoryBrowser is actually used.

I think that was actually a bug in the debugger, or inspector, not in MonticelloRepositoryBrowser.

And yes, this appear to be dead code. I found it before finding the actual browser, and thought it might be a good place to start exploring Monticello.

I tried the old debugger in my new latest image (in the hope that syntax highlighting would not be broken there), but I realised it advertised no keyboard shortcut for stepping through the code…

Starting to wonder if I should go back to spending my evenings playing Elite Dangerous, and come back in a few weeks when stuff is hopefully less buggy everywhere :-(

On Sat, Jan 16, 2016 at 6:57 AM, David Allouche <david@allouche.net> wrote:

On 15 Jan 2016, at 23:49, Nicolai Hess <nicolaihess@gmail.com> wrote:

2016-01-15 23:33 GMT+01:00 David Allouche <david@allouche.net>:

...

So yes, it's broken, but maybe it's something else entirely, I get new reproducible errors every few minutes: errors when activating pretty printing in the diff viewer in Monticello,

this is a known error from the "recent" change for rubric text component

...

unexpected contextual menu showing up when I do Cmd-T in the debugger,

known ( from the recent changes for debugger :) )

...

infinite recursions when trying to debug through the MonticelloRepositoryBrowser...

I don't know this error. But I don't think "this" MonticelloRepositoryBrowser is actually used.

I think that was actually a bug in the debugger, or inspector, not in MonticelloRepositoryBrowser.

And yes, this appear to be dead code. I found it before finding the actual browser, and thought it might be a good place to start exploring Monticello.

I tried the old debugger in my new latest image (in the hope that syntax highlighting would not be broken there), but I realised it advertised no keyboard shortcut for stepping through the code…

Starting to wonder if I should go back to spending my evenings playing Elite Dangerous, and come back in a few weeks when stuff is hopefully less buggy everywhere :-(

Generally I'm impressed at Pharo's stability given depth of the surgery that is sometimes done on it - but after all it is bleeding edge and sometimes things get in a bind. Since all the tools are built and run inside the Image, sometimes its a bit like changing the tyres on a moving car - but there is the advantage that we are *always* eating our own dogfood. So if occasionally if everything seems against you, after trying a fresh Image as you already did, slide back to the last build that worked for you (I don't think the issues you experience are Spur related, but 50496 might be a good fallback as Pre-Spur and Pre-GTDebugger). Then create a fresh Image from the latest build to test your Slice and in that time maybe the other issues have gone. cheers -ben

On 16 Jan 2016, at 02:42, Ben Coman <btc@openinworld.com> wrote:

So if occasionally if everything seems against you, after trying a fresh Image as you already did, slide back to the last build that worked for you (I don't think the issues you experience are Spur related, but 50496 might be a good fallback as Pre-Spur and Pre-GTDebugger). Then create a fresh Image from the latest build to test your Slice and in that time maybe the other issues have gone.

Ok. So my earlier question (in Slack, I think) about how "unstable" is unstable in Pharo is now answered. It's really unstable, not "Debian-like unstable". It would also be nice to add some public info about that. I'll create an issue. At the moment, I am stuck with GTDebugger, because it seems the old debugger does not have any keyboard shortcut form stepping through code. And that's something I would really really miss. So I'll suffer the pain of all the other glitches, and see how that goes.

Hi david

...

On 16 Jan 2016, at 02:42, Ben Coman <btc@openinworld.com> wrote:

So if occasionally if everything seems against you, after trying a fresh Image as you already did, slide back to the last build that worked for you (I don't think the issues you experience are Spur related, but 50496 might be a good fallback as Pre-Spur and Pre-GTDebugger). Then create a fresh Image from the latest build to test your Slice and in that time maybe the other issues have gone. Ok. So my earlier question (in Slack, I think) about how "unstable" is unstable in Pharo is now answered. It's really unstable, not "Debian-like unstable".

No this is not really the case. In general we try hard to get alpha stable after you get some hiccups. This is not every day that we change FFI, GC, object internal representations and the text editors, syntax highli And our problems is that sometimes often we just jump over glitches (this is why I'm getting mad because when I do videos or other I see them in my face). Now I would love to have task forces that check glitches.

It would also be nice to add some public info about that. I'll create an issue.

At the moment, I am stuck with GTDebugger, because it seems the old debugger does not have any keyboard shortcut form stepping through code. And that's something I would really really miss. So I'll suffer the pain of all the other glitches, and see how that goes.

...

No this is not really the case. In general we try hard to get alpha stable after you get some hiccups. This is not every day that we change FFI, GC, object internal representations and the text editors, syntax highli And our problems is that sometimes often we just jump over glitches (this is why I'm getting mad because when I do videos or other I see them in my face). Now I would love to have task forces that check glitches. I get the picture now. I just checked in when the lobby, the roof and the plumbing of the hotel were undergoing renovations: it's not how it is usually.

I could question whether it's a good idea to cram so much in a single release cycle. But release management is a difficult topic, and often there is no good solution, just one that has less downsides than the others.

We do not control all the agendas. We were hesitating to get Pharo 50 without spur and for some people it would have been sad. Then it would have meant 2 months in beta and slow mode. And Esteban worked a lot on the new FFI because it was needed: you do not change an object header internal representation without impact and since the previous FFI was generating assembly on the fly and few people knew its implementation and Igor got a depression we got stuck. Then this is important to get the new FFI out because Ronie is coming to visit us and he should work with Clement and esteban on a faster version. This is as simple as that. It means that all the libraries using FFI would not have been available with Spur = no use for certain people like roassal, moose, synectique. Stef

Le 16/1/16 09:59, David Allouche a écrit :

...

On 16 Jan 2016, at 02:15, Ben Coman <btc@openinworld.com <mailto:btc@openinworld.com>> wrote: On Sat, Jan 16, 2016 at 3:45 AM, David Allouche<david@allouche.net <mailto:david@allouche.net>>wrote:

1. it shows a bunch of packages I have not touched 2. it does not show the package that contains my fix (Spec-Tools)

I am curious on item 2. Was this just because the dirty package was sorted to the top of the list (per Nicolai's snapshots)?

I think that was indeed the source of confusion.

But, really, this "list of dirty packages" should really only list dirty packages. What is the use case for selecting a non-dirty package?

because you never know Sometimes you save a package then you do a slice and you should be able to decide.

...

I looked in the world menu, and I found this menu item called "Komitter". That sounded like a good candidate.

I see how the term "Commit" is misleading here without context. Probably "Press the <Save> button to commit your slice to the inbox repository" would probably be better.

Yes, that would help, too.

...

https://pharo.fogbugz.com/default.asp?17398

Cool!

On Sat, Jan 16, 2016 at 7:25 PM, David Allouche <david@allouche.net> wrote:

...

On 16 Jan 2016, at 12:14, stepharo <stepharo@free.fr> wrote:

Le 16/1/16 09:59, David Allouche a écrit :

...

But, really, this "list of dirty packages" should really only list dirty packages. What is the use case for selecting a non-dirty package?

because you never know Sometimes you save a package then you do a slice and you should be able to decide.

That does not make sense to me. I guess it will, when I understand Monticello better.

Its a very corner case. I've had to do it on occasion but I can't exactly remember the circumstance. Now what might be useful is a checkbox for displaying non-dirty packages - so we only display the short list in the usual case. (??) cheers -ben

Ben Coman wrote

checkbox for displaying non-dirty packages - so we only display the short list in the usual case. (??)

Ah! That would be very helpful. Particularly annoying is when you commit one of your project's dirty packages, the list auto-scrolls to the package's new location among the non-dirty packages, and you have to manually scroll back to the top of the package list to find your next dirty package. ----- Cheers, Sean -- View this message in context: http://forum.world.st/Contributing-to-Pharo-tp4871604p4873422.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Hi Dale, Le 27 janv. 2016 à 04:11, Dale Henrichs a écrit :

I've been wondering no-one in the Pharo community has started building a "Project Browser" to replace the "Package Browser" .... I've had a "project browser" in tODE for several years now and you work with a manageble list of Metacello projects like the following:

Of course, we have that in mind. We would love to have it but lack of time / manpower ... We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;) If someone comes with a better browser, we would love to set it as default. Cheers, Christophe

Dale, That looks like something very interesting and worth considering. How easy is it to load tODE in a current (4.x 5.x) Pharo image ? I assume there is a Metacello configuration ;-) Is it possible to experiment with it in Pharo, independent from Gemstone ? I did not see it in the Catalog Browser (our current point & click installer). Making tODE easily loadable would, IMHO, be the first step to a broader adaption of your work (in one form or another). The next step would be a getting started tutorial. Sven

On 27 Jan 2016, at 15:13, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:

Haha, I certainly understand the lack of time/manpower ... Just in case it wasn't clear, when I said "project browser" I wasn't talking about replacing the code browser (Nautilus), but replacing/augmenting the Monticello Browser - which should turn out to be a much simpler undertaking --- "Project Browser" that manages the projects within the image - load, save, changes, etc.:

<Screen Shot 2016-01-27 at 6.11.33 AM.png>

Dale

On 1/26/16 11:29 PM, Christophe Demarey wrote:

...

Hi Dale,

Le 27 janv. 2016 à 04:11, Dale Henrichs a écrit :

...

I've been wondering no-one in the Pharo community has started building a "Project Browser" to replace the "Package Browser" .... I've had a "project browser" in tODE for several years now and you work with a manageble list of Metacello projects like the following:

Of course, we have that in mind. We would love to have it but lack of time / manpower ... We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;) If someone comes with a better browser, we would love to set it as default.

Cheers, Christophe

We have the catalog. So how different it is. And yes we would to change packages in nautilus. Le 27/1/16 15:13, Dale Henrichs a écrit :

Haha, I certainly understand the lack of time/manpower ... Just in case it wasn't clear, when I said "project browser" I wasn't talking about replacing the code browser (Nautilus), but replacing/augmenting the Monticello Browser - which should turn out to be a much simpler undertaking --- "Project Browser" that manages the projects within the image - load, save, changes, etc.:

project list screenshot

Dale

On 1/26/16 11:29 PM, Christophe Demarey wrote:

...

Hi Dale,

Le 27 janv. 2016 à 04:11, Dale Henrichs a écrit :

...

I've been wondering no-one in the Pharo community has started building a "Project Browser" to replace the "Package Browser" .... I've had a "project browser" in tODE for several years now and you work with a manageble list of Metacello projects like the following: Of course, we have that in mind. We would love to have it but lack of time / manpower ... We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;) If someone comes with a better browser, we would love to set it as default.

Cheers, Christophe

I think that this is important that we do not think only in terms of git. People will jump on the next cool and better because nicer versioning system. I think about fossil for example. I'm waiting for that. Because each time I pair program with smart git users I see how this is "simple". So I would really be against having git menu in Pharo. We should have an abstraction over the generic commands that a decentralized versioning system should offer. Stef Le 27/1/16 18:23, Dale Henrichs a écrit :

On 1/27/16 8:30 AM, stepharo wrote:

...

We have the catalog. So how different it is. AFAIK the catalog only manages loads of projects ... to make it a "Project Browser" you would add menu items for committing/saving projects (i.e., save all dirty packages and for git repos, do a git commit after updating packages); viewing the diffs for all packages in a project; and of course the load functionality ....

As I said somewhere else, I don't think that it is a major job to implement a "Project Browser", but it is something that really needs to be done for supporting git-based projects. There is a natural 1-to-1 mapping between a Metacello project and it's git repository and as you can see in the Git menu below, there are a number of additional git-specific functions that need to be supported ...

Dale

...

And yes we would to change packages in nautilus.

Le 27/1/16 15:13, Dale Henrichs a écrit :

...

Haha, I certainly understand the lack of time/manpower ... Just in case it wasn't clear, when I said "project browser" I wasn't talking about replacing the code browser (Nautilus), but replacing/augmenting the Monticello Browser - which should turn out to be a much simpler undertaking --- "Project Browser" that manages the projects within the image - load, save, changes, etc.:

project list screenshot

Dale

On 1/26/16 11:29 PM, Christophe Demarey wrote:

...

Hi Dale,

Le 27 janv. 2016 à 04:11, Dale Henrichs a écrit :

...

I've been wondering no-one in the Pharo community has started building a "Project Browser" to replace the "Package Browser" .... I've had a "project browser" in tODE for several years now and you work with a manageble list of Metacello projects like the following: Of course, we have that in mind. We would love to have it but lack of time / manpower ... We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;) If someone comes with a better browser, we would love to set it as default.

Cheers, Christophe

2016-01-27 17:30 GMT-03:00 Dale Henrichs <dale.henrichs@gemtalksystems.com>:

Totally agree with having an abstraction layer or common set of functions ...

As I've said before, I talk about git mainly because of github and because there needs to be a "first" and I chose git ...

Other than details, I think that most if not all of the information I plan to share is "disk-based SCM neutral" and applies equally to each SCM ....

AFAIU FileTree has that purpose, although I don't know if it is used with something other than Git as a SCM. Esteban A. Maringolo

So we are in sync then! Le 27/1/16 21:30, Dale Henrichs a écrit :

Totally agree with having an abstraction layer or common set of functions ...

As I've said before, I talk about git mainly because of github and because there needs to be a "first" and I chose git ...

Other than details, I think that most if not all of the information I plan to share is "disk-based SCM neutral" and applies equally to each SCM ....

Dale

On 01/27/2016 11:47 AM, stepharo wrote:

...

I think that this is important that we do not think only in terms of git. People will jump on the next cool and better because nicer versioning system. I think about fossil for example. I'm waiting for that. Because each time I pair program with smart git users I see how this is "simple". So I would really be against having git menu in Pharo. We should have an abstraction over the generic commands that a decentralized versioning system should offer.

Stef

Le 27/1/16 18:23, Dale Henrichs a écrit :

...

On 1/27/16 8:30 AM, stepharo wrote:

...

We have the catalog. So how different it is. AFAIK the catalog only manages loads of projects ... to make it a "Project Browser" you would add menu items for committing/saving projects (i.e., save all dirty packages and for git repos, do a git commit after updating packages); viewing the diffs for all packages in a project; and of course the load functionality ....

As I said somewhere else, I don't think that it is a major job to implement a "Project Browser", but it is something that really needs to be done for supporting git-based projects. There is a natural 1-to-1 mapping between a Metacello project and it's git repository and as you can see in the Git menu below, there are a number of additional git-specific functions that need to be supported ...

Dale

...

And yes we would to change packages in nautilus.

Le 27/1/16 15:13, Dale Henrichs a écrit :

...

Haha, I certainly understand the lack of time/manpower ... Just in case it wasn't clear, when I said "project browser" I wasn't talking about replacing the code browser (Nautilus), but replacing/augmenting the Monticello Browser - which should turn out to be a much simpler undertaking --- "Project Browser" that manages the projects within the image - load, save, changes, etc.:

project list screenshot

Dale

On 1/26/16 11:29 PM, Christophe Demarey wrote:

...

Hi Dale,

Le 27 janv. 2016 à 04:11, Dale Henrichs a écrit :

...

I've been wondering no-one in the Pharo community has started building a "Project Browser" to replace the "Package Browser" .... I've had a "project browser" in tODE for several years now and you work with a manageble list of Metacello projects like the following: Of course, we have that in mind. We would love to have it but lack of time / manpower ... We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;) If someone comes with a better browser, we would love to set it as default.

Cheers, Christophe

On 28 Jan 2016, at 09:23, Christophe Demarey <Christophe.Demarey@inria.fr> wrote:

I read too fast. But my comment is still true. We also want a new version browser. Monticello browser is too tied to ... Monticello. We would like a better approach for both git and monticello.

It is totally wrong to put Git and Monticello at the same level, they are related but different things. Monticello models our code and your changes to it. It also has a way of putting a package into a persistent representation that can be copied around. An MC server is nothing more than storage. Filetree is another way of doing that, using a different representation, but it still needs MC for the high level stuff. Git is versioned storage. And there is a conflict there, yes. But throwing away the MC code and changes model would leave you with file outs. You do not want to go there, really. Never, ever.

Esteban did a prototype for himself but he has no time to work on it, sadly

Le 27 janv. 2016 à 15:13, Dale Henrichs a écrit :

...

Haha, I certainly understand the lack of time/manpower ... Just in case it wasn't clear, when I said "project browser" I wasn't talking about replacing the code browser (Nautilus), but replacing/augmenting the Monticello Browser - which should turn out to be a much simpler undertaking --- "Project Browser" that manages the projects within the image - load, save, changes, etc.:

<Screen Shot 2016-01-27 at 6.11.33 AM.png>

Dale

On 1/26/16 11:29 PM, Christophe Demarey wrote:

...

Hi Dale,

Le 27 janv. 2016 à 04:11, Dale Henrichs a écrit :

...

I've been wondering no-one in the Pharo community has started building a "Project Browser" to replace the "Package Browser" .... I've had a "project browser" in tODE for several years now and you work with a manageble list of Metacello projects like the following: Of course, we have that in mind. We would love to have it but lack of time / manpower ... We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;) If someone comes with a better browser, we would love to set it as default.

Cheers, Christophe

On 28 Jan 2016, at 09:55, Christophe Demarey <Christophe.Demarey@inria.fr> wrote:

Le 28 janv. 2016 à 09:34, Sven Van Caekenberghe a écrit :

...

...

On 28 Jan 2016, at 09:23, Christophe Demarey <Christophe.Demarey@inria.fr> wrote:

I read too fast. But my comment is still true. We also want a new version browser. Monticello browser is too tied to ... Monticello. We would like a better approach for both git and monticello.

It is totally wrong to put Git and Monticello at the same level, they are related but different things.

Monticello models our code and your changes to it. It also has a way of putting a package into a persistent representation that can be copied around. An MC server is nothing more than storage.

Filetree is another way of doing that, using a different representation, but it still needs MC for the high level stuff. Git is versioned storage. And there is a conflict there, yes.

But throwing away the MC code and changes model would leave you with file outs. You do not want to go there, really. Never, ever.

I fully agree. The problem is that Monticello does too much things: code representation, versioning, loading of the code and call of #initialize methods, etc. We want to keep a code representation but the question that comes: is it the better code representation? We also have ring, ficus (hierarchical model on top of ring). For code representation, we would like to represent code (from packages to method source code) for multiple versions, in a hierarchical way (to be able to query easily), potentially coming from a remote environment. We also need to split the loading part (loading from the code representation) and the serialization / deserialization part.

At the end, we want an uniform way to represent code (not SCM related), a common API for versioning and different backends (git, Monticello). By the way, we can add the storage format: there are already different versions of the filetree format (with metadata, without, one file per method, one file per class). At some point, we need to handle this point of variability.

Excellent summary. Lots of work ;-)

Hi Dale, I am trying to understand this a little better. If a package containing metadata would be changed using a dialect which cannot interpret the metadata, wouldn’t or at least couldn’t it be broken or lost afterwards for a dialect which tries to interpret the metadata? At least if my assumption is correct that the metadata is related to the code in some way? Bernhard

Am 28.01.2016 um 21:28 schrieb Dale Henrichs <dale.henrichs@gemtalksystems.com>: On 01/28/2016 12:55 AM, Christophe Demarey wrote:

...

Le 28 janv. 2016 à 09:34, Sven Van Caekenberghe a écrit : By the way, we can add the storage format: there are already different versions of the filetree format (with metadata, without, one file per method, one file per class). At some point, we need to handle this point of variability. On this point I think that we will always want to support multiple disk formats .... in a perfect world, there is no reason not to have one file per method and no Monticello meta data ... but perfection does not exist.

Monticello meta data was included in FileTree because it was needed over the last 4 years so that developers could dip there toes into using git without completely abandoning Monticello .... Now that we are gaining a critical mass of users and the tool support is starting to show up, dropping the meta data is a practical thing to do ... but there is code committed on github that hasn't been updated in years that is still being used and Monticello meta data is still present, so it is necessary to "support the older formats".

The one file per method was chosen for two reasons.

The first reason has to do with the fact that disk based SCMs are designed manage files as atomic units and in Smalltalk the method is the atomic unit for source code ... it is very convenient to be able to leverage native SCN files for a) versioning files and b) maintaining version history for a method as it moves from class to class ...

The second reason has to do with the fact that there is no cross platform fileout format for Smalltalk. Chunk format comes is "common" but chunk format also relies on executing Smalltalk code to define classes and there hasn't been a standard class creation method since the second Smalltalk implementation showed up on the planet ... Technically, class and method definitions can be derived from a file per class approach, but without a standardized file format you end up with each dialect isolating itself from the others and this is not desirable nor necessary ... It just takes more work to come up with a file format that can accommodate the different needs of the different dialects .... the one method per file and a generic properties file for class definitions[1] and the provision for allowing additional platform specific property files and directories (like monticello meta data) simplified the whole process ...

So I would like changing the disk format for sharing source code to be a multi-platform effort - which means that format that is chosen is flexible enough to allow dialects to include the meta data that they need ... It is important to note that there has never been a guarantees that all platforms would PRESERVE and WRITE foreign meta data ... the only provision that exists is that ALL platforms can READ the format while ignoring anything it doesn't recognize ....

Dale

[1] https://raw.githubusercontent.com/CampSmalltalk/Cypress/master/img/CypressSt...

Thanks Dale for all the explanations. How Monticello and version control relate in the big picture is starting to make sense for me. Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem. I think more package-based user interfaces would indeed be a very good idea, for browsing and for source code management. Stef, I have the impression you think that git is popular because it is a new shiny toy. I disagree with this idea. Git is a typical worse-is-better tool. It's good enough for most people, but it still has many shortcomings. It is popular in spite of its shortcomings. It became popular as destination for projects shifting from CVS and Subversion. So it is unlikely to be displaced by a newer, incrementally shinier tools. Anything that will displace it will have to provide an improvement of a similar magnitude as the jump between centralised and distributed version control. Still, I think it's a good idea not to restrict high level models to what git provides if that's a less than ideal fit to the image model. I have a lot of ideas to improve browsing and source code management in Pharo. I can make no promises, but I would like to produce something there.

On 01/29/2016 05:17 PM, Eliot Miranda wrote:

Hi David,

...

On Jan 29, 2016, at 2:45 PM, David Allouche <david@allouche.net> wrote:

Thanks Dale for all the explanations.

How Monticello and version control relate in the big picture is starting to make sense for me.

Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem. I agree so strongly. Class file outs which are eg sorted by selector make much more sense. They won't hit the file name length limit. They make it trivial to maintain method and class comment time stamps. They're easier to construct into snapshots because it's easier to decode the file name.

And then it's easy to add files for package load/unload scripts and for the history. And then one is much more decoupled from the specific back end. It could be mercurial just as easily as git.

...

I think more package-based user interfaces would indeed be a very good idea, for browsing and for source code management.

Stef, I have the impression you think that git is popular because it is a new shiny toy. I disagree with this idea. Git is a typical worse-is-better tool. It's good enough for most people, but it still has many shortcomings. It is popular in spite of its shortcomings. It became popular as destination for projects shifting from CVS and Subversion. So it is unlikely to be displaced by a newer, incrementally shinier tools. Anything that will displace it will have to provide an improvement of a similar magnitude as the jump between centralised and distributed version control. This is a good analysis. What's valuable to the Pharo community is not displacing an already functional dvcs (Monticello) with an ill-suited one (git), but in being able to function in ecosystems like github where people can display their identity and where infrastructure for bug reports etc exist.

...

Still, I think it's a good idea not to restrict high level models to what git provides if that's a less than ideal fit to the image model. Absolutely. Dale's talk of ditching Monticello metadata fills me with repulsion and makes me want to ask is he trying to sabotage or what? Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how about you assume that I'm doing "or what":)

The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story .... Despite the fact that the Monticello metadata is redundant, I have made sure that the Monticello metadata was included in FileTree from the very beginning for the very reason that I wanted developers to be able to try out FileTree, git and github without having to burn any Monticello bridges .... if they didn't like FileTree, git and github, then they would be able to back out of their use of git without losing data ...

It seems entirely destructive. It is not destructive ... We have a functional package manager which currently supports interchange between Pharo, Squeak and Cuis, and GemStone?

I assume that you are talking about Monticello packages and Monticello repositories ... or what? I am really not trying to do anything but "invent the future" --- I am not trying to destroy, I am trying to improve ... If you are not able to see the shortcomings of Monticello repositories (Note that I am distinguishing between Monticello packages and Monticello repositories --- FileTree uses Monticello packages and replaces Monticello repositories with git) and where git has advantages over Monticello repositories, then you should continue to use Monticello repositories ... Personally I don't see Monticello repositories going away anytime soon and expect to support Monticello repositories in GsDevKit_home, tODE, and Metacello for the rest of my life:) Dale

On 02/02/2016 12:54 PM, Eliot Miranda wrote:

Hi Dale,

On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> wrote:

On 01/29/2016 05:17 PM, Eliot Miranda wrote:

Hi David,

On Jan 29, 2016, at 2:45 PM, David Allouche <david@allouche.net <mailto:david@allouche.net>> wrote:

Thanks Dale for all the explanations.

How Monticello and version control relate in the big picture is starting to make sense for me.

Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem.

I agree so strongly. Class file outs which are eg sorted by selector make much more sense. They won't hit the file name length limit. They make it trivial to maintain method and class comment time stamps. They're easier to construct into snapshots because it's easier to decode the file name.

And then it's easy to add files for package load/unload scripts and for the history. And then one is much more decoupled from the specific back end. It could be mercurial just as easily as git.

I think more package-based user interfaces would indeed be a very good idea, for browsing and for source code management.

Stef, I have the impression you think that git is popular because it is a new shiny toy. I disagree with this idea. Git is a typical worse-is-better tool. It's good enough for most people, but it still has many shortcomings. It is popular in spite of its shortcomings. It became popular as destination for projects shifting from CVS and Subversion. So it is unlikely to be displaced by a newer, incrementally shinier tools. Anything that will displace it will have to provide an improvement of a similar magnitude as the jump between centralised and distributed version control.

This is a good analysis. What's valuable to the Pharo community is not displacing an already functional dvcs (Monticello) with an ill-suited one (git), but in being able to function in ecosystems like github where people can display their identity and where infrastructure for bug reports etc exist.

Still, I think it's a good idea not to restrict high level models to what git provides if that's a less than ideal fit to the image model.

Absolutely. Dale's talk of ditching Monticello metadata fills me with repulsion and makes me want to ask is he trying to sabotage or what?

Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how about you assume that I'm doing "or what":)

The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....

No it's /not/ the end of the story. The essential part of the story is how Monticello remains compatible and interoperable between dialects. I haven't seen you account for how you maintain that compatibility. As far as I can tell, you propose replacing the Monticello metadata with that from git. How do I, as a Squeak user with Monticello, ever get to look at your package again? As I understand it, moving the metadata from Monticello commit time to git means that the metadata is in a format that git determines, not Monticello. Good question, FileTree has been supported on Squeak since the very beginning (I along with a small number of Squeak users have made sure of that).

So TODAY, any Squeak user can "look at, load and commit" any package that has been written using FileTree (with or without Monticello meta data). [1] https://github.com/dalehenrich/filetree/tree/squeak4.3#squeak

So I don't understand how on the one hand you can say "The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....", which implies you want to eliminate the Monticello metadata, and on the other hand you say you're keeping the Monticello metadata. I'm hopelessly confused. How does the Monticello metadata get reconstituted if it's been thrown away?

Monticello meta data is not an integral part of the "package-ness" of a Monticello package ... it _is_ integral to the "repository-ness" of a Monticello package ... If the Monticello metadata is "thrown away" then the revision history for Monticello is lost, but for a package that is "born" in a git repository, the Monticello metadata is not needed. Git has it's own commit meta data and the Monticello metadata is redundant. If you want to see the revision history from a git-based FileTree repo, then: 1. one can include the Monticello meta data as part of the package - this is what FileTree currently does 2. one can build a tool that reconstructs the Monticello version history from the git metadata making it possible to use "old" Monitcello tools to look at the git repo - I believe that Thierry's GitFileTree takes this approach for metadata-less repositories 3. One can build a new tool that presents the git metadata without reconstructing the Monticello metadata at all. Note that by "embracing git", it is possible to present revision history at the package level (current mcz techology) as well as the class and method level - which is what I do in tODE

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image change P, commit via git to local git repository G load P from G into an image store P to R via Monticello

The above workflow can be accomplished whether or not Monticello metadata is present, however, if one does not make an effort to preserve the revision history then at the end of your workflow the Monticello metadata is lost. If one takes the pains to preserve the Monticello metadata before committing to the git repository and the metadata is updated with each commit during the git lifetime, then the full metadata will be present at the end ... This I think is the crux of the discussion. There are a number of alternate schemes that can be used to preserve the metadata through this scenario: 1. (current FileTree implementation) store the Monticello meta data in git and update on every git commit 2. duplicate the existing Monticello revision history by committing in order all of the package ancestors and arrange for a way to reconstruct Monticello metadata from git meta data 3. (variant of 1) store the original monticello meta data in a file, do not update on every commit but arrange for a way to reconstruct Monticello metadata from git meta data and graft onto original Monticello meta data for use in mcz repository --- on demand 4. ???? Option 3 seems to be a good compromise solution and perhaps is the approach that should be adopted moving forward ... we get to preserver Monticello metadata while avoiding the messy commit conflicts for git while providing a (somewhat) seamless path for a package to migrate back into the mcz repository world ... if we somehow incorporate the SHA of the commit and the github/bitbucket url into the revision history, then it would be possible to perform a 3 way merge involving two mcz package versions and common ancestor that is only present in git ...

Despite the fact that the Monticello metadata is redundant, I have made sure that the Monticello metadata was included in FileTree from the very beginning for the very reason that I wanted developers to be able to try out FileTree, git and github without having to burn any Monticello bridges .... if they didn't like FileTree, git and github, then they would be able to back out of their use of git without losing data ...

Then forgive me. I couldn't see the wood for the trees. When I read your talk of eliminating the conflicts from git commits due to the Monticello metadata I infer that you're eliminating the Monticello metadata. I'm not sure I understand the implications of this, but it seems to me that a natural consequence is that the Montivcello metadata is lost and at that point merges and the like become problematic. What am I missing?

You've hit the nail on the head, but I think that option 3 above gives us a way to avoid losing the Monticello metadata without incurring a hit for a packages lifetime while in git ... A package that starts its life in git will have an empty Monticello metadata and a package that never makes it's way into an mcz repository will not incur per commit penalties ...

It seems entirely destructive.

It is not destructive ...

We have a functional package manager which currently supports interchange between Pharo, Squeak and Cuis,

and GemStone?

I assume that you are talking about Monticello packages and Monticello repositories ... or what?

Yes.

I am really not trying to do anything but "invent the future" --- I am not trying to destroy, I am trying to improve ... If you are not able to see the shortcomings of Monticello repositories (Note that I am distinguishing between Monticello packages and Monticello repositories --- FileTree uses Monticello packages and replaces Monticello repositories with git) and where git has advantages over Monticello repositories, then you should continue to use Monticello repositories ...

But what does this imply to some package that starts off in a Monticello repository and then spends some time in gitland? Can I merge again? If I can I'm happy. If I can't, I feel sabotaged.

and sabotage was never my intent ...

Personally I don't see Monticello repositories going away anytime soon and expect to support Monticello repositories in GsDevKit_home, tODE, and Metacello for the rest of my life:)

Fine. Except merging is, IIUC, about method time stamps and ancestry. If that gets preserved then I'm happy. But for the life of me I haven't read an explanation that reassures me that these are being preserved. Do you see the roots of my fear?

Haha, from the very beginning back in 2012, I understood that there would be fear and resistance to change and anger and joy and excitement but it was not clear when if ever we'd reach point where a resolution for this "problem" was needed: either no one would be interested or everyone would be interested or ??? I really think that option 3 is going to be the best compromise moving forward - there is some implementation work that will be required but I really think option 3 gives you (and frankly me) a way to preserve the Monticello revision history for packages that make their way back and forth between lifetimes in git and Monticello repositories. Eliot, I appreciate the fact that you demanded a better solution! Dale

https://github.com/ThierryGoubier/GitFileTree-MergeDriver 2016-02-03 6:00 GMT+01:00 Ben Coman <btc@openinworld.com>:

On Wed, Feb 3, 2016 at 7:00 AM, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:

...

On 02/02/2016 12:54 PM, Eliot Miranda wrote:

Hi Dale,

On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:

...

On 01/29/2016 05:17 PM, Eliot Miranda wrote:

...

Hi David,

...

On Jan 29, 2016, at 2:45 PM, David Allouche <david@allouche.net>

wrote:

...

...

...

...

Thanks Dale for all the explanations.

How Monticello and version control relate in the big picture is

starting

...

to make sense for me.

Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem.

I agree so strongly. Class file outs which are eg sorted by selector make much more sense. They won't hit the file name length limit. They make it trivial to maintain method and class comment time stamps. They're easier to construct into snapshots because it's easier to decode the file name.

And then it's easy to add files for package load/unload scripts and for the history. And then one is much more decoupled from the specific back end. It could be mercurial just as easily as git.

...

I think more package-based user interfaces would indeed be a very good idea, for browsing and for source code management.

Stef, I have the impression you think that git is popular because it is a new shiny toy. I disagree with this idea. Git is a typical worse-is-better tool. It's good enough for most people, but it still has many shortcomings. It is popular in spite of its shortcomings. It became popular as destination for projects shifting from CVS and Subversion. So it is unlikely to be displaced by a newer, incrementally shinier tools. Anything that will displace it will have to provide an improvement of a similar magnitude as the jump between centralised and distributed version control.

This is a good analysis. What's valuable to the Pharo community is not displacing an already functional dvcs (Monticello) with an ill-suited one (git), but in being able to function in ecosystems like github where people can display their identity and where infrastructure for bug reports etc exist.

...

Still, I think it's a good idea not to restrict high level models to what git provides if that's a less than ideal fit to the image model.

Absolutely. Dale's talk of ditching Monticello metadata fills me with repulsion and makes me want to ask is he trying to sabotage or what?

Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how about you assume that I'm doing "or what":)

The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....

No it's /not/ the end of the story. The essential part of the story is how Monticello remains compatible and interoperable between dialects. I haven't seen you account for how you maintain that compatibility. As far as I can tell, you propose replacing the Monticello metadata with that from git. How do I, as a Squeak user with Monticello, ever get to look at your package again? As I understand it, moving the metadata from Monticello commit time to git means that the metadata is in a format that git determines, not Monticello.

Good question, FileTree has been supported on Squeak since the very beginning (I along with a small number of Squeak users have made sure of that).

So TODAY, any Squeak user can "look at, load and commit" any package that has been written using FileTree (with or without Monticello meta data).

[1] https://github.com/dalehenrich/filetree/tree/squeak4.3#squeak

So I don't understand how on the one hand you can say "The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....", which implies you want to eliminate the Monticello metadata, and on the other hand you say you're keeping the Monticello metadata. I'm hopelessly confused. How does the Monticello metadata get reconstituted if it's been thrown away?

Monticello meta data is not an integral part of the "package-ness" of a Monticello package ... it _is_ integral to the "repository-ness" of a Monticello package ...

If the Monticello metadata is "thrown away" then the revision history for Monticello is lost, but for a package that is "born" in a git repository, the Monticello metadata is not needed. Git has it's own commit meta data and the Monticello metadata is redundant.

If you want to see the revision history from a git-based FileTree repo, then:

1. one can include the Monticello meta data as part of the package - this is what FileTree currently does 2. one can build a tool that reconstructs the Monticello version history from the git metadata making it possible to use "old" Monitcello tools to look at the git repo - I believe that Thierry's GitFileTree takes this approach for metadata-less repositories 3. One can build a new tool that presents the git metadata without reconstructing the Monticello metadata at all. Note that by "embracing git", it is possible to present revision history at the package level (current mcz techology) as well as the class and method level - which is what I do in tODE

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image change P, commit via git to local git repository G load P from G into an image store P to R via Monticello

The above workflow can be accomplished whether or not Monticello metadata is present, however, if one does not make an effort to preserve the revision history then at the end of your workflow the Monticello metadata is lost.

If one takes the pains to preserve the Monticello metadata before committing to the git repository and the metadata is updated with each commit during the git lifetime, then the full metadata will be present at the end ...

This I think is the crux of the discussion.

There are a number of alternate schemes that can be used to preserve the metadata through this scenario:

1. (current FileTree implementation) store the Monticello meta data in git and update on every git commit 2. duplicate the existing Monticello revision history by committing in order all of the package ancestors and arrange for a way to reconstruct Monticello metadata from git meta data 3. (variant of 1) store the original monticello meta data in a file, do not update on every commit but arrange for a way to reconstruct Monticello metadata from git meta data and graft onto original Monticello meta data for use in mcz repository --- on demand 4. ????

Option 3 seems to be a good compromise solution and perhaps is the approach that should be adopted moving forward ... we get to preserver Monticello metadata while avoiding the messy commit conflicts for git while providing a (somewhat) seamless path for a package to migrate back into the mcz repository world ... if we somehow incorporate the SHA of the commit and the github/bitbucket url into the revision history, then it would be possible to perform a 3 way merge involving two mcz package versions and common ancestor that is only present in git ...

...

Despite the fact that the Monticello metadata is redundant, I have made sure that the Monticello metadata was included in FileTree from the very beginning for the very reason that I wanted developers to be able to try out FileTree, git and github without having to burn any Monticello bridges .... if they didn't like FileTree, git and github, then they would be able to back out of their use of git without losing data ...

Then forgive me. I couldn't see the wood for the trees. When I read your talk of eliminating the conflicts from git commits due to the Monticello metadata I infer that you're eliminating the Monticello metadata. I'm not sure I understand the implications of this, but it seems to me that a natural consequence is that the Montivcello metadata is lost and at that point merges and the like become problematic. What am I missing?

You've hit the nail on the head, but I think that option 3 above gives us a way to avoid losing the Monticello metadata without incurring a hit for a packages lifetime while in git ...

A package that starts its life in git will have an empty Monticello metadata and a package that never makes it's way into an mcz repository will not incur per commit penalties ...

...

...

It seems entirely destructive.

It is not destructive ...

...

We have a functional package manager which currently supports

interchange

...

...

between Pharo, Squeak and Cuis,

and GemStone?

I assume that you are talking about Monticello packages and Monticello repositories ... or what?

Yes.

...

I am really not trying to do anything but "invent the future" --- I am

not

...

trying to destroy, I am trying to improve ... If you are not able to see the shortcomings of Monticello repositories (Note that I am distinguishing between Monticello packages and Monticello repositories --- FileTree uses Monticello packages and replaces Monticello repositories with git) and where git has advantages over Monticello repositories, then you should continue to use Monticello repositories ...

But what does this imply to some package that starts off in a Monticello repository and then spends some time in gitland? Can I merge again? If I can I'm happy. If I can't, I feel sabotaged.

and sabotage was never my intent ...

...

Personally I don't see Monticello repositories going away anytime soon

and

...

expect to support Monticello repositories in GsDevKit_home, tODE, and Metacello for the rest of my life:)

Fine. Except merging is, IIUC, about method time stamps and ancestry. If that gets preserved then I'm happy. But for the life of me I haven't read an explanation that reassures me that these are being preserved. Do you see the roots of my fear?

Haha, from the very beginning back in 2012, I understood that there would be fear and resistance to change and anger and joy and excitement but it was not clear when if ever we'd reach point where a resolution for this "problem" was needed: either no one would be interested or everyone would be interested or ???

I really think that option 3 is going to be the best compromise moving forward - there is some implementation work that will be required but I really think option 3 gives you (and frankly me) a way to preserve the Monticello revision history for packages that make their way back and forth between lifetimes in git and Monticello repositories.

Eliot, I appreciate the fact that you demanded a better solution!

Dale

I'm curious how the merge driver is implemented. I think it was mentioned that git calls-back to Pharo to do the processing. Is this it something like this...

* How to make Git preserve specific files while merging

https://medium.com/@porteneuve/how-to-make-git-preserve-specific-files-while...

* A few of my Git tricks, tips and workflows section under gitattribute(5) http://nuclearsquid.com/writings/git-tricks-tips-workflows/

Or... rather than the metadata being a specific file (e.g. 'metadata'), the file could change for each commit (e.g. 'metadata.nnn') where nnn is the revision number, so each commit deletes the old metadata file and writes the new metadata file. Monticello could know to use whichever single 'metadata.*' file exists. Would that avoid the problem of metadata merge conflict?

cheers -ben

Thierry, Very good points and I agree with all of them ... Regarding a Smalltalk-based merge tool, I've written a git mergetool for tODE (consider it a prototype) that could be adapted/ported for Pharo .... My thoughts regarding the MC issues ... Fyou make good points some of the "internal bugs" in MC and I think that it is important to point out that the "internal bugs" you are referring to only affect Monticello with regards to its repository functionality and not its package functionality . My inclination is to move forward and create a new, simpler package model (Cypress is a good name) that is used with disk-based repos ... the new package model would be protocol compatible with MC so that definition-based snapshot comparisons continue. The new package model would only be used for loading and there could/should be facilities for converting between MC and this new package model. FileTree would continue to be MC based and Cypress would be able to read FileTree and FileTree would be able to read Cypress. The improvements and changes that we are discussing (many of which require a fair amount of implementation work) would be done in Cypress ... The rationale is that Monticello packages encompass both the repository and the code/definitions and for disk-based scms, Cypress would only need to worry about the code/definitions --- and would make it possible to move away from the version number-based package names that aren't necessary when the underlying scm is doing the versioning without affecting the MC implementation .... Of course, if a new Cypress package model were introduced, the tools would need to change, but since we are already considering changing tools, this would be the perfect time to improve the underlying model for disk-based SCMs. Dale On 2/3/16 1:13 AM, Thierry Goubier wrote:

I went through all the different possible file formats, class-based, package-based, method-based, log metadata and the like, and I concluded that:

- the method based format is as good as any other. - method based format allow for method-history queries on the git/vcs history (as well as class based / package based queries). - the tree structure on github or bitbucket is quite convenient (and browsable) to the point one could edit a package directly in it (I do when I need to do a quick fix). - anything that can compress a bit the metadata version is probably good to consider. version files can be huge. - merge drivers really work and releave us from conflict resolution - we need a merge tool written in Smalltalk/MC - MC version numbering is a very bad idea - MC almost never using properly UUIDs is a very bad behavior - MC packages history can be considered broken in the general case - it takes time to define, implement, test and really use a new format and tooling

Thanks Thierry for pushing this subject and working so patiently with the community.

On 03 Feb 2016, at 10:18, Thierry Goubier <thierry.goubier@gmail.com> wrote:

I went through all the different possible file formats, class-based, package-based, method-based, log metadata and the like, and I concluded that:

- the method based format is as good as any other. Even better since it has a spec (cypress). - method based format allow for method-history queries on the git/vcs history (as well as class based / package based queries). - the tree structure on github or bitbucket is quite convenient (and browsable) to the point one could edit a package directly in it (I do when I need to do a quick fix). - anything that can compress a bit the metadata version is probably good to consider. version files can be huge. - merge drivers really work and releave us from conflict resolution - we need a merge tool written in Smalltalk/MC - MC version numbering is a very bad idea - MC almost never using properly UUIDs is a very bad behavior - MC packages history can be considered broken in the general case - it takes time to define, implement, test and really use a new format and tooling

Excellent summary. I also feel that the current structure (basic filetree) is more than OK. You can /almost/ browse through github. I am sure we'll get there with the meta data and tooling. Sven

Le 05/02/2016 11:33, Christophe Demarey a écrit :

Hi Thierry,

Just some thoughts I wanted to share:

Le 3 févr. 2016 à 10:18, Thierry Goubier a écrit :

...

I went through all the different possible file formats, class-based, package-based, method-based, log metadata and the like, and I concluded that:

- the method based format is as good as any other. Even better since it has a spec (cypress).

I see cons that a class (or package) format would not have. One file per method approach leads to generate plenty of small files. In general, file systems do not like that: - it may consumes a lot of space. I remember I had a Java/maven project with a lot of small files and I got to fill the inodes tables on my unix system. - you generate long pathes. Long pathes are not user-friendly and some OS have restrictions on path length.

The method based structure of filetree is very close to how code is navigated in Smalltalk browsers: one method at a time, with a package/class/protocol hierarchical layering on top. The one file per class / one file per package is a reference to the base unit of C / C++. And no OS in general use has path restrictions that matter. Ok, the windows vm has issues, but this is a vm bug, not a filesystem issue.

By adopting a file per method approach, you also increase the distance to get a common script format for Smalltalk. Here I mean a file where you could define classes, methods, and run arbitrary portions of Smalltalk code.

This format is called fileout, and already exist. All you describe is also available in the FileTree/Cypress format and is technically better specified.

...

- method based format allow for method-history queries on the git/vcs history (as well as class based / package based queries). - the tree structure on github or bitbucket is quite convenient (and browsable) to the point one could edit a package directly in it (I do when I need to do a quick fix).

but is a pain to navigate: too much click to effectively browse a method content.

You must hate Nautilus, then, since this is Nautilus approach as well. Just count the number of clicks you do in a Nautilus, and the number in github. If we remove the instance sub-directory and write instance-side methods just below the class name in filetree, then you'll get the exact same number of clicks to reach a method than in Nautilus. Fun fact: if you do that with the Mac finder in NexT mode over a filetree repository (miller columns), you'll see that it almost looks like a Nautilus top panes.

I do not know what would be the best format but I think we need to take care to do not generate too much files / folders. File system and VCS will appreciate also.

I'd say, overall, what we need to remember is that we produce a lot less lines of code than other languages, and that we shouldn't over-optimize. I'll probably look into optimising FileTree-like writing in the future; I wasn't that good into planning for it and it shows in specific cases. Thierry

Le 05/02/2016 14:55, Christophe Demarey a écrit :

Le 5 févr. 2016 à 14:33, Thierry Goubier a écrit :

...

Le 05/02/2016 11:33, Christophe Demarey a écrit :

...

Hi Thierry,

Just some thoughts I wanted to share:

Le 3 févr. 2016 à 10:18, Thierry Goubier a écrit :

...

I went through all the different possible file formats, class-based, package-based, method-based, log metadata and the like, and I concluded that:

- the method based format is as good as any other. Even better since it has a spec (cypress).

I see cons that a class (or package) format would not have. One file per method approach leads to generate plenty of small files. In general, file systems do not like that: - it may consumes a lot of space. I remember I had a Java/maven project with a lot of small files and I got to fill the inodes tables on my unix system. - you generate long pathes. Long pathes are not user-friendly and some OS have restrictions on path length.

The method based structure of filetree is very close to how code is navigated in Smalltalk browsers: one method at a time, with a package/class/protocol hierarchical layering on top. The one file per class / one file per package is a reference to the base unit of C / C++.

I fully agree with that. As we have a lot (small) methods, we will have a lot of small files and some file-system does not like that. I remember huge slow-down be cause of that. It is good to keep that in mind.

The problem is linked with writing too many files. Because of a possible uncertainty about the on-disk state, FileTree erase the complete package directory then rewrites everything, letting the vcs decide what has really changed. This is doubly slow, because it hits the filesystem and the vcs. I said to Dale I'd see into a diff based writer; it should improve things a lot.

...

And no OS in general use has path restrictions that matter. Ok, the windows vm has issues, but this is a vm bug, not a filesystem issue.

Windows command-line has (had) this limitation.

Good to know.

...

...

By adopting a file per method approach, you also increase the distance to get a common script format for Smalltalk. Here I mean a file where you could define classes, methods, and run arbitrary portions of Smalltalk code.

This format is called fileout, and already exist.

I mean something like a python script: http://archive.stsci.edu/vo/python_examples.html

Not entirely keen in going that way. I prefer declarative formats for storing packages. And I still think that the fileout format is that (a sequence of scripts to execute, separated by !!).

...

All you describe is also available in the FileTree/Cypress format and is technically better specified.

...

...

- method based format allow for method-history queries on the git/vcs history (as well as class based / package based queries). - the tree structure on github or bitbucket is quite convenient (and browsable) to the point one could edit a package directly in it (I do when I need to do a quick fix).

but is a pain to navigate: too much click to effectively browse a method content.

You must hate Nautilus, then, since this is Nautilus approach as well. Just count the number of clicks you do in a Nautilus, and the number in github.

but we have spotter! (I just miss the exact search to not click and scroll too much)

Then you want spotter on the web :)

...

If we remove the instance sub-directory and write instance-side methods just below the class name in filetree, then you'll get the exact same number of clicks to reach a method than in Nautilus.

it would be a good idea

Why not.

...

Fun fact: if you do that with the Mac finder in NexT mode over a filetree repository (miller columns), you'll see that it almost looks like a Nautilus top panes.

...

I do not know what would be the best format but I think we need to take care to do not generate too much files / folders. File system and VCS will appreciate also.

I'd say, overall, what we need to remember is that we produce a lot less lines of code than other languages, and that we shouldn't over-optimize.

I'll probably look into optimising FileTree-like writing in the future; I wasn't that good into planning for it and it shows in specific cases.

It is actually the problem: we generate a lot of small files. I do not have numbers but I think it would be good to stress a bit a file system to see where we hit the barrier and compare with the pharo code base. From the git side, I'm not aware of a limitation regarding small files.

I'm sure the numbers are already available. And, as I said above, you may be measuring FileTree implementation limitations and nothing related to filesystem issues (or git issues). Thierry

On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

Hi Eliot,

Le 02/02/2016 21:54, Eliot Miranda a écrit : ....

...

No it's /not/ the end of the story. The essential part of the story is how Monticello remains compatible and interoperable between dialects. I haven't seen you account for how you maintain that compatibility. As far as I can tell, you propose replacing the Monticello metadata with that from git. How do I, as a Squeak user with Monticello, ever get to look at your package again? As I understand it, moving the metadata from Monticello commit time to git means that the metadata is in a format that git determines, not Monticello.

Yes. See below why.

So I don't understand how on the one hand you can say "The Monticello

...

metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....", which implies you want to eliminate the Monticello metadata, and on the other hand you say you're keeping the Monticello metadata. I'm hopelessly confused. How does the Monticello metadata get reconstituted if it's been thrown away?

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image change P, commit via git to local git repository G load P from G into an image store P to R via Monticello

It's not a scenario I've specifically worked on, but all the tech is implemented / implementable to do that perfectly.

The only thing that is problematic there is that the only safe history is the one generated from git... there are so many MC packages with broken history that, on mcz packages, you have to admit that it's not safe to base things on their history.

I'm sorry but I don't accept that. In the Squeak trunk we have history in our mczs that is correct. Certainly in VMMaker.oscog I have history that goes back a long time. If bugs have broken history then efforts should be made to repair that history. But you can't just write off Monticello history like that. But what does this imply to some package that starts off in a Monticello

...

repository and then spends some time in gitland? Can I merge again? If I can I'm happy. If I can't, I feel sabotaged.

You could. Just express your needs and wait until one of us has enough free time to solve it for you, that is.

Fine. Except merging is, IIUC, about method time stamps and ancestry.

...

If that gets preserved then I'm happy. But for the life of me I haven't read an explanation that reassures me that these are being preserved. Do you see the roots of my fear?

Of course. But that also project a bit what you think of the people working on it... which may makes it a bit hard to answer.

Thierry

_,,,^..^,,,_ best, Eliot

On 02/03/2016 02:34 PM, Thierry Goubier wrote:

Le 03/02/2016 22:51, Eliot Miranda a écrit :

...

On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>> wrote:

Hi Eliot,

Le 02/02/2016 21:54, Eliot Miranda a écrit : ....

No it's /not/ the end of the story. The essential part of the story is how Monticello remains compatible and interoperable between dialects. I haven't seen you account for how you maintain that compatibility. As far as I can tell, you propose replacing the Monticello metadata with that from git. How do I, as a Squeak user with Monticello, ever get to look at your package again? As I understand it, moving the metadata from Monticello commit time to git means that the metadata is in a format that git determines, not Monticello.

Yes. See below why.

So I don't understand how on the one hand you can say "The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....", which implies you want to eliminate the Monticello metadata, and on the other hand you say you're keeping the Monticello metadata. I'm hopelessly confused. How does the Monticello metadata get reconstituted if it's been thrown away?

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image change P, commit via git to local git repository G load P from G into an image store P to R via Monticello

It's not a scenario I've specifically worked on, but all the tech is implemented / implementable to do that perfectly.

The only thing that is problematic there is that the only safe history is the one generated from git... there are so many MC packages with broken history that, on mcz packages, you have to admit that it's not safe to base things on their history.

I'm sorry but I don't accept that. In the Squeak trunk we have history in our mczs that is correct. Certainly in VMMaker.oscog I have history that goes back a long time. If bugs have broken history then efforts should be made to repair that history. But you can't just write off Monticello history like that.

I don't. You presuppose.

I write tools that work with Monticello repositories, not just yours. I have to do with what is given to me. On a general level, as a mcz user, I'll just have to consider that you are as susceptible to be trusted as with any other mcz producer. This means not much...

... And this is the reason why I am inclined to favor option 3, which records the package version history as it existed at the point it was copied into a git repo. When copied back out from the git universe, create a version history that starts with the original version history and generates a history of the package in git .... Correct or not, the Monticello version history should be preserved .... Dale

I don't understand broken history either. Yes there can be .mcz name clashes but UUID history is stored together in metadata no? If some tool only trust .mcz name without checking UUID, consider it's a bug, and let's correct it (there is some in Monticello Configuration Map) Or is it the fact that some .mcz could be missing? Consider this is a feature, MC tools are robust to missing .mcz (it's just that you'll have to redo the merge if you lost a common ancestor). Or is it the fact that some repository might contain only a slice of history? This is another feature... You can view all the versions in a collection of repositories without needing to replicate. You can replicate if you want but it's not mandatory and completely orthogonal. So yes, this information - the list or repositories you want to consider - has to be stored separetely and this can sound quite unconventional. But IMO, it's an important feature: it gives much resilience for a very low investment. And that also mean that you can hardly break things (have unconsistent history). Maybe when you say broken, you mean not 100% git compatible? 2016-02-04 7:53 GMT+01:00 Thierry Goubier <thierry.goubier@gmail.com>:

Le 03/02/2016 23:58, Dale Henrichs a écrit :

...

On 02/03/2016 02:34 PM, Thierry Goubier wrote:

...

Le 03/02/2016 22:51, Eliot Miranda a écrit :

...

On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>> wrote:

Hi Eliot,

Le 02/02/2016 21:54, Eliot Miranda a écrit : ....

No it's /not/ the end of the story. The essential part of the story is how Monticello remains compatible and interoperable between dialects. I haven't seen you account for how you maintain that compatibility. As far as I can tell, you propose replacing the Monticello metadata with that from git. How do I, as a Squeak user with Monticello, ever get to look at your package again? As I understand it, moving the metadata from Monticello commit time to git means that the metadata is in a format that git determines, not Monticello.

Yes. See below why.

So I don't understand how on the one hand you can say "The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....", which implies you want to eliminate the Monticello metadata, and on the other hand you say you're keeping the Monticello metadata. I'm hopelessly confused. How does the Monticello metadata get reconstituted if it's been thrown away?

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image change P, commit via git to local git repository G load P from G into an image store P to R via Monticello

It's not a scenario I've specifically worked on, but all the tech is implemented / implementable to do that perfectly.

The only thing that is problematic there is that the only safe history is the one generated from git... there are so many MC packages with broken history that, on mcz packages, you have to admit that it's not safe to base things on their history.

I'm sorry but I don't accept that. In the Squeak trunk we have history in our mczs that is correct. Certainly in VMMaker.oscog I have history that goes back a long time. If bugs have broken history then efforts should be made to repair that history. But you can't just write off Monticello history like that.

I don't. You presuppose.

I write tools that work with Monticello repositories, not just yours. I have to do with what is given to me. On a general level, as a mcz user, I'll just have to consider that you are as susceptible to be trusted as with any other mcz producer. This means not much...

... And this is the reason why I am inclined to favor option 3, which records the package version history as it existed at the point it was copied into a git repo. When copied back out from the git universe, create a version history that starts with the original version history and generates a history of the package in git ....

Which is not very difficult to do given how GitFileTree is implemented. And I agree this may well be the way to go. But ...

Correct or not, the Monticello version history should be preserved ....

...

I wonder about that. The property of the Monticello version history is that it has value when you can access the versions listed in it. If you mix repositories like that, unless you maintain a link to the previous mcz repository, pre-git versions can't be accessed. So, most of the time, what we do with a project moved under git is to clone the previous repository, not take just the current head.

(you'll notice, by the way, that vcs usually work that way when moving from, say, CVS to git - you move the entire repository, not just the latest version).

In short, the question would really be: 1- should we invest into making that integration of past history a selling point (but I foresee issues down the road; I've only described one so far, and I have seen others) or 2- into making a better "clone" of a package history, timestamps and everything when moving a complete repository to git?

Honestly, I'd consider 1- to be the easiest to implement. 2- there is already some code floating around.

Thierry

On 04 Feb 2016, at 10:26 , Thierry Goubier <thierry.goubier@gmail.com> wrote:

Le 04/02/2016 10:04, Nicolas Cellier a écrit :

...

I don't understand broken history either. Yes there can be .mcz name clashes but UUID history is stored together in metadata no?

Yes.

...

If some tool only trust .mcz name without checking UUID, consider it's a bug, and let's correct it (there is some in Monticello Configuration Map)

Gofer. In fact, most of Monticello never checks the UUID, only dependencies do, but this looks like a deprecated feature given how often it is used.

...

Or is it the fact that some .mcz could be missing? Consider this is a feature, MC tools are robust to missing .mcz (it's just that you'll have to redo the merge if you lost a common ancestor).

What? You really consider that a feature?

...

Or is it the fact that some repository might contain only a slice of history? This is another feature... You can view all the versions in a collection of repositories without needing to replicate.

Understandable in theory. Unworkable over time and change (repositories disappear and die, and this stops working)

...

You can replicate if you want but it's not mandatory and completely orthogonal. So yes, this information - the list or repositories you want to consider - has to be stored separetely and this can sound quite unconventional. But IMO, it's an important feature: it gives much resilience for a very low investment. And that also mean that you can hardly break things (have unconsistent history).

...

Maybe when you say broken, you mean not 100% git compatible?

No, what I say is true mcz inconsistent history (missing versions making merges very unreliable, basically).

I describe a while ago a case where, thanks to mcz features, I couldn't merge a small change done to Roassal without generating a ton of conflicts. I moved the three needed mcz(s) to git (the change ancestor, the change, and the current head), did git merge and had no conflicts.

If you consider those features, then I disagree.

Also, the ancestry of all packages in Pharo Core was truncated a few years back, which also screwed up merging*. I don't think anyone looked into why it happened. Cheers, Henry *http://forum.world.st/FogBugz-Case-Issue-12776-Tools-Cancelling-quot-Previou...

Le 04/02/2016 13:21, Stephan Eggermont a écrit :

On 04-02-16 13:09, Henrik Johansen wrote:

...

Also, the ancestry of all packages in Pharo Core was truncated a few years back, which also screwed up merging*. I don't think anyone looked into why it happened.

My experiment in creating one file for a whole project would be a suitable starting point for fixing that.

Yes. I think we need a MC version history recreation / garbage collect. Something that removes dead versions and compress history to make MC merge work correctly. (unless you move to git on metadataless mode and back...). Thierry

Stephan

On 04-02-16 17:22, Dale Henrichs wrote:

Stephan,

Have you described this anywhere?

Not very much http://forum.world.st/Overhead-in-mczs-MCVersionInfo-td4869374.html MCHttpRepository location: 'http://smalltalkhub.com/mc/StephanEggermont/MonticelloProjects/main' user: '' password: '' Jan 5. on Slack Stephan Eggermont 20:21 So I deduplicated the ancestry, putting the UUID of missing ancestors in place of the MCVersionInfo 20:25 123 MB instead of 1.2 GB, compressed 70.8 MB 20:26 The whole history of VMMaker Thierry Goubier 20:34 With all versions? Stephan Eggermont 20:34 missing 2, 20:35 and only those on source.squeak.org Stephan Eggermont 20:35 CompactVMMaker.fl.tar.gz 67MB Binary Click to download Thierry Goubier 20:37 I'd be interested to see how large would be the git repository containing that. Maybe I'll make a try. Stephan Eggermont 20:42 Nice compression factor. package cache of 2725 items, 6.1 GB Thierry Goubier 20:44 ! Thierry Goubier 21:21 Then I can't test the git thing. Not enough HD space on the Chromebook.

2016-02-04 13:09 GMT+01:00 Henrik Johansen <henrik.s.johansen@veloxit.no>:

...

On 04 Feb 2016, at 10:26 , Thierry Goubier <thierry.goubier@gmail.com> wrote:

Le 04/02/2016 10:04, Nicolas Cellier a écrit :

...

I don't understand broken history either. Yes there can be .mcz name clashes but UUID history is stored together in metadata no?

Yes.

...

If some tool only trust .mcz name without checking UUID, consider it's a bug, and let's correct it (there is some in Monticello Configuration Map)

Gofer. In fact, most of Monticello never checks the UUID, only dependencies do, but this looks like a deprecated feature given how often it is used.

...

Or is it the fact that some .mcz could be missing? Consider this is a feature, MC tools are robust to missing .mcz (it's just that you'll have to redo the merge if you lost a common ancestor).

What? You really consider that a feature?

...

Or is it the fact that some repository might contain only a slice of history? This is another feature... You can view all the versions in a collection of repositories without needing to replicate.

Understandable in theory. Unworkable over time and change (repositories disappear and die, and this stops working)

...

You can replicate if you want but it's not mandatory and completely orthogonal. So yes, this information - the list or repositories you want to consider - has to be stored separetely and this can sound quite unconventional. But IMO, it's an important feature: it gives much resilience for a very low investment. And that also mean that you can hardly break things (have unconsistent history).

...

Maybe when you say broken, you mean not 100% git compatible?

No, what I say is true mcz inconsistent history (missing versions making merges very unreliable, basically).

I describe a while ago a case where, thanks to mcz features, I couldn't merge a small change done to Roassal without generating a ton of conflicts. I moved the three needed mcz(s) to git (the change ancestor, the change, and the current head), did git merge and had no conflicts.

If you consider those features, then I disagree.

Also, the ancestry of all packages in Pharo Core was truncated a few years back, which also screwed up merging*. I don't think anyone looked into why it happened.

Cheers, Henry

* http://forum.world.st/FogBugz-Case-Issue-12776-Tools-Cancelling-quot-Previou...

Henry, you perfectly nailed it... Break it first/throw away/replace is not allways a winning strategy. We are ready to re-integrate source code in image at the price of a few MBytes, but it was urgent to throw history away BEFORE we even get a working replacement... Here is a mail i did not send because I didn't want to be to start the year too negativily: ---------------------------- "snip... was about a regexp bug that I tried to bisect... (a post of january 2015) So I wanted to be informed of the other changes going on since plenty of interesting things are going on in Pharo. Ah hem, but the package has been renamed and the 'version history' menu is there only to show me that history has been purposely expunged from image... It became such a useless menu now, that i think you could as well remove it. Spawning the 4.0 repository and using the History button, reveals a bit of history this time, sounds like the 3.0 bits... Notice in attachment how the versions 22 and 23 which are in the 3.0 repository seems to be missing... Ah, no this is a brand new strategy for presenting the merged branches... without any care for a common ancestor. Notice in other attachment how it's not allways been so. And what if I want to look a bit further behind? Shall I go outside the image and browse github now? The web UI did absolutely not work for me, the 3.0 history is cut there too, and blame only reveals me the name of Jenkins... Git command line might have done a better work, but anyway, doesn't the I in IDE stands for Integrated? I won't know because at this stage i gave up. snip... that's enough ranting "

2016-02-04 10:26 GMT+01:00 Thierry Goubier <thierry.goubier@gmail.com>:

Le 04/02/2016 10:04, Nicolas Cellier a écrit :

...

I don't understand broken history either. Yes there can be .mcz name clashes but UUID history is stored together in metadata no?

Yes.

If some tool only trust .mcz name without checking UUID, consider it's a

...

bug, and let's correct it (there is some in Monticello Configuration Map)

Gofer. In fact, most of Monticello never checks the UUID, only dependencies do, but this looks like a deprecated feature given how often it is used.

Or is it the fact that some .mcz could be missing?

...

Consider this is a feature, MC tools are robust to missing .mcz (it's just that you'll have to redo the merge if you lost a common ancestor).

What? You really consider that a feature?

Sure. Consider it's somehow like a git squash...

Or is it the fact that some repository might contain only a slice of

...

history? This is another feature... You can view all the versions in a collection of repositories without needing to replicate.

Understandable in theory. Unworkable over time and change (repositories disappear and die, and this stops working)

But that's the same with over vcs, repositories disappear and die (google code etc...). Knowing that you can take usual defensive decisions (replicate what you depend on and not trust too much).

You can replicate if you want but it's not mandatory and completely

...

orthogonal. So yes, this information - the list or repositories you want to consider - has to be stored separetely and this can sound quite unconventional. But IMO, it's an important feature: it gives much resilience for a very low investment. And that also mean that you can hardly break things (have unconsistent history).

Maybe when you say broken, you mean not 100% git compatible?

...

No, what I say is true mcz inconsistent history (missing versions making merges very unreliable, basically).

Loosing a .mcz on which you branched is a bit more than unfortunate, it's

a lack of understanding of the tool... I say give git squash and rebase to unexperimented/uninformed people and that will be far worse.

I describe a while ago a case where, thanks to mcz features, I couldn't merge a small change done to Roassal without generating a ton of conflicts. I moved the three needed mcz(s) to git (the change ancestor, the change, and the current head), did git merge and had no conflicts.

Do you mean that git succeeded in merging text because different lines were changed in different branches? I agree that it can solve some basic refactoring (renaming...) But couldn't it lead to syntactically incorrect code? Traditional Smalltalk IDE are not well equipped for handling this... The biggest grief I have with MC is more about package delimitation, when code is moved from 1 package to another. For example, this effectively prevent merging very distant branches like Squeak and Pharo (but we do not want to merge, just cherry pick in fact).

If you consider those features, then I disagree.

I'd really like to improve MC and get a better integration. Considering those as features just make me think that Eliot may well be right in believing we'll end up throwing away MC completely.

As long as it's as efficient, and well integrated in IDE, but again, it's against the small steps approach... For me small steps would be to replace mcz name indexing by UUID indexing and just keep name as a façade. I'm quite sure most work is at server side. How difficult would it really be? No one will consider this maybe, because it's against the stream ;)

Coping with MC idiosyncrasies is hard as it is, and at least a complete replacement is being considered for Pharo. In the meantime, Cuis has completely given up managing packages in Smalltalk: all is done in git, externally.

But Cuis vcs come from dinosaure era (change sets) so anything else is already a progress. Doing that in Squeak would feel like a loss of tool/functionality/integration. Why didn't Pharo took the same path? Thierry

2016-02-04 7:53 GMT+01:00 Thierry Goubier <thierry.goubier@gmail.com

...

<mailto:thierry.goubier@gmail.com>>:

Le 03/02/2016 23:58, Dale Henrichs a écrit :

On 02/03/2016 02:34 PM, Thierry Goubier wrote:

Le 03/02/2016 22:51, Eliot Miranda a écrit :

On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com> <mailto:thierry.goubier@gmail.com

<mailto:thierry.goubier@gmail.com>>> wrote:

Hi Eliot,

Le 02/02/2016 21:54, Eliot Miranda a écrit : ....

No it's /not/ the end of the story. The essential part of the story is how Monticello remains compatible and interoperable between dialects. I haven't seen you account for how you maintain that compatibility. As far as I can tell, you propose replacing the Monticello metadata with that from git. How do I, as a Squeak user with Monticello, ever get to look at your package again? As I understand it, moving the metadata from Monticello commit time to git means that the metadata is in a format that git determines, not Monticello.

Yes. See below why.

So I don't understand how on the one hand you can say "The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....", which implies you want to eliminate the Monticello metadata, and on the other hand you say you're keeping the Monticello metadata. I'm hopelessly confused. How does the Monticello metadata get reconstituted if it's been thrown away?

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image change P, commit via git to local git repository G load P from G into an image store P to R via Monticello

It's not a scenario I've specifically worked on, but all the tech is implemented / implementable to do that perfectly.

The only thing that is problematic there is that the only safe history is the one generated from git... there are so many MC packages with broken history that, on mcz packages, you have to admit that it's not safe to base things on their history.

I'm sorry but I don't accept that. In the Squeak trunk we have history in our mczs that is correct. Certainly in VMMaker.oscog I have history that goes back a long time. If bugs have broken history then efforts should be made to repair that history. But you can't just write off Monticello history like that.

I don't. You presuppose.

I write tools that work with Monticello repositories, not just yours. I have to do with what is given to me. On a general level, as a mcz user, I'll just have to consider that you are as susceptible to be trusted as with any other mcz producer. This means not much...

... And this is the reason why I am inclined to favor option 3, which records the package version history as it existed at the point it was copied into a git repo. When copied back out from the git universe, create a version history that starts with the original version history and generates a history of the package in git ....

Which is not very difficult to do given how GitFileTree is implemented. And I agree this may well be the way to go. But ...

Correct or not, the Monticello version history should be preserved ....

I wonder about that. The property of the Monticello version history is that it has value when you can access the versions listed in it. If you mix repositories like that, unless you maintain a link to the previous mcz repository, pre-git versions can't be accessed. So, most of the time, what we do with a project moved under git is to clone the previous repository, not take just the current head.

(you'll notice, by the way, that vcs usually work that way when moving from, say, CVS to git - you move the entire repository, not just the latest version).

In short, the question would really be: 1- should we invest into making that integration of past history a selling point (but I foresee issues down the road; I've only described one so far, and I have seen others) or 2- into making a better "clone" of a package history, timestamps and everything when moving a complete repository to git?

Honestly, I'd consider 1- to be the easiest to implement. 2- there is already some code floating around.

Thierry

On 2/3/16 10:53 PM, Thierry Goubier wrote:

Le 03/02/2016 23:58, Dale Henrichs a écrit :

...

On 02/03/2016 02:34 PM, Thierry Goubier wrote:

...

Le 03/02/2016 22:51, Eliot Miranda a écrit :

...

On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>> wrote:

Hi Eliot,

Le 02/02/2016 21:54, Eliot Miranda a écrit : ....

No it's /not/ the end of the story. The essential part of the story is how Monticello remains compatible and interoperable between dialects. I haven't seen you account for how you maintain that compatibility. As far as I can tell, you propose replacing the Monticello metadata with that from git. How do I, as a Squeak user with Monticello, ever get to look at your package again? As I understand it, moving the metadata from Monticello commit time to git means that the metadata is in a format that git determines, not Monticello.

Yes. See below why.

So I don't understand how on the one hand you can say "The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....", which implies you want to eliminate the Monticello metadata, and on the other hand you say you're keeping the Monticello metadata. I'm hopelessly confused. How does the Monticello metadata get reconstituted if it's been thrown away?

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image change P, commit via git to local git repository G load P from G into an image store P to R via Monticello

It's not a scenario I've specifically worked on, but all the tech is implemented / implementable to do that perfectly.

The only thing that is problematic there is that the only safe history is the one generated from git... there are so many MC packages with broken history that, on mcz packages, you have to admit that it's not safe to base things on their history.

I'm sorry but I don't accept that. In the Squeak trunk we have history in our mczs that is correct. Certainly in VMMaker.oscog I have history that goes back a long time. If bugs have broken history then efforts should be made to repair that history. But you can't just write off Monticello history like that.

I don't. You presuppose.

I write tools that work with Monticello repositories, not just yours. I have to do with what is given to me. On a general level, as a mcz user, I'll just have to consider that you are as susceptible to be trusted as with any other mcz producer. This means not much...

... And this is the reason why I am inclined to favor option 3, which records the package version history as it existed at the point it was copied into a git repo. When copied back out from the git universe, create a version history that starts with the original version history and generates a history of the package in git ....

Which is not very difficult to do given how GitFileTree is implemented. And I agree this may well be the way to go. But ...

...

Correct or not, the Monticello version history should be preserved ....

I wonder about that. The property of the Monticello version history is that it has value when you can access the versions listed in it. If you mix repositories like that, unless you maintain a link to the previous mcz repository, pre-git versions can't be accessed. So, most of the time, what we do with a project moved under git is to clone the previous repository, not take just the current head.

(you'll notice, by the way, that vcs usually work that way when moving from, say, CVS to git - you move the entire repository, not just the latest version).

In short, the question would really be: 1- should we invest into making that integration of past history a selling point (but I foresee issues down the road; I've only described one so far, and I have seen others) or 2- into making a better "clone" of a package history, timestamps and everything when moving a complete repository to git?

Honestly, I'd consider 1- to be the easiest to implement. 2- there is already some code floating around. Thierry,

My take on this is that we are not trying to "fix" Monticello. The developers who prefer to use Monticello should continue to use Monticello and the existing Monticello tools should continue to be available. Any new formats should provide a migration path from Monticello and any new tools like a `project list` tool should accommodate projects based on both git and Monticello repositories. This particular branch of the conversation has been aimed at trying to resolve the tension that has been created by including Monticello metadata in FileTree repositories and I think that "option 3": including past history in the FileTree repository, but not updating the version history for each git commit is a workable compromise --- there are always "issues down the road" whether or not we see them and that shouldn't keep us from trying to evolve:) FileTree was invented (props to Otto Behrens) to smoothly integrate the use of disk-based repositories into a Monticello development environment dominated by Monticello tools. Thierry your work building on the original FileTree has gone a long way towards making git a viable development option for Pharo. I think that you'd agree that the next step of evolution for "git-based" development involves improved tool support and I would think that we should turn our focus in that direction... Dale

2016-02-04 14:39 GMT-03:00 Ben Coman <btc@openinworld.com>:

Pertinent is Joel's description of how Microsoft Excel's tipping point [1] in competing against Lotus123 was when it became able to export to Lotu123 format. Making it easy for people to move from git back to Monticello lower the barrier for entry for people to use git.

I always remember this article, but regarding Pharo and Git (or any other file based SCM) I always thought about it the other way, it is... the tipping point could be a seamless/smooth adoption of file based SCMs, with git being the most prominent case. Esteban A. Maringolo

On 2/4/16 9:39 AM, Ben Coman wrote:

On Fri, Feb 5, 2016 at 1:08 AM, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:

...

On 2/3/16 10:53 PM, Thierry Goubier wrote:

...

Le 03/02/2016 23:58, Dale Henrichs a écrit :

...

On 02/03/2016 02:34 PM, Thierry Goubier wrote:

...

Le 03/02/2016 22:51, Eliot Miranda a écrit :

...

On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>> wrote:

Hi Eliot,

Le 02/02/2016 21:54, Eliot Miranda a écrit : ....

No it's /not/ the end of the story. The essential part of the story is how Monticello remains compatible and interoperable between dialects. I haven't seen you account for how you maintain that compatibility. As far as I can tell, you propose replacing the Monticello metadata with that from git. How do I, as a Squeak user with Monticello, ever get to look at your package again? As I understand it, moving the metadata from Monticello commit time to git means that the metadata is in a format that git determines, not Monticello.

Yes. See below why.

So I don't understand how on the one hand you can say "The Monticello metadata in a git repository is redundant and leads to unnecessary commit conflicts -- end of story ....", which implies you want to eliminate the Monticello metadata, and on the other hand you say you're keeping the Monticello metadata. I'm hopelessly confused. How does the Monticello metadata get reconstituted if it's been thrown away?

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image change P, commit via git to local git repository G load P from G into an image store P to R via Monticello

It's not a scenario I've specifically worked on, but all the tech is implemented / implementable to do that perfectly.

The only thing that is problematic there is that the only safe history is the one generated from git... there are so many MC packages with broken history that, on mcz packages, you have to admit that it's not safe to base things on their history.

I'm sorry but I don't accept that. In the Squeak trunk we have history in our mczs that is correct. Certainly in VMMaker.oscog I have history that goes back a long time. If bugs have broken history then efforts should be made to repair that history. But you can't just write off Monticello history like that.

I don't. You presuppose.

I write tools that work with Monticello repositories, not just yours. I have to do with what is given to me. On a general level, as a mcz user, I'll just have to consider that you are as susceptible to be trusted as with any other mcz producer. This means not much...

... And this is the reason why I am inclined to favor option 3, which records the package version history as it existed at the point it was copied into a git repo. When copied back out from the git universe, create a version history that starts with the original version history and generates a history of the package in git ....

Which is not very difficult to do given how GitFileTree is implemented. And I agree this may well be the way to go. But ...

...

Correct or not, the Monticello version history should be preserved ....

I wonder about that. The property of the Monticello version history is that it has value when you can access the versions listed in it. If you mix repositories like that, unless you maintain a link to the previous mcz repository, pre-git versions can't be accessed. So, most of the time, what we do with a project moved under git is to clone the previous repository, not take just the current head.

(you'll notice, by the way, that vcs usually work that way when moving from, say, CVS to git - you move the entire repository, not just the latest version).

In short, the question would really be: 1- should we invest into making that integration of past history a selling point (but I foresee issues down the road; I've only described one so far, and I have seen others) or 2- into making a better "clone" of a package history, timestamps and everything when moving a complete repository to git?

Honestly, I'd consider 1- to be the easiest to implement. 2- there is already some code floating around. Thierry,

My take on this is that we are not trying to "fix" Monticello.

The developers who prefer to use Monticello should continue to use Monticello and the existing Monticello tools should continue to be available. Any new formats should provide a migration path from Monticello and any new tools like a `project list` tool should accommodate projects based on both git and Monticello repositories.

This particular branch of the conversation has been aimed at trying to resolve the tension that has been created by including Monticello metadata in FileTree repositories and I think that "option 3": including past history in the FileTree repository, but not updating the version history for each git commit is a workable compromise --- there are always "issues down the road" whether or not we see them and that shouldn't keep us from trying to evolve:)

FileTree was invented (props to Otto Behrens) to smoothly integrate the use of disk-based repositories into a Monticello development environment dominated by Monticello tools.

Thierry your work building on the original FileTree has gone a long way towards making git a viable development option for Pharo.

I think that you'd agree that the next step of evolution for "git-based" development involves improved tool support and I would think that we should turn our focus in that direction...

Dale

Pertinent is Joel's description of how Microsoft Excel's tipping point [1] in competing against Lotus123 was when it became able to export to Lotu123 format. Making it easy for people to move from git back to Monticello lower the barrier for entry for people to use git.

[1] Scroll down to the second "tipping point" http://www.joelonsoftware.com/articles/fog0000000052.html

That was the original rationale for including the Monticello metadata. Unfortunately the implementation I chose interferes with the natural use of git, so an alternate solution is needed. I think the technology is present to be able to implement "option 3" in relatively short order and we can look forward to this in a future release of FileTree[1]. But we also need to focus on improving the "git" support by the tools so that developers can leverage some of the advantages that "git" brings to the tble... developers need to be able to see version history at the package, class, and method level; they need to be able perform the "standard" git operations (commit, checkout, branch, diff, push, pull ...) without leaving the image; and they need to be able to do git merges in the image .... At least that's my opinion:) Dale [1] https://github.com/dalehenrich/filetree/issues/177

On 1/29/16 2:45 PM, David Allouche wrote:

Thanks Dale for all the explanations.

How Monticello and version control relate in the big picture is starting to make sense for me.

Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem. I agree 100%, but as I've mentioned in another post, I think that switching to a file-per-class exchange format needs to be a cross-dialect effort. A format that accommodates the diversity of Smalltalk class definition and even method metadata while still being readable is a bit of a challenge - solvable, but challenging....

Personally, I have other fish to fry, but I would be willing to participate in such an effort if someone else is willing to lead the charge:) Dale

On 1/30/16 1:54 AM, Bernhard Pieber wrote:

Dale,

Thanks for your thorough answer. I really appreciate how you include links to helpful articles.

I find the description of the workflow you actually use very enlightening. However, one thing still remains unclear. In the last step, when merging the pull request. How is the unchanged metadata reconciled with the code changes? I just realized that I just don’t know what information is in the Monticello metadata, which is not in the code?

Monticello metadata is basically the entire Monticello version history of the package, it includes direct ancestors, commit comments, the GUID, etc. For a FileTree repo, the meta data is stashed a separate file ... The form of the data is actually serialized Smalltalk object graph - a deeply nested set of Arrays - all written on a single line, so git has very little chance of being able to merge two files ... The other bit of Monticello metadata that is maintained by FileTree involves the per method timestamp and developer initials - these are stored in a method properties file associated with each class. In fact is nearly impossible for a human to properly merge to Monticello version histories, which is why Thierry Goubier's GitFileTree-MergeDriver[1] is especially useful. Thierry's MergeDriver fires up a Pharo image (called by git) and performs object surgery to merge the conflicting version histories and method properties. Anyway, in the scenario that I described, I simplified the process a bit and left out some of the gorier details. When I cherry-picked the commit for my topic branch and merged that commit into Sven's latest master branch, the Monticello metadata was certain to conflict, but when you perform a merge on a local git repository and you've registered Thierry's merge driver with git, the Monticello metadata is automatically merged. I also made the assumption that Sven would not have made any additional commits to his master branch between the time I merged his master branch and the time I submitted the pull request - the latest version of a file wins in a merge as long as they share the same immediate previous ancestor. If Sven had made a new commit, the Monticello metadata would have conflicted ... For the record, these "guaranteed conflicts" are the only reason that "removing Monticello metadata" enters into the conversation .... git maintains it's own version history and method properties, so for developers who are "exclusively using git" the Monticello metadata is an annoying source of "meaningless conflicts." All of the information that is present in Monticello meta data is present in git in another form As I said, Thierry Goubier's magical GitFileTree-MergeDriver[1] eliminates the conflict pain when the merges are performed in a local git repository, but on GitHub, merging the Monticello metadata is a constant thorn in one's side, since it becomes necessary to perform a manual merge (and leverage Thierry's Merge Driver) when Monticello metadata is involved. Dale [1] https://github.com/ThierryGoubier/GitFileTree-MergeDriver

On 01/30/2016 08:27 PM, Ben Coman wrote:

On Sun, Jan 31, 2016 at 3:38 AM, Dale Henrichs <dale.henrichs@gemtalksystems.com> wrote:

...

On 1/30/16 1:54 AM, Bernhard Pieber wrote:

...

Dale,

Thanks for your thorough answer. I really appreciate how you include links to helpful articles.

I find the description of the workflow you actually use very enlightening. However, one thing still remains unclear. In the last step, when merging the pull request. How is the unchanged metadata reconciled with the code changes? I just realized that I just don’t know what information is in the Monticello metadata, which is not in the code?

Monticello metadata is basically the entire Monticello version history of the package, it includes direct ancestors, commit comments, the GUID, etc. For a FileTree repo, the meta data is stashed a separate file ... The form of the data is actually serialized Smalltalk object graph - a deeply nested set of Arrays - all written on a single line, so git has very little chance of being able to merge two files

What would it take to have the meta data spread out over multiple lines (if that would work better with git?)

Hmmm I suppose that if the meta data were represented in STON (using pretty print) the changes might be more mergable, but I think that Thierry's algorithm might be still to a proper merge, since it takes "object surgery" to get things right .... STON might make it possible for a human to do the necessary edits though ... Dale

Le 02/02/2016 21:56, Eliot Miranda a écrit :

On Tue, Feb 2, 2016 at 11:38 AM, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> wrote:

On 01/30/2016 08:27 PM, Ben Coman wrote:

On Sun, Jan 31, 2016 at 3:38 AM, Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>> wrote:

On 1/30/16 1:54 AM, Bernhard Pieber wrote:

Dale,

Thanks for your thorough answer. I really appreciate how you include links to helpful articles.

I find the description of the workflow you actually use very enlightening. However, one thing still remains unclear. In the last step, when merging the pull request. How is the unchanged metadata reconciled with the code changes? I just realized that I just don’t know what information is in the Monticello metadata, which is not in the code?

Monticello metadata is basically the entire Monticello version history of the package, it includes direct ancestors, commit comments, the GUID, etc. For a FileTree repo, the meta data is stashed a separate file ... The form of the data is actually serialized Smalltalk object graph - a deeply nested set of Arrays - all written on a single line, so git has very little chance of being able to merge two files

What would it take to have the meta data spread out over multiple lines (if that would work better with git?)

Hmmm I suppose that if the meta data were represented in STON (using pretty print) the changes might be more mergable, but I think that Thierry's algorithm might be still to a proper merge, since it takes "object surgery" to get things right .... STON might make it possible for a human to do the necessary edits though ...

Please tell me this isn't about line endings? Why can't the version history be written with lf line endings? That's hardly a bone of contention is it?

No, it's a lot more complex than that. It's simply hard to do a merge-able text log format. Git also makes a mess of json / ston files, simply because they are structured data in what appears to be text files, git treat them as text files, and three-way line by line merging does not work. The only hope you can have is to make a format which is easier for a human to correct when git creates a conflict when merging ... in short, something which would only make it easier as long as the history is limited to 50 versions and a single developper :( Now, given how much cleaner, safer and sound is extracting that data from the dvcs log, I'll never bother with that sort of things again. Thierry

Hi Dale, then a question: how does one query a baselineOf or a configOf to retrieve the current project state (i.e. packages) loaded in the image? Thierry 2016-01-27 4:11 GMT+01:00 Dale Henrichs <dale.henrichs@gemtalksystems.com>:

I've been wondering no-one in the Pharo community has started building a "Project Browser" to replace the "Package Browser" .... I've had a "project browser" in tODE for several years now and you work with a manageble list of Metacello projects like the following:

[image: project list]

instead of a giant list of packages like the following (only half of the packages loaded in the image):

[image: package list]

Like a package browser, the project shows up as dirty whenever a package in the project is dirtied and all of the operations `save`, `diff`, etc. apply equally well to projects instead of packages .... with the added bonus of performing the action on the the project itself instead of having to view diffs on each dirty package or save each dirty package ... for the git-based repos, a single git commit is performed after all of the dirty packages are committed , thus allowing git to track the moves of methods between packages and combining all of the changes for a unit of work in a single commit ...

Also, in the `project list`, the loaded version and blessing (1.0.5.4 [release]) or SHA of the git commit and branch name (b674ad0 [master]) is displayed:

Gofer 1.0.5.4 [release] Grease b674ad0 [master]

... and this is very useful information. Metacello provides an api for this information and a project-based workflow is just much simpler to work in than a package-based workflow...

I'm willing to share code from tODE and Metacello if anyone is interested ... and I think that a tool like this is really required to properly support a git (or any other disk-based SCM) workflow ...

Perhaps work is already underway, but whenever I see folks talking about the funky Monticello Browser interface, I can't help thinking about the fact that I haven't used an interface like that for several years now and it's time for the tooling to move forward a bit more:)

Dale

On 1/22/16 9:50 AM, Sean P. DeNigris wrote:

Ben Coman wrote

checkbox for displaying non-dirty packages - so we only display the short list in the usual case. (??)

Ah! That would be very helpful. Particularly annoying is when you commit one of your project's dirty packages, the list auto-scrolls to the package's new location among the non-dirty packages, and you have to manually scroll back to the top of the package list to find your next dirty package.

----- Cheers, Sean -- View this message in context: http://forum.world.st/Contributing-to-Pharo-tp4871604p4873422.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Thierry, I should mention that to get a registration for a project in the first place, you call: MetacelloProjectRegistration class>>registrationForClassNamed:ifAbsent:[1] ... and yes you pass in the full class name for the ConfigurationOf or BaselineOf ... Dale [1] https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacel... On 1/27/16 6:41 AM, Dale Henrichs wrote:

On 1/27/16 12:03 AM, Thierry Goubier wrote:

...

Hi Dale,

then a question: how does one query a baselineOf or a configOf to retrieve the current project state (i.e. packages) loaded in the image?

Thierry

Thierry,

Good question:)

To start with in tODE I have a class TDMetacelloProjectRegistrationDefinition which is part of the following hierarchy:

Object TDDefinition TDPackageBasedDefinition TDAbstractProjectRegistrationDefinition TDGitProjectRegistrationDefinition TDMetacelloProjectRegistrationDefinition TDMetacelloRegistrationDefinition TDMetacelloBaselineRegistrationDefinition TDMetacelloConfigurationRegistrationDefinition TDMetacelloHybridRegistrationDefinition TDMetacelloUnloadedProjectRegistrationDefinition

The method TDMetacelloProjectRegistrationDefinition>>workingCopies[1] calculates the list of MCWorkingCopy instances for the loaded packages associated with a project (including the configurationof or baselineof... or both).

The key Metacello methods called include:

MetacelloProjectRegistration>>projectVersion[2] MetacelloMCVersion>>packages[3] MetacelloPackageSpec>>workingCopy[4] MetacelloMCProjectSpec>>projectPackage[5]

This is really the key information for doing all of the other operations for a "Project Browser" like "save all dirty packages and commit" and "give me diffs for all dirty packages in project"...

Dale

[1] https://github.com/dalehenrich/tode/blob/master/repository/Topez-Server-Core... [2] https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacel... [3] https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacel... [4] https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacel... [5] https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacel...

On 01/28/2016 12:50 AM, Thierry Goubier wrote:

Hi Dale,

2016-01-27 15:41 GMT+01:00 Dale Henrichs <dale.henrichs@gemtalksystems.com <mailto:dale.henrichs@gemtalksystems.com>>:

On 1/27/16 12:03 AM, Thierry Goubier wrote:

Hi Dale,

then a question: how does one query a baselineOf or a configOf to retrieve the current project state (i.e. packages) loaded in the image?

Thierry

Thierry,

Good question:)

To start with in tODE I have a class TDMetacelloProjectRegistrationDefinition which is part of the following hierarchy:

Object TDDefinition TDPackageBasedDefinition TDAbstractProjectRegistrationDefinition TDGitProjectRegistrationDefinition TDMetacelloProjectRegistrationDefinition TDMetacelloRegistrationDefinition TDMetacelloBaselineRegistrationDefinition TDMetacelloConfigurationRegistrationDefinition TDMetacelloHybridRegistrationDefinition TDMetacelloUnloadedProjectRegistrationDefinition

The method TDMetacelloProjectRegistrationDefinition>>workingCopies[1] calculates the list of MCWorkingCopy instances for the loaded packages associated with a project (including the configurationof or baselineof... or both).

The key Metacello methods called include:

MetacelloProjectRegistration>>projectVersion[2] MetacelloMCVersion>>packages[3] MetacelloPackageSpec>>workingCopy[4] MetacelloMCProjectSpec>>projectPackage[5]

This is really the key information for doing all of the other operations for a "Project Browser" like "save all dirty packages and commit" and "give me diffs for all dirty packages in project"...

I was just asking that because I'd like to see that information merged as much as possible; separating concepts just strikes me as unneeded complexity, at least for baselines. Not sure what you mean by "separating concepts and unneeded complexity". I provided a picture of some of the abstractions that I am using in tODE ... these abstractions are used in a number of commands, but I also tried to provide you with the basic implementation used in tODE in case you were interested in the fact that tODE includes the BaselineOf package in the list of packages for a project...

The simple answer is MetacelloMCVersion>>packages[3]

Configurations are a bit different, since a configuration can snapshot multiple versions. But I'd really like to make it simple and obvious how to setup a project. I'm not sure what you are talking about here.

In Metacello a BaselineOf has a single version and the MetacelloMCVersion class manages the version ... Yes, a Configuration has multiple versions and each of those versions is an instance of MetacelloMCVersion.... Perhaps when you have time, you can provide a bit more information about what you are trying to do ... I apologize if I provided too much information but when working with folks in different time zones, I am in the habit of packing as much information in a message as possible, because it's often a 24 hour turnaround for responses... (as it is in this case:) Dale

On 01/28/2016 12:16 PM, Thierry Goubier wrote:

Le 28/01/2016 20:55, Dale Henrichs a écrit :

...

On 01/28/2016 12:50 AM, Thierry Goubier wrote:

...

Configurations are a bit different, since a configuration can snapshot multiple versions. But I'd really like to make it simple and obvious how to setup a project. I'm not sure what you are talking about here.

In Metacello a BaselineOf has a single version and the MetacelloMCVersion class manages the version ... Yes, a Configuration has multiple versions and each of those versions is an instance of MetacelloMCVersion....

Perhaps when you have time, you can provide a bit more information about what you are trying to do ...

I was just looking at it from the angle of having to explain it to one of my students (non-smalltalkers)... The simplest path to creating a project.

I guess one must have a tool like Versionner. From this perspective you should look at the MetacelloToolBox and specifically MetacelloToolBox class>>createBaselineOf:requiredProjects:packages:repositories:dependencies:groups:.

Versionner uses the MetacelloToolBox api and about 3 months ago I added the basic support for creating a BaselineOf as I added a `project new` command to tODE ... I would be glad to expand the MetacelloToolBox api for BaselineOf if folks ask for specific functionality ... The MetacelloToolBox is meant to be the common point for programmatically creating and updating ConfigurationOf and BaselineOf ... Dale