February 2016 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Contributing to Pharo

by Sven Van Caekenberghe

Thanks Thierry for pushing this subject and working so patiently with the community. > On 03 Feb 2016, at 10:18, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > I went through all the different possible file formats, class-based, package-based, method-based, log metadata and the like, and I concluded that: > > - the method based format is as good as any other. Even better since it has a spec (cypress). > - method based format allow for method-history queries on the git/vcs history (as well as class based / package based queries). > - the tree structure on github or bitbucket is quite convenient (and browsable) to the point one could edit a package directly in it (I do when I need to do a quick fix). > - anything that can compress a bit the metadata version is probably good to consider. version files can be huge. > - merge drivers really work and releave us from conflict resolution > - we need a merge tool written in Smalltalk/MC > - MC version numbering is a very bad idea > - MC almost never using properly UUIDs is a very bad behavior > - MC packages history can be considered broken in the general case > - it takes time to define, implement, test and really use a new format and tooling Excellent summary. I also feel that the current structure (basic filetree) is more than OK. You can /almost/ browse through github. I am sure we'll get there with the meta data and tooling. Sven

Feb. 3, 2016

Re: [Pharo-dev] Monkey: "Check seems to be stuck. I will try it again"

by Denis Kudriashov

I increased timeout for two hours 2016-02-03 10:10 GMT+01:00 Denis Kudriashov <dionisiydk(a)gmail.com>: > We added new job to resume hanging status "Monkey is checking". > Maybe we should increase timeout. Now when Monkey checks issue more than 1 > hour we revert it status to "Fix Review Needed" > > 2016-02-03 4:24 GMT+01:00 Martin Dias <tinchodias(a)gmail.com>: > >> Hi, >> >> That's Monkey's result in my case ( >> https://pharo.fogbugz.com/f/cases/14605/Integrate-Epicea) I don't see >> other cases with the same result. Don't know where to look up to understand >> it... Any pointer? >> >> Thanks >> Martin >> > >

Feb. 3, 2016

Re: [Pharo-dev] Contributing to Pharo

by Thierry Goubier

Hi Ben, Le 03/02/2016 06:00, Ben Coman a Ã©crit :> > I'm curious how the merge driver is implemented. I think it was > mentioned that git calls-back to Pharo to do the processing. Is this > it something like this... > > * How to make Git preserve specific files while merging > https://medium.com/@porteneuve/how-to-make-git-preserve-specific-files-whil… Probably, haven't read it. > * A few of my Git tricks, tips and workflows > section under gitattribute(5) > http://nuclearsquid.com/writings/git-tricks-tips-workflows/ Yes. You use git attributes to trigger the use of a pharo-implemented command line tool which does the merge for git. > Or... rather than the metadata being a specific file (e.g. > 'metadata'), the file could change for each commit (e.g. > 'metadata.nnn') where nnn is the revision number, so each commit > deletes the old metadata file and writes the new metadata file. > Monticello could know to use whichever single 'metadata.*' file > exists. Would that avoid the problem of metadata merge conflict? I've thought of that, and it seems hard to get right (which file goes first? how do you recognize a merge?). You have to remember that packages have hundreds, even thousands of versions so this would mean that number of files. I went through all the different possible file formats, class-based, package-based, method-based, log metadata and the like, and I concluded that: - the method based format is as good as any other. Even better since it has a spec (cypress). - method based format allow for method-history queries on the git/vcs history (as well as class based / package based queries). - the tree structure on github or bitbucket is quite convenient (and browsable) to the point one could edit a package directly in it (I do when I need to do a quick fix). - anything that can compress a bit the metadata version is probably good to consider. version files can be huge. - merge drivers really work and releave us from conflict resolution - we need a merge tool written in Smalltalk/MC - MC version numbering is a very bad idea - MC almost never using properly UUIDs is a very bad behavior - MC packages history can be considered broken in the general case - it takes time to define, implement, test and really use a new format and tooling Thierry

Feb. 3, 2016

Re: [Pharo-dev] Contributing to Pharo

by Thierry Goubier

Hi Ben, Le 03/02/2016 06:00, Ben Coman a Ã©crit :> > I'm curious how the merge driver is implemented. I think it was > mentioned that git calls-back to Pharo to do the processing. Is this > it something like this... > > * How to make Git preserve specific files while merging > https://medium.com/@porteneuve/how-to-make-git-preserve-specific-files-whil… Probably, haven't read it. > * A few of my Git tricks, tips and workflows > section under gitattribute(5) > http://nuclearsquid.com/writings/git-tricks-tips-workflows/ Yes. You use git attributes to trigger the use of a pharo-implemented command line tool which does the merge for git. > Or... rather than the metadata being a specific file (e.g. > 'metadata'), the file could change for each commit (e.g. > 'metadata.nnn') where nnn is the revision number, so each commit > deletes the old metadata file and writes the new metadata file. > Monticello could know to use whichever single 'metadata.*' file > exists. Would that avoid the problem of metadata merge conflict? I've thought of that, and it seems hard to get right (which file goes first? how do you recognize a merge?). You have to remember that packages have hundreds, even thousands of versions so this would mean that number of files. I went through all the different possible file formats, class-based, package-based, method-based, log metadata and the like, and I concluded that: - the method based format is as good as any other. - method based format allow for method-history queries on the git/vcs history (as well as class based / package based queries). - the tree structure on github or bitbucket is quite convenient (and browsable) to the point one could edit a package directly in it (I do when I need to do a quick fix). - anything that can compress a bit the metadata version is probably good to consider. version files can be huge. - merge drivers really work and releave us from conflict resolution - we need a merge tool written in Smalltalk/MC - MC version numbering is a very bad idea - MC almost never using properly UUIDs is a very bad behavior - MC packages history can be considered broken in the general case - it takes time to define, implement, test and really use a new format and tooling Thierry

Feb. 3, 2016

Re: [Pharo-dev] Monkey: "Check seems to be stuck. I will try it again"

by Denis Kudriashov

We added new job to resume hanging status "Monkey is checking". Maybe we should increase timeout. Now when Monkey checks issue more than 1 hour we revert it status to "Fix Review Needed" 2016-02-03 4:24 GMT+01:00 Martin Dias <tinchodias(a)gmail.com>: > Hi, > > That's Monkey's result in my case ( > https://pharo.fogbugz.com/f/cases/14605/Integrate-Epicea) I don't see > other cases with the same result. Don't know where to look up to understand > it... Any pointer? > > Thanks > Martin >

Feb. 3, 2016

Re: [Pharo-dev] Contributing to Pharo

by Thierry Goubier

Hi Eliot, Le 02/02/2016 21:54, Eliot Miranda a Ã©crit : .... > > No it's /not/ the end of the story. The essential part of the story is > how Monticello remains compatible and interoperable between dialects. I > haven't seen you account for how you maintain that compatibility. As > far as I can tell, you propose replacing the Monticello metadata with > that from git. How do I, as a Squeak user with Monticello, ever get to > look at your package again? As I understand it, moving the metadata > from Monticello commit time to git means that the metadata is in a > format that git determines, not Monticello. Yes. See below why. > So I don't understand how on the one hand you can say "The Monticello > metadata in a git repository is redundant and leads to unnecessary > commit conflicts -- end of story ....", which implies you want to > eliminate the Monticello metadata, and on the other hand you say you're > keeping the Monticello metadata. I'm hopelessly confused. How does the > Monticello metadata get reconstituted if it's been thrown away? > > What happens to the metadata in the following workflow? > > load package P from Monticello repository R into an image > change P, commit via git to local git repository G > load P from G into an image > store P to R via Monticello It's not a scenario I've specifically worked on, but all the tech is implemented / implementable to do that perfectly. The only thing that is problematic there is that the only safe history is the one generated from git... there are so many MC packages with broken history that, on mcz packages, you have to admit that it's not safe to base things on their history. > But what does this imply to some package that starts off in a Monticello > repository and then spends some time in gitland? Can I merge again? If > I can I'm happy. If I can't, I feel sabotaged. You could. Just express your needs and wait until one of us has enough free time to solve it for you, that is. > Fine. Except merging is, IIUC, about method time stamps and ancestry. > If that gets preserved then I'm happy. But for the life of me I haven't > read an explanation that reassures me that these are being preserved. > Do you see the roots of my fear? Of course. But that also project a bit what you think of the people working on it... which may makes it a bit hard to answer. Thierry

Feb. 3, 2016

Re: [Pharo-dev] Contributing to Pharo

by Nicolas Cellier

https://github.com/ThierryGoubier/GitFileTree-MergeDriver 2016-02-03 6:00 GMT+01:00 Ben Coman <btc(a)openinworld.com>: > On Wed, Feb 3, 2016 at 7:00 AM, Dale Henrichs > <dale.henrichs(a)gemtalksystems.com> wrote: > > > > > > On 02/02/2016 12:54 PM, Eliot Miranda wrote: > > > > Hi Dale, > > > > On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs > > <dale.henrichs(a)gemtalksystems.com> wrote: > >> > >> > >> > >> On 01/29/2016 05:17 PM, Eliot Miranda wrote: > >>> > >>> Hi David, > >>> > >>>> On Jan 29, 2016, at 2:45 PM, David Allouche <david(a)allouche.net> > wrote: > >>>> > >>>> Thanks Dale for all the explanations. > >>>> > >>>> How Monticello and version control relate in the big picture is > starting > >>>> to make sense for me. > >>>> > >>>> Now, I better understand why filetree ended up uses a file-per-method > >>>> format, even though that is relatively hostile to git user interfaces > >>>> optimised for other languages. There is really a need for a > file-per-class > >>>> exchange format, because that would works a lot better with the > existing VCS > >>>> ecosystem. > >>> > >>> I agree so strongly. Class file outs which are eg sorted by selector > >>> make much more sense. They won't hit the file name length limit. > They make > >>> it trivial to maintain method and class comment time stamps. They're > easier > >>> to construct into snapshots because it's easier to decode the file > name. > >>> > >>> And then it's easy to add files for package load/unload scripts and for > >>> the history. And then one is much more decoupled from the specific > back > >>> end. It could be mercurial just as easily as git. > >>> > >>>> I think more package-based user interfaces would indeed be a very good > >>>> idea, for browsing and for source code management. > >>>> > >>>> Stef, I have the impression you think that git is popular because it > is > >>>> a new shiny toy. I disagree with this idea. Git is a typical > worse-is-better > >>>> tool. It's good enough for most people, but it still has many > shortcomings. > >>>> It is popular in spite of its shortcomings. It became popular as > destination > >>>> for projects shifting from CVS and Subversion. So it is unlikely to be > >>>> displaced by a newer, incrementally shinier tools. Anything that will > >>>> displace it will have to provide an improvement of a similar > magnitude as > >>>> the jump between centralised and distributed version control. > >>> > >>> This is a good analysis. What's valuable to the Pharo community is not > >>> displacing an already functional dvcs (Monticello) with an ill-suited > one > >>> (git), but in being able to function in ecosystems like github where > people > >>> can display their identity and where infrastructure for bug reports etc > >>> exist. > >>> > >>>> Still, I think it's a good idea not to restrict high level models to > >>>> what git provides if that's a less than ideal fit to the image model. > >>> > >>> Absolutely. Dale's talk of ditching Monticello metadata fills me with > >>> repulsion and makes me want to ask is he trying to sabotage or what? > >> > >> Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how > >> about you assume that I'm doing "or what":) > >> > >> The Monticello metadata in a git repository is redundant and leads to > >> unnecessary commit conflicts -- end of story .... > > > > > > No it's /not/ the end of the story. The essential part of the story is > how > > Monticello remains compatible and interoperable between dialects. I > haven't > > seen you account for how you maintain that compatibility. As far as I > can > > tell, you propose replacing the Monticello metadata with that from git. > How > > do I, as a Squeak user with Monticello, ever get to look at your package > > again? As I understand it, moving the metadata from Monticello commit > time > > to git means that the metadata is in a format that git determines, not > > Monticello. > > > > Good question, FileTree has been supported on Squeak since the very > > beginning (I along with a small number of Squeak users have made sure of > > that). > > > > So TODAY, any Squeak user can "look at, load and commit" any package that > > has been written using FileTree (with or without Monticello meta data). > > > > [1] https://github.com/dalehenrich/filetree/tree/squeak4.3#squeak > > > > > > So I don't understand how on the one hand you can say "The Monticello > > metadata in a git repository is redundant and leads to unnecessary commit > > conflicts -- end of story ....", which implies you want to eliminate the > > Monticello metadata, and on the other hand you say you're keeping the > > Monticello metadata. I'm hopelessly confused. How does the Monticello > > metadata get reconstituted if it's been thrown away? > > > > Monticello meta data is not an integral part of the "package-ness" of a > > Monticello package ... it _is_ integral to the "repository-ness" of a > > Monticello package ... > > > > If the Monticello metadata is "thrown away" then the revision history for > > Monticello is lost, but for a package that is "born" in a git repository, > > the Monticello metadata is not needed. Git has it's own commit meta data > and > > the Monticello metadata is redundant. > > > > If you want to see the revision history from a git-based FileTree repo, > > then: > > > > 1. one can include the Monticello meta data as part of the package - > this > > is what FileTree > > currently does > > 2. one can build a tool that reconstructs the Monticello version > history > > from the git metadata > > making it possible to use "old" Monitcello tools to look at the git > > repo - I believe that Thierry's > > GitFileTree takes this approach for metadata-less repositories > > 3. One can build a new tool that presents the git metadata without > > reconstructing the > > Monticello metadata at all. Note that by "embracing git", it is > > possible to present revision > > history at the package level (current mcz techology) as well as the > > class and method level - > > which is what I do in tODE > > > > > > What happens to the metadata in the following workflow? > > > > load package P from Monticello repository R into an image > > change P, commit via git to local git repository G > > load P from G into an image > > store P to R via Monticello > > > > > > The above workflow can be accomplished whether or not Monticello > metadata is > > present, however, if one does not make an effort to preserve the revision > > history then at the end of your workflow the Monticello metadata is lost. > > > > If one takes the pains to preserve the Monticello metadata before > committing > > to the git repository and the metadata is updated with each commit during > > the git lifetime, then the full metadata will be present at the end ... > > > > This I think is the crux of the discussion. > > > > There are a number of alternate schemes that can be used to preserve the > > metadata through this scenario: > > > > 1. (current FileTree implementation) store the Monticello meta data in > git > > and update on > > every git commit > > 2. duplicate the existing Monticello revision history by committing in > > order all of the > > package ancestors and arrange for a way to reconstruct Monticello > > metadata from git > > meta data > > 3. (variant of 1) store the original monticello meta data in a file, do > > not update on every commit > > but arrange for a way to reconstruct Monticello metadata from git > meta > > data and graft onto > > original Monticello meta data for use in mcz repository --- on > demand > > 4. ???? > > > > Option 3 seems to be a good compromise solution and perhaps is the > approach > > that should be adopted moving forward ... we get to preserver Monticello > > metadata while avoiding the messy commit conflicts for git while > providing a > > (somewhat) seamless path for a package to migrate back into the mcz > > repository world ... if we somehow incorporate the SHA of the commit and > the > > github/bitbucket url into the revision history, then it would be > possible to > > perform a 3 way merge involving two mcz package versions and common > ancestor > > that is only present in git ... > > > > > > > >> Despite the fact that the Monticello metadata is redundant, I have made > >> sure that the Monticello metadata was included in FileTree from the very > >> beginning for the very reason that I wanted developers to be able to > try out > >> FileTree, git and github without having to burn any Monticello bridges > .... > >> if they didn't like FileTree, git and github, then they would be able to > >> back out of their use of git without losing data ... > > > > > > Then forgive me. I couldn't see the wood for the trees. When I read > your > > talk of eliminating the conflicts from git commits due to the Monticello > > metadata I infer that you're eliminating the Monticello metadata. I'm > not > > sure I understand the implications of this, but it seems to me that a > > natural consequence is that the Montivcello metadata is lost and at that > > point merges and the like become problematic. What am I missing? > > > > You've hit the nail on the head, but I think that option 3 above gives > us a > > way to avoid losing the Monticello metadata without incurring a hit for a > > packages lifetime while in git ... > > > > A package that starts its life in git will have an empty Monticello > metadata > > and a package that never makes it's way into an mcz repository will not > > incur per commit penalties ... > > > > > > > >> > >> > >>> It seems entirely destructive. > >> > >> It is not destructive ... > >>> > >>> We have a functional package manager which currently supports > interchange > >>> between Pharo, Squeak and Cuis, > >> > >> and GemStone? > >> > >> I assume that you are talking about Monticello packages and Monticello > >> repositories ... or what? > > > > > > Yes. > > > >> > >> > >> I am really not trying to do anything but "invent the future" --- I am > not > >> trying to destroy, I am trying to improve ... If you are not able to > see the > >> shortcomings of Monticello repositories (Note that I am distinguishing > >> between Monticello packages and Monticello repositories --- FileTree > uses > >> Monticello packages and replaces Monticello repositories with git) and > where > >> git has advantages over Monticello repositories, then you should > continue to > >> use Monticello repositories ... > > > > > > But what does this imply to some package that starts off in a Monticello > > repository and then spends some time in gitland? Can I merge again? If > I > > can I'm happy. If I can't, I feel sabotaged. > > > > and sabotage was never my intent ... > > > > > >> > >> Personally I don't see Monticello repositories going away anytime soon > and > >> expect to support Monticello repositories in GsDevKit_home, tODE, and > >> Metacello for the rest of my life:) > > > > > > Fine. Except merging is, IIUC, about method time stamps and ancestry. > If > > that gets preserved then I'm happy. But for the life of me I haven't > read > > an explanation that reassures me that these are being preserved. Do you > see > > the roots of my fear? > > > > Haha, from the very beginning back in 2012, I understood that there > would be > > fear and resistance to change and anger and joy and excitement but it was > > not clear when if ever we'd reach point where a resolution for this > > "problem" was needed: either no one would be interested or everyone > would be > > interested or ??? > > > > I really think that option 3 is going to be the best compromise moving > > forward - there is some implementation work that will be required but I > > really think option 3 gives you (and frankly me) a way to preserve the > > Monticello revision history for packages that make their way back and > forth > > between lifetimes in git and Monticello repositories. > > > > Eliot, I appreciate the fact that you demanded a better solution! > > > > Dale > > I'm curious how the merge driver is implemented. I think it was > mentioned that git calls-back to Pharo to do the processing. Is this > it something like this... > > * How to make Git preserve specific files while merging > > https://medium.com/@porteneuve/how-to-make-git-preserve-specific-files-whil… > > * A few of my Git tricks, tips and workflows > section under gitattribute(5) > http://nuclearsquid.com/writings/git-tricks-tips-workflows/ > > > Or... rather than the metadata being a specific file (e.g. > 'metadata'), the file could change for each commit (e.g. > 'metadata.nnn') where nnn is the revision number, so each commit > deletes the old metadata file and writes the new metadata file. > Monticello could know to use whichever single 'metadata.*' file > exists. Would that avoid the problem of metadata merge conflict? > > cheers -ben > >

Feb. 3, 2016

Re: [Pharo-dev] Contributing to Pharo

by Ben Coman

On Wed, Feb 3, 2016 at 7:00 AM, Dale Henrichs <dale.henrichs(a)gemtalksystems.com> wrote: > > > On 02/02/2016 12:54 PM, Eliot Miranda wrote: > > Hi Dale, > > On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs > <dale.henrichs(a)gemtalksystems.com> wrote: >> >> >> >> On 01/29/2016 05:17 PM, Eliot Miranda wrote: >>> >>> Hi David, >>> >>>> On Jan 29, 2016, at 2:45 PM, David Allouche <david(a)allouche.net> wrote: >>>> >>>> Thanks Dale for all the explanations. >>>> >>>> How Monticello and version control relate in the big picture is starting >>>> to make sense for me. >>>> >>>> Now, I better understand why filetree ended up uses a file-per-method >>>> format, even though that is relatively hostile to git user interfaces >>>> optimised for other languages. There is really a need for a file-per-class >>>> exchange format, because that would works a lot better with the existing VCS >>>> ecosystem. >>> >>> I agree so strongly. Class file outs which are eg sorted by selector >>> make much more sense. They won't hit the file name length limit. They make >>> it trivial to maintain method and class comment time stamps. They're easier >>> to construct into snapshots because it's easier to decode the file name. >>> >>> And then it's easy to add files for package load/unload scripts and for >>> the history. And then one is much more decoupled from the specific back >>> end. It could be mercurial just as easily as git. >>> >>>> I think more package-based user interfaces would indeed be a very good >>>> idea, for browsing and for source code management. >>>> >>>> Stef, I have the impression you think that git is popular because it is >>>> a new shiny toy. I disagree with this idea. Git is a typical worse-is-better >>>> tool. It's good enough for most people, but it still has many shortcomings. >>>> It is popular in spite of its shortcomings. It became popular as destination >>>> for projects shifting from CVS and Subversion. So it is unlikely to be >>>> displaced by a newer, incrementally shinier tools. Anything that will >>>> displace it will have to provide an improvement of a similar magnitude as >>>> the jump between centralised and distributed version control. >>> >>> This is a good analysis. What's valuable to the Pharo community is not >>> displacing an already functional dvcs (Monticello) with an ill-suited one >>> (git), but in being able to function in ecosystems like github where people >>> can display their identity and where infrastructure for bug reports etc >>> exist. >>> >>>> Still, I think it's a good idea not to restrict high level models to >>>> what git provides if that's a less than ideal fit to the image model. >>> >>> Absolutely. Dale's talk of ditching Monticello metadata fills me with >>> repulsion and makes me want to ask is he trying to sabotage or what? >> >> Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how >> about you assume that I'm doing "or what":) >> >> The Monticello metadata in a git repository is redundant and leads to >> unnecessary commit conflicts -- end of story .... > > > No it's /not/ the end of the story. The essential part of the story is how > Monticello remains compatible and interoperable between dialects. I haven't > seen you account for how you maintain that compatibility. As far as I can > tell, you propose replacing the Monticello metadata with that from git. How > do I, as a Squeak user with Monticello, ever get to look at your package > again? As I understand it, moving the metadata from Monticello commit time > to git means that the metadata is in a format that git determines, not > Monticello. > > Good question, FileTree has been supported on Squeak since the very > beginning (I along with a small number of Squeak users have made sure of > that). > > So TODAY, any Squeak user can "look at, load and commit" any package that > has been written using FileTree (with or without Monticello meta data). > > [1] https://github.com/dalehenrich/filetree/tree/squeak4.3#squeak > > > So I don't understand how on the one hand you can say "The Monticello > metadata in a git repository is redundant and leads to unnecessary commit > conflicts -- end of story ....", which implies you want to eliminate the > Monticello metadata, and on the other hand you say you're keeping the > Monticello metadata. I'm hopelessly confused. How does the Monticello > metadata get reconstituted if it's been thrown away? > > Monticello meta data is not an integral part of the "package-ness" of a > Monticello package ... it _is_ integral to the "repository-ness" of a > Monticello package ... > > If the Monticello metadata is "thrown away" then the revision history for > Monticello is lost, but for a package that is "born" in a git repository, > the Monticello metadata is not needed. Git has it's own commit meta data and > the Monticello metadata is redundant. > > If you want to see the revision history from a git-based FileTree repo, > then: > > 1. one can include the Monticello meta data as part of the package - this > is what FileTree > currently does > 2. one can build a tool that reconstructs the Monticello version history > from the git metadata > making it possible to use "old" Monitcello tools to look at the git > repo - I believe that Thierry's > GitFileTree takes this approach for metadata-less repositories > 3. One can build a new tool that presents the git metadata without > reconstructing the > Monticello metadata at all. Note that by "embracing git", it is > possible to present revision > history at the package level (current mcz techology) as well as the > class and method level - > which is what I do in tODE > > > What happens to the metadata in the following workflow? > > load package P from Monticello repository R into an image > change P, commit via git to local git repository G > load P from G into an image > store P to R via Monticello > > > The above workflow can be accomplished whether or not Monticello metadata is > present, however, if one does not make an effort to preserve the revision > history then at the end of your workflow the Monticello metadata is lost. > > If one takes the pains to preserve the Monticello metadata before committing > to the git repository and the metadata is updated with each commit during > the git lifetime, then the full metadata will be present at the end ... > > This I think is the crux of the discussion. > > There are a number of alternate schemes that can be used to preserve the > metadata through this scenario: > > 1. (current FileTree implementation) store the Monticello meta data in git > and update on > every git commit > 2. duplicate the existing Monticello revision history by committing in > order all of the > package ancestors and arrange for a way to reconstruct Monticello > metadata from git > meta data > 3. (variant of 1) store the original monticello meta data in a file, do > not update on every commit > but arrange for a way to reconstruct Monticello metadata from git meta > data and graft onto > original Monticello meta data for use in mcz repository --- on demand > 4. ???? > > Option 3 seems to be a good compromise solution and perhaps is the approach > that should be adopted moving forward ... we get to preserver Monticello > metadata while avoiding the messy commit conflicts for git while providing a > (somewhat) seamless path for a package to migrate back into the mcz > repository world ... if we somehow incorporate the SHA of the commit and the > github/bitbucket url into the revision history, then it would be possible to > perform a 3 way merge involving two mcz package versions and common ancestor > that is only present in git ... > > > >> Despite the fact that the Monticello metadata is redundant, I have made >> sure that the Monticello metadata was included in FileTree from the very >> beginning for the very reason that I wanted developers to be able to try out >> FileTree, git and github without having to burn any Monticello bridges .... >> if they didn't like FileTree, git and github, then they would be able to >> back out of their use of git without losing data ... > > > Then forgive me. I couldn't see the wood for the trees. When I read your > talk of eliminating the conflicts from git commits due to the Monticello > metadata I infer that you're eliminating the Monticello metadata. I'm not > sure I understand the implications of this, but it seems to me that a > natural consequence is that the Montivcello metadata is lost and at that > point merges and the like become problematic. What am I missing? > > You've hit the nail on the head, but I think that option 3 above gives us a > way to avoid losing the Monticello metadata without incurring a hit for a > packages lifetime while in git ... > > A package that starts its life in git will have an empty Monticello metadata > and a package that never makes it's way into an mcz repository will not > incur per commit penalties ... > > > >> >> >>> It seems entirely destructive. >> >> It is not destructive ... >>> >>> We have a functional package manager which currently supports interchange >>> between Pharo, Squeak and Cuis, >> >> and GemStone? >> >> I assume that you are talking about Monticello packages and Monticello >> repositories ... or what? > > > Yes. > >> >> >> I am really not trying to do anything but "invent the future" --- I am not >> trying to destroy, I am trying to improve ... If you are not able to see the >> shortcomings of Monticello repositories (Note that I am distinguishing >> between Monticello packages and Monticello repositories --- FileTree uses >> Monticello packages and replaces Monticello repositories with git) and where >> git has advantages over Monticello repositories, then you should continue to >> use Monticello repositories ... > > > But what does this imply to some package that starts off in a Monticello > repository and then spends some time in gitland? Can I merge again? If I > can I'm happy. If I can't, I feel sabotaged. > > and sabotage was never my intent ... > > >> >> Personally I don't see Monticello repositories going away anytime soon and >> expect to support Monticello repositories in GsDevKit_home, tODE, and >> Metacello for the rest of my life:) > > > Fine. Except merging is, IIUC, about method time stamps and ancestry. If > that gets preserved then I'm happy. But for the life of me I haven't read > an explanation that reassures me that these are being preserved. Do you see > the roots of my fear? > > Haha, from the very beginning back in 2012, I understood that there would be > fear and resistance to change and anger and joy and excitement but it was > not clear when if ever we'd reach point where a resolution for this > "problem" was needed: either no one would be interested or everyone would be > interested or ??? > > I really think that option 3 is going to be the best compromise moving > forward - there is some implementation work that will be required but I > really think option 3 gives you (and frankly me) a way to preserve the > Monticello revision history for packages that make their way back and forth > between lifetimes in git and Monticello repositories. > > Eliot, I appreciate the fact that you demanded a better solution! > > Dale I'm curious how the merge driver is implemented. I think it was mentioned that git calls-back to Pharo to do the processing. Is this it something like this... * How to make Git preserve specific files while merging https://medium.com/@porteneuve/how-to-make-git-preserve-specific-files-whil… * A few of my Git tricks, tips and workflows section under gitattribute(5) http://nuclearsquid.com/writings/git-tricks-tips-workflows/ Or... rather than the metadata being a specific file (e.g. 'metadata'), the file could change for each commit (e.g. 'metadata.nnn') where nnn is the revision number, so each commit deletes the old metadata file and writes the new metadata file. Monticello could know to use whichever single 'metadata.*' file exists. Would that avoid the problem of metadata merge conflict? cheers -ben

Feb. 3, 2016

Monkey: "Check seems to be stuck. I will try it again"

by Martin Dias

Hi, That's Monkey's result in my case ( https://pharo.fogbugz.com/f/cases/14605/Integrate-Epicea) I don't see other cases with the same result. Don't know where to look up to understand it... Any pointer? Thanks Martin

Feb. 3, 2016

Re: [Pharo-dev] Contributing to Pharo

by Dale Henrichs

On 02/02/2016 12:54 PM, Eliot Miranda wrote: > Hi Dale, > > On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs > <dale.henrichs(a)gemtalksystems.com > <mailto:dale.henrichs@gemtalksystems.com>> wrote: > > > > On 01/29/2016 05:17 PM, Eliot Miranda wrote: > > Hi David, > > On Jan 29, 2016, at 2:45 PM, David Allouche > <david(a)allouche.net <mailto:david@allouche.net>> wrote: > > Thanks Dale for all the explanations. > > How Monticello and version control relate in the big > picture is starting to make sense for me. > > Now, I better understand why filetree ended up uses a > file-per-method format, even though that is relatively > hostile to git user interfaces optimised for other > languages. There is really a need for a file-per-class > exchange format, because that would works a lot better > with the existing VCS ecosystem. > > I agree so strongly. Class file outs which are eg sorted by > selector make much more sense. They won't hit the file name > length limit. They make it trivial to maintain method and > class comment time stamps. They're easier to construct into > snapshots because it's easier to decode the file name. > > And then it's easy to add files for package load/unload > scripts and for the history. And then one is much more > decoupled from the specific back end. It could be mercurial > just as easily as git. > > I think more package-based user interfaces would indeed be > a very good idea, for browsing and for source code management. > > Stef, I have the impression you think that git is popular > because it is a new shiny toy. I disagree with this idea. > Git is a typical worse-is-better tool. It's good enough > for most people, but it still has many shortcomings. It is > popular in spite of its shortcomings. It became popular as > destination for projects shifting from CVS and Subversion. > So it is unlikely to be displaced by a newer, > incrementally shinier tools. Anything that will displace > it will have to provide an improvement of a similar > magnitude as the jump between centralised and distributed > version control. > > This is a good analysis. What's valuable to the Pharo > community is not displacing an already functional dvcs > (Monticello) with an ill-suited one (git), but in being able > to function in ecosystems like github where people can display > their identity and where infrastructure for bug reports etc exist. > > Still, I think it's a good idea not to restrict high level > models to what git provides if that's a less than ideal > fit to the image model. > > Absolutely. Dale's talk of ditching Monticello metadata fills > me with repulsion and makes me want to ask is he trying to > sabotage or what? > > Eliot, you can't be serious - accusing me of sabotage? Ah, > well.... how about you assume that I'm doing "or what":) > > The Monticello metadata in a git repository is redundant and leads > to unnecessary commit conflicts -- end of story .... > > > No it's /not/ the end of the story. The essential part of the story > is how Monticello remains compatible and interoperable between > dialects. I haven't seen you account for how you maintain that > compatibility. As far as I can tell, you propose replacing the > Monticello metadata with that from git. How do I, as a Squeak user > with Monticello, ever get to look at your package again? As I > understand it, moving the metadata from Monticello commit time to git > means that the metadata is in a format that git determines, not > Monticello. Good question, FileTree has been supported on Squeak since the very beginning (I along with a small number of Squeak users have made sure of that). So TODAY, any Squeak user can "look at, load and commit" any package that has been written using FileTree (with or without Monticello meta data). [1] https://github.com/dalehenrich/filetree/tree/squeak4.3#squeak > > So I don't understand how on the one hand you can say "The Monticello > metadata in a git repository is redundant and leads to unnecessary > commit conflicts -- end of story ....", which implies you want to > eliminate the Monticello metadata, and on the other hand you say > you're keeping the Monticello metadata. I'm hopelessly confused. How > does the Monticello metadata get reconstituted if it's been thrown away? Monticello meta data is not an integral part of the "package-ness" of a Monticello package ... it _is_ integral to the "repository-ness" of a Monticello package ... If the Monticello metadata is "thrown away" then the revision history for Monticello is lost, but for a package that is "born" in a git repository, the Monticello metadata is not needed. Git has it's own commit meta data and the Monticello metadata is redundant. If you want to see the revision history from a git-based FileTree repo, then: 1. one can include the Monticello meta data as part of the package - this is what FileTree currently does 2. one can build a tool that reconstructs the Monticello version history from the git metadata making it possible to use "old" Monitcello tools to look at the git repo - I believe that Thierry's GitFileTree takes this approach for metadata-less repositories 3. One can build a new tool that presents the git metadata without reconstructing the Monticello metadata at all. Note that by "embracing git", it is possible to present revision history at the package level (current mcz techology) as well as the class and method level - which is what I do in tODE > > What happens to the metadata in the following workflow? > > load package P from Monticello repository R into an image > change P, commit via git to local git repository G > load P from G into an image > store P to R via Monticello The above workflow can be accomplished whether or not Monticello metadata is present, however, if one does not make an effort to preserve the revision history then at the end of your workflow the Monticello metadata is lost. If one takes the pains to preserve the Monticello metadata before committing to the git repository and the metadata is updated with each commit during the git lifetime, then the full metadata will be present at the end ... This I think is the crux of the discussion. There are a number of alternate schemes that can be used to preserve the metadata through this scenario: 1. (current FileTree implementation) store the Monticello meta data in git and update on every git commit 2. duplicate the existing Monticello revision history by committing in order all of the package ancestors and arrange for a way to reconstruct Monticello metadata from git meta data 3. (variant of 1) store the original monticello meta data in a file, do not update on every commit but arrange for a way to reconstruct Monticello metadata from git meta data and graft onto original Monticello meta data for use in mcz repository --- on demand 4. ???? Option 3 seems to be a good compromise solution and perhaps is the approach that should be adopted moving forward ... we get to preserver Monticello metadata while avoiding the messy commit conflicts for git while providing a (somewhat) seamless path for a package to migrate back into the mcz repository world ... if we somehow incorporate the SHA of the commit and the github/bitbucket url into the revision history, then it would be possible to perform a 3 way merge involving two mcz package versions and common ancestor that is only present in git ... > > > Despite the fact that the Monticello metadata is redundant, I have > made sure that the Monticello metadata was included in FileTree > from the very beginning for the very reason that I wanted > developers to be able to try out FileTree, git and github without > having to burn any Monticello bridges .... if they didn't like > FileTree, git and github, then they would be able to back out of > their use of git without losing data ... > > > Then forgive me. I couldn't see the wood for the trees. When I read > your talk of eliminating the conflicts from git commits due to the > Monticello metadata I infer that you're eliminating the Monticello > metadata. I'm not sure I understand the implications of this, but it > seems to me that a natural consequence is that the Montivcello > metadata is lost and at that point merges and the like become > problematic. What am I missing? You've hit the nail on the head, but I think that option 3 above gives us a way to avoid losing the Monticello metadata without incurring a hit for a packages lifetime while in git ... A package that starts its life in git will have an empty Monticello metadata and a package that never makes it's way into an mcz repository will not incur per commit penalties ... > > > It seems entirely destructive. > > It is not destructive ... > > We have a functional package manager which currently supports > interchange between Pharo, Squeak and Cuis, > > and GemStone? > > I assume that you are talking about Monticello packages and > Monticello repositories ... or what? > > > Yes. > > > I am really not trying to do anything but "invent the future" --- > I am not trying to destroy, I am trying to improve ... If you are > not able to see the shortcomings of Monticello repositories (Note > that I am distinguishing between Monticello packages and > Monticello repositories --- FileTree uses Monticello packages and > replaces Monticello repositories with git) and where git has > advantages over Monticello repositories, then you should continue > to use Monticello repositories ... > > > But what does this imply to some package that starts off in a > Monticello repository and then spends some time in gitland? Can I > merge again? If I can I'm happy. If I can't, I feel sabotaged. and sabotage was never my intent ... > > Personally I don't see Monticello repositories going away anytime > soon and expect to support Monticello repositories in > GsDevKit_home, tODE, and Metacello for the rest of my life:) > > > Fine. Except merging is, IIUC, about method time stamps and > ancestry. If that gets preserved then I'm happy. But for the life of > me I haven't read an explanation that reassures me that these are > being preserved. Do you see the roots of my fear? > Haha, from the very beginning back in 2012, I understood that there would be fear and resistance to change and anger and joy and excitement but it was not clear when if ever we'd reach point where a resolution for this "problem" was needed: either no one would be interested or everyone would be interested or ??? I really think that option 3 is going to be the best compromise moving forward - there is some implementation work that will be required but I really think option 3 gives you (and frankly me) a way to preserve the Monticello revision history for packages that make their way back and forth between lifetimes in git and Monticello repositories. Eliot, I appreciate the fact that you demanded a better solution! Dale

Feb. 2, 2016