and if we remove the method properties from FileTree, then the monticello files stored there could not be transparently copied into and out of a FileTree repo ... and FileTree is SCM neutral ... Personally I use https://github.com/ThierryGoubier/GitFileTree-MergeDriver and never think twice about the properties files ... Dale On 04/07/2015 08:48 AM, Thierry Goubier wrote:

Hi Peter,

Le 07/04/2015 17:39, Peter Uhnák a écrit :

...

Hi,

are "methodProperties.json" files needed?

No.

...

Every time someone makes even small changes it is accompanied by ten times more json file changes which is a mess.

Yes, it creates quite some noise on the git logs and diffs.

...

So is it needed with git? Cannot the information be retrieved from git commits?

It is not needed for git.

GitFileTree retrieves the information via the git logs.

But, if we remove it entirely, then the repository won't be usable by filetree. So GitFileTree keeps writing that metadata to stay compatible.

Thierry

Yeah, I do use the MergeDriver and it saved me a lot of headache, but when I see things like this https://github.com/dynacase/dynacase/commit/90141d63bfdd433e51a768c2191e035b... where one five lines long method generated 14 file changes with 180 additions and 172 deletions... it makes the log on github and pull requests incredibly messy. I don't want to cut branch under myself if I were to remove the properties file. So my question now is: how hard would it be to regenerate those files? Or maybe if it was moved to some metadirectory. This reminds me a bit of svn which polluted the whole folder tree with .svn files everywhere. Peter On Wed, Apr 8, 2015 at 1:14 AM, Sean P. DeNigris <sean@clipperadams.com> wrote:

Dale Henrichs-3 wrote

...

Personally I use https://github.com/ThierryGoubier/GitFileTree-MergeDriver and never think twice about the properties files ...

Ooh, intriguing. In other to make it easier to view code on GitHub, I've been toying with the idea of generating one-class-per-file in addition to the regular gitfiletree files. Could this be used to make that possible without complicating Git?

----- Cheers, Sean -- View this message in context: http://forum.world.st/Metacello-GIT-methodProperties-json-tp4818097p4818233.... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Le 08/04/2015 02:05, Dale Henrichs a écrit :

Peter,

That looks like a bug in the FileTree writer ... wiht one method change there should have been only one method property file change ...

Yes, I'd agree. This is surprising. Unless you had been loading that package via another repository / another version / another branch before in the same image? MCMethodDefinition has a cache which consider equals methods with different timestamps, and, in that case, it would carry the timestamps of the previous version, and writing those upon save instead of the originals.

Peter and Sean,

If you are interested in contributing code/bugfixes to FileTree, I will welcome pull requests ... As I have mentioned in several posts today, I do not have the time to fiddle with changes to the FileTree format myself, but I welcome contributions.

Damien has started work on an updated version of the FileTree format[2].

I have threatened in the past to add an option to a repository that would eliminate the need to store monticello meta data ... Damien is working on "starting from scratch" on the new format, because the current implementation supports 4-5 different FileTree formats. Damien's work could be leveraged to add an "optional Monticello meta data" option to FileTree and if your SCM (like git) gives you per method history with the proper tools you can leverage that information..

I would say this could be a reasonable possibility, in that you could have two modes of compatibility: - Complete filetree compatibility for gitfiletree: the current situation, with properties and version written to disk. - Partial filetree compatibility, where that metadata isn't written and you rely on gitfiletree to recreate it to export as .mcz. Partial compatibility has two effects, which need to be considered. When using github:// or bitbucket:// urls, filetree will be used and your packages will end up having no version number (except .1?), no author, no timestamps for methods, etc... And the second one is that, if you clone a mcz inside the gitfiletree repository, the mcz ancestry of versions and author timestamps on methods will be lost, which is something you may not want. Thierry

As I ranted in another post ... changing the disk format is the easy part ... building and maintaining tools for the 4-5 different Smalltalk dialects is a different matter ...

Dale

[1] https://github.com/dalehenrich/filetree/issues [2] https://github.com/dalehenrich/filetree/issues/144 On 04/07/2015 04:38 PM, Peter Uhnák wrote:

...

Yeah, I do use the MergeDriver and it saved me a lot of headache, but when I see things like this

https://github.com/dynacase/dynacase/commit/90141d63bfdd433e51a768c2191e035b...

where one five lines long method generated 14 file changes with 180 additions and 172 deletions... it makes the log on github and pull requests incredibly messy.

I don't want to cut branch under myself if I were to remove the properties file. So my question now is: how hard would it be to regenerate those files?

Or maybe if it was moved to some metadirectory. This reminds me a bit of svn which polluted the whole folder tree with .svn files everywhere.

Peter

On Wed, Apr 8, 2015 at 1:14 AM, Sean P. DeNigris <sean@clipperadams.com <mailto:sean@clipperadams.com>> wrote:

Dale Henrichs-3 wrote > Personally I use > https://github.com/ThierryGoubier/GitFileTree-MergeDriver and never > think twice about the properties files ...

Ooh, intriguing. In other to make it easier to view code on GitHub, I've been toying with the idea of generating one-class-per-file in addition to the regular gitfiletree files. Could this be used to make that possible without complicating Git?

----- Cheers, Sean -- View this message in context: http://forum.world.st/Metacello-GIT-methodProperties-json-tp4818097p4818233.... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Excerpts from Peter Uhnák's message of 2015-04-08 08:47:15 +0200:

...

I thnk that rather than focus on the disk format which I hardly ever actually look at

i think the disk format is crucial for interoperability with other tools, such as github webinterface, or external editors to work on things like pictures or data. (which is not yet supported (i have an idea how support could look like though)) i don't mind file-per-method though, it helps to make the diff-stat more readable, showing which methods have been changed, added or removed...

... that folks should be looking at tools support (like

...

Thierry) ... this is where the real work needs to happen ... good tools can hide the disk fomat completely so why does the disk format matter ... I am personally not really a fan of this; I've been using git for a while and I am perfectly content with using command line on the disk (maybe I'm rare breed); I have yet to see a GUI/tool that would come even close to the power of command line, but I've been using Linux for a long time.

except for displaying the history graph. can't beat gitk or other gui tools for that. having a smalltalk version of that graph would really be great!

As I've said to Thierry some time ago in different thread, I would be interested in idea of having everything on disk side and Pharo would only somehow refresh it's content (just like a Java IDE / text editor would).

can't you already do that, except that you need to manually update the image by loading/adopting the current state from the repo? greetings, martin. -- eKita - the online platform for your entire academic life -- chief engineer eKita.co pike programmer pike.lysator.liu.se caudium.net societyserver.org secretary beijinglug.org mentor fossasia.org foresight developer foresightlinux.org realss.com unix sysadmin Martin Bähr working in china http://societyserver.org/mbaehr/

2015-04-08 8:47 GMT+02:00 Peter Uhnák <i.uhnak@gmail.com>:

I thnk that rather than focus on the disk format which I hardly ever

...

actually look at ... that folks should be looking at tools support (like Thierry) ... this is where the real work needs to happen ... good tools can hide the disk fomat completely so why does the disk format matter ...

I am personally not really a fan of this; I've been using git for a while and I am perfectly content with using command line on the disk (maybe I'm rare breed); I have yet to see a GUI/tool that would come even close to the power of command line, but I've been using Linux for a long time.

Agreed and this is in that spirit I did GitFileTree. Dale went a bit further and even integrated the git command line access inside smalltalk via CommandShell... so that you get the best of both worlds without leaving your beloved Smalltalk :)

As I've said to Thierry some time ago in different thread, I would be interested in idea of having everything on disk side and Pharo would only somehow refresh it's content (just like a Java IDE / text editor would). But that may be a lot o work so I can only dream about it, as nobody has time for that (me included).

Well, as I said elsewhere, Smalltalk doesn't work like that: code has to be lived in the image for most tools to work properly... it doesn't really have an "editor of file" mindset, but an "object inspector" mindset. What we're looking at in another thread is how to emulate that live object approach over dead files, and, thinking of it, I'm not sure we will succeed well. In short, what you want may never work properly because it is not how it should be conceptualized.

But to bottom line this thread (for me at least, because I'm getting lost): 1. methodProperties.json provide compatibility for FileTree, so I don't want to get rid of it (for now) 2. the large amount of file changes is probably a bug (I'll try to be more observant about the commits in the future and hopefully some pattern will emerge.) 3. I shouldn't want to break things for now at least 4. more people are starting to work on Git 5. I need to learn about FileTree at some point if I want to contribute and experiment with (my) ideas (which won't be soon as I'm preparing for my finals)

Please ask around for 5. It is valuable to have to expose what we already know. Thierry

Thanks for all your answers.

Peter

On Wed, Apr 8, 2015 at 7:24 AM, Thierry Goubier <thierry.goubier@gmail.com

...

wrote:

...

Le 08/04/2015 02:05, Dale Henrichs a écrit :

...

Peter,

That looks like a bug in the FileTree writer ... wiht one method change there should have been only one method property file change ...

Yes, I'd agree. This is surprising.

Unless you had been loading that package via another repository / another version / another branch before in the same image?

MCMethodDefinition has a cache which consider equals methods with different timestamps, and, in that case, it would carry the timestamps of the previous version, and writing those upon save instead of the originals.

Peter and Sean,

...

If you are interested in contributing code/bugfixes to FileTree, I will welcome pull requests ... As I have mentioned in several posts today, I do not have the time to fiddle with changes to the FileTree format myself, but I welcome contributions.

Damien has started work on an updated version of the FileTree format[2].

I have threatened in the past to add an option to a repository that would eliminate the need to store monticello meta data ... Damien is working on "starting from scratch" on the new format, because the current implementation supports 4-5 different FileTree formats. Damien's work could be leveraged to add an "optional Monticello meta data" option to FileTree and if your SCM (like git) gives you per method history with the proper tools you can leverage that information..

I would say this could be a reasonable possibility, in that you could have two modes of compatibility:

- Complete filetree compatibility for gitfiletree: the current situation, with properties and version written to disk.

- Partial filetree compatibility, where that metadata isn't written and you rely on gitfiletree to recreate it to export as .mcz.

Partial compatibility has two effects, which need to be considered. When using github:// or bitbucket:// urls, filetree will be used and your packages will end up having no version number (except .1?), no author, no timestamps for methods, etc... And the second one is that, if you clone a mcz inside the gitfiletree repository, the mcz ancestry of versions and author timestamps on methods will be lost, which is something you may not want.

Thierry

As I ranted in another post ... changing the disk format is the easy

...

part ... building and maintaining tools for the 4-5 different Smalltalk dialects is a different matter ...

Dale

[1] https://github.com/dalehenrich/filetree/issues [2] https://github.com/dalehenrich/filetree/issues/144 On 04/07/2015 04:38 PM, Peter Uhnák wrote:

...

Yeah, I do use the MergeDriver and it saved me a lot of headache, but when I see things like this

https://github.com/dynacase/dynacase/commit/ 90141d63bfdd433e51a768c2191e035b76c5da83

where one five lines long method generated 14 file changes with 180 additions and 172 deletions... it makes the log on github and pull requests incredibly messy.

I don't want to cut branch under myself if I were to remove the properties file. So my question now is: how hard would it be to regenerate those files?

Or maybe if it was moved to some metadirectory. This reminds me a bit of svn which polluted the whole folder tree with .svn files everywhere.

Peter

On Wed, Apr 8, 2015 at 1:14 AM, Sean P. DeNigris <sean@clipperadams.com <mailto:sean@clipperadams.com>> wrote:

Dale Henrichs-3 wrote > Personally I use > https://github.com/ThierryGoubier/GitFileTree-MergeDriver and never > think twice about the properties files ...

Ooh, intriguing. In other to make it easier to view code on GitHub, I've been toying with the idea of generating one-class-per-file in addition to the regular gitfiletree files. Could this be used to make that possible without complicating Git?

----- Cheers, Sean -- View this message in context: http://forum.world.st/Metacello-GIT-methodProperties-json- tp4818097p4818233.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

On 04/08/2015 01:11 AM, Thierry Goubier wrote:

2015-04-08 8:47 GMT+02:00 Peter Uhnák <i.uhnak@gmail.com <mailto:i.uhnak@gmail.com>>:

I thnk that rather than focus on the disk format which I hardly ever actually look at ... that folks should be looking at tools support (like Thierry) ... this is where the real work needs to happen ... good tools can hide the disk fomat completely so why does the disk format matter ...

I am personally not really a fan of this; I've been using git for a while and I am perfectly content with using command line on the disk (maybe I'm rare breed); I have yet to see a GUI/tool that would come even close to the power of command line, but I've been using Linux for a long time.

Agreed and this is in that spirit I did GitFileTree. Dale went a bit further and even integrated the git command line access inside smalltalk via CommandShell... so that you get the best of both worlds without leaving your beloved Smalltalk :)

Actually it is not quite CommandShell based ... the shell is tODE shell, where Smalltalk is the scripting language. The tODE git commands are implemented by parsing the command line args, resolving options and args to objects, and then calling a Smalltalk git command API that eventually does make shell calls (sometimes multiple git commands are folded into a single tODE git command ... thus making it possible to ultimately switch out the actual `git calls` and use something FFI-based, like libgit2... This approach also makes it quite feasible to write fairly complicated Smalltalk scripts that call the smalltalk git command api directly, bypassing the command line interface altogether ...

As I've said to Thierry some time ago in different thread, I would be interested in idea of having everything on disk side and Pharo would only somehow refresh it's content (just like a Java IDE / text editor would). But that may be a lot o work so I can only dream about it, as nobody has time for that (me included).

Well, as I said elsewhere, Smalltalk doesn't work like that: code has to be lived in the image for most tools to work properly... it doesn't really have an "editor of file" mindset, but an "object inspector" mindset.

What we're looking at in another thread is how to emulate that live object approach over dead files, and, thinking of it, I'm not sure we will succeed well.

In short, what you want may never work properly because it is not how it should be conceptualized.

In tODE I regularly drop off live objects to the file system using STON ... The tODE shell which allows you to navigate an object-based directory also allows you to mount the nodes from the file system and when traversing the file system recognizes STON files and presents objects to you instead of the dead files:) I haven't tried taking this approach all the way to source files ... but???? Dale

On 04/07/2015 10:24 PM, Thierry Goubier wrote:

Le 08/04/2015 02:05, Dale Henrichs a écrit : I have threatened in the past to add an option to a repository that would eliminate the need to store monticello meta data ... Damien is working on "starting from scratch" on the new format, because the current implementation supports 4-5 different FileTree formats. Damien's work could be leveraged to add an "optional Monticello meta data" option to FileTree and if your SCM (like git) gives you per method history with the proper tools you can leverage that information..

I would say this could be a reasonable possibility, in that you could have two modes of compatibility:

- Complete filetree compatibility for gitfiletree: the current situation, with properties and version written to disk.

- Partial filetree compatibility, where that metadata isn't written and you rely on gitfiletree to recreate it to export as .mcz.

Partial compatibility has two effects, which need to be considered. When using github:// or bitbucket:// urls, filetree will be used and your packages will end up having no version number (except .1?), no author, no timestamps for methods, etc... And the second one is that, if you clone a mcz inside the gitfiletree repository, the mcz ancestry of versions and author timestamps on methods will be lost, which is something you may not want. Yes for Metacello some modifications would need to be made because in a normal mcz universe Metacello will not load a package if the version is older or the same as the one in the image ... and following in the tradition of other Monticello tools the UUID is ignored:( .. I was working with a pure cypress implementation and have written the Metacello code to support these repos ...

A better long term solution is to treat these as a CypressPackage or FileTreePackage which has some similarities to Monticello packages ... because faking out Monitcello version numbers and users is not much different than storing the metadata in FileTree ... But someone were to add those options to filetree, I think we could reduce the overall number of hacks:) Dale

On 04/08/2015 12:47 PM, Thierry Goubier wrote:

Hi Dale,

Le 08/04/2015 19:30, Dale Henrichs a écrit :

...

On 04/07/2015 10:24 PM, Thierry Goubier wrote:

...

Le 08/04/2015 02:05, Dale Henrichs a écrit : I have threatened in the past to add an option to a repository that would eliminate the need to store monticello meta data ... Damien is working on "starting from scratch" on the new format, because the current implementation supports 4-5 different FileTree formats. Damien's work could be leveraged to add an "optional Monticello meta data" option to FileTree and if your SCM (like git) gives you per method history with the proper tools you can leverage that information..

I would say this could be a reasonable possibility, in that you could have two modes of compatibility:

- Complete filetree compatibility for gitfiletree: the current situation, with properties and version written to disk.

- Partial filetree compatibility, where that metadata isn't written and you rely on gitfiletree to recreate it to export as .mcz.

Partial compatibility has two effects, which need to be considered. When using github:// or bitbucket:// urls, filetree will be used and your packages will end up having no version number (except .1?), no author, no timestamps for methods, etc... And the second one is that, if you clone a mcz inside the gitfiletree repository, the mcz ancestry of versions and author timestamps on methods will be lost, which is something you may not want. Yes for Metacello some modifications would need to be made because in a normal mcz universe Metacello will not load a package if the version is older or the same as the one in the image ... and following in the tradition of other Monticello tools the UUID is ignored:( .. I was working with a pure cypress implementation and have written the Metacello code to support these repos ...

Ok.

Yes, the version number handling is delicate; even the one used by gitfiletree feels unsatisfying, even if it allows Metacello/Monticello to work as usual. The work I had done was at the Metacello level and is in the Metacello-Cypress package where I changed the load semantics to unconditionally load the package ... the snapshot comparison logic in play when loading packages is pretty good so it isn't necessarily a penalty to unconditionally compare snapshots when loading a package from FileTree ...

...

A better long term solution is to treat these as a CypressPackage or FileTreePackage which has some similarities to Monticello packages ... because faking out Monitcello version numbers and users is not much different than storing the metadata in FileTree ...

Not sure I understand. I know where the Cypress/FileTree code handles the lack of metadata when reading, that's all :)

...

But someone were to add those options to filetree, I think we could reduce the overall number of hacks:)

Another solution I'd see possible is a reduced metadata format, where just the version number of the current package is stored and retrieved (i.e. a version file without ancestry).

It would allow FileTree and Metacello to work exactly as usual.

And, maybe, I could even unify the UUIDs?

There are things in Monticello which needs a rethink to better adapt to SCMs like git (or Fossil: it's so similar to git :) ) while keeping more or less the same appearance: branches, hashes, tags and semantic versionning.

But I don't see yet how to handle that or to propose anything.

I know that it would be great to reduce the cognitive distance between MC version and history handling, and the same as seen from Metacello baselines and configurations. Branches + baselines are doing a lot to simplify, but MC stuff is unaware of that. And MC should delegate merging to the underlying repository, as well.

So this "extraneous" Monticello meta data is really related to the versioning aspects of the Monticello package ... aspects that really needed if you are using "git" (read "git" as svn, hg, git, fossil, etc.:) for version control .. in working with "git" the SHA of the commit for the repository as a whole is more important than any of the package meta data ... from the SHA one can see the commit comment for the "git" commit the SHA is related to a project not a package ... the project contents is represented by the BaselineOf ... in git one can reason about the collection of commits associated with a directory (package or class or method) and look at diffs at that level of granularity .... The primary point being that a "git" package can be stripped down to the basic list of definitions and the ancestry and version names and method properties can be dropped on the floor instead of fabricated ... What I imagine is a separate software stack for these "git" packages the commonality is at the snapshot/definition level and the core definitions can migrate between Monticello and "git" packages almost without change ... if one wanted to know the "history" of a package the Monticello package would answer back a representation of the ancestor graph, while the "git" package would answer back a representation of the commit history ... there would be different objects in the histories, since the meta data for extracting the useful bits varies considerably between the two package implementations, but from a tools perspective and arranging for a common tool-level api would be able to accommodate these two completely different packages as well as any number of package implementations that could wrap packages stored in the wide variety of SCMS floating around out there .... and those to come ... what the hell:) Well there's a year or so of work:) Dale

Sean, My mail may have been a bit harsh, so please read my reply to Peter ... where I think I take a bit more positive tack:) I am really pressed for time so I don't have the luxury to blue sky ideas ... but I will make the time to review any code contributions that folks have ... Dale On 04/07/2015 04:52 PM, Dale Henrichs wrote:

Sean,

The primary reason that the method properties.json file exists is to preserve the date and timestamp for monticello ... git doesn't need this but, to be able to seamlessly copy monticello files from filetree to mcz and back again without losing data the information has to be stored somewhere ... a pure git repo does not need this information and it can be eliminated, but you pay the price of cutting off mcz file compatibility ... FileTree is agnostic as to the SCM being used ... it is a disk format ... so it cannot leverage the meta data from git ... if git is not used to manage the disk ...

I appreciate the fact that folks are coming up with all sorts of ideas ... but there are constraints as to what is done for FileTree ... FileTree is supposed to be cross platform so changing disk formats is not as easy as just writing some smalltalk code, unless you want to write the smalltalk code for the 4 or 5 different Smalltalk dialects that read/write FileTree ...

Thierry has taken the approach of maintaining compatibility with FileTree, which is a convenient thing to do ... convenient for all of the folks that have been using Filetree for the last 3 years and have been living with the limitations and patiently waiting for folks to begin adoption ...

the problem with patiently waiting is that I am very busy right now and I do not have the time to fiddle with FileTree ... I know that a number of you guys don't care about other platforms and don't care about backwards compatibility ... but I do and I do not have the free time to fiddle around with the fad of the moment ...

I thnk that rather than focus on the disk format which I hardly ever actually look at ... that folks should be looking at tools support (like Thierry) ... this is where the real work needs to happen ... good tools can hide the disk fomat completely so why does the disk format matter ...

BTW, we had some major email wars several years ago and frankly I don't have the time to justify decisions that were made 3 years ago every 2 months ....

Dale

On 04/07/2015 04:14 PM, Sean P. DeNigris wrote:

...

Dale Henrichs-3 wrote

...

Personally I use https://github.com/ThierryGoubier/GitFileTree-MergeDriver and never think twice about the properties files ... Ooh, intriguing. In other to make it easier to view code on GitHub, I've been toying with the idea of generating one-class-per-file in addition to the regular gitfiletree files. Could this be used to make that possible without complicating Git?

----- Cheers, Sean -- View this message in context: http://forum.world.st/Metacello-GIT-methodProperties-json-tp4818097p4818233.... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Le 08/04/2015 13:23, Sean P. DeNigris a écrit :

Dale Henrichs-3 wrote

...

My mail may have been a bit harsh, so please read my reply to Peter ... where I think I take a bit more positive tack:)

We are beyond politeness, my friend... Not our first rodeo :)

Dale Henrichs-3 wrote

...

I am really pressed for time so I don't have the luxury to blue sky ideas ...

I understand. I was not suggesting anyone do anything. Also, I agree that methods seem to be the proper level for SCM versioning as they are in the Smalltalk image. I was simply asking, in light of the appearance that the merge driver could seemingly ignore the changes to certain files while still committing them with Git, if it would be feasible to add read-only one-class-per files "for viewing on GitHub". This is more of a sanity check to see if I understood what the merge driver makes possible.

The merge driver seemingly ignoring the changes to certain files while still committing? No, doesn't work that way. The merge driver never commits. Unless you mean that, in case of conflicts on merging that file, that the merge driver should be disregarding the conflicts and choose one version over another... Not sure to see how that would work. Remember that the merge driver is only called by git merge. Nothing else. Adding a description file would enter it into the git and make it tracked as are all other files. And adding a file is really not a big deal. Thierry

----- Cheers, Sean -- View this message in context: http://forum.world.st/Metacello-GIT-methodProperties-json-tp4818097p4818323.... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.