Hi Nicolai, On Mon, Sep 7, 2015 at 1:21 PM, Nicolai Hess <nicolaihess@web.de> wrote:

2015-09-07 20:10 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

On Sun, Sep 6, 2015 at 12:56 PM, Nicolai Hess <nicolaihess@web.de> wrote:

...

Do we have a documentation about the format of - class definitions - methods - class comments for the changes file ?

Google "specification of smalltalk chunk format" and you will see:

Overviews: Chunk File Format <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact...> Smalltalk Interchange Format - Draft Smalltalk Standard v1.9 <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact...> The Smalltalk fileOut format - 1 <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&cad=rja&uact...>

and more.

Thank you eliot (should have google that myself !)

...

...

I am asking because some entries are using two separator "! !" and some only one "!"

Indeed. It's in the spec. ! ! ends a run of methods in the same protocol. ! ends a method or a doit, etc. Read the spec.

ah! grouping multiple methods for one protocol.

Forgive me, I don't mean to complain, but... It's rather sad now I look at it that the specification misses a very cool thing about the format. Glenn Krasner invented it and he's no dummy. What the specs miss out is that the syntax for a protocol is actually a general escape mechanism. For example in !Object methodsFor: 'testing'! isUnderstood ^false! ! the fileIn parser evaluates Object methodsFor: 'testing' and then asks that object to scanFrom: the stream. And the above answers a ClassCategoryReader that reads chunks, compiling them and adding them to the relevant category, until it finds an empty chunk (hence ! ! is not special syntax, but actually a chunk followed by an empty chunk). So one /could/, if one wanted, add another kind of reader, for example !Preferences loadSavedPreferences! beCool true! beKnowledgeable true! rememberOnersHistorytLestOneWishesToRepeatIt true! ! and have this not file in, but store preferences. I think that's neat, and it's very sad that the ANSI standard did;t define this but reduced the mechanism to support only protocols, and thence cause people to wonder why the syntax was so weird, when in fact it was very simple and wonderfully extensible. Now I'm not blaming the standards process; the real problem here is that the scheme wasn't documented other than by its implementation and so the flexibility was never made apparent to the standardizers, and anyway they wanted an interchange format, which is not what Squeak and Pharo chunk readers provide; they provide an extensible format. The ANSI standard /could/ however have specified that the syntax is indeed extensible as described, but that conforming implementations are only expected to provide ClassCategoryReaders, and are expected to raise an error if given expressions they don't recognize, or some such. Also, given this history you can see why a single ! following a method definition isn't seen anymore; we've put the stamp information in the ClassCategoryreader so it can only parse a single method. If we'd done something like this (and I'm not suggesting that's a good idea) !Object methodsFor: 'testing'! [stamp: 'mad 01/04/2015 12:00'] isUnderstood ^#barely! [stamp: 'mad 01/04/2015 12:01'] isMisunderstood ^#almostCertainly! ! we'd still see the single !'s after methods. Two key points for me are - specify carefully and visibly - extensible standards are better than fixed standards And we have a problem with the "Recover lost changes" browser

...

...

(https://pharo.fogbugz.com/f/cases/16498/unrecoverable-changes) and I am not sure if this is a bug on the code importer or if it is a bug on how we write changes to the changes file.

_,,,^..^,,,_ best, Eliot

Hi Eliot Thanks for the discussion. I will archive it and reread it. Now I think that historically they missed several points I learned while working on the Moose file format: - having doit or expressions to be executed instead of a declaration puts the burden on the tools. - for example not having a class definition declaration and just a do it forces the tools to parse and guess. - relying on sequence is not good because if you file is cut then this impacts several entities instead of having just one impacted. - I do not get why each entity in the chunk format is not self contained because in any case no decent programmer would use it to write code. - Mixing annotation and essential information makes the format really verbose. - what we see is that if we want to exchange on mail about code in several method in different classes, the chunk format is a pain vs. Point >> signature [ code here ] - This is not by accident that in all the books we wrote we use Clas >> signature code This way is not good because we do not have the warranty that code is not cut. Hence the previous one is better. I will restart one of these days to work on an alternate file format for textual editor. I do not like the GNUSt syntax. Now indeed how to store metadata is a good question. Stef

On Thu, Sep 10, 2015 at 11:17 PM, Nicolai Hess <nicolaihess@web.de> wrote:

Now that this is fixed, should we (is it possible?) fix the changes file?

A script that replaces the "! !" by "! " in class comments should work. It's important to replace the ! mark by a space, but keep the byte positions of the strings. This is important, because the system recovers a method's source code by fetching it from the .changes (or .sources) file using the byte position where the source code was written (the sourcePointer in the trailer of the CompiledMethods). A concrete example: !OSTouchTwoFingersPinchInMoveEvent commentStamp: 'MerwanOuddane 5/11/2015 15:14' prior: 0! Two fingers are pinching in! ! <--- replace last ! by space I think this can work as a template for a script: fileStream := SourceFiles changesFileStream. fileStream resetToStart. <iterate over the fileStream and replace some !s> SourceFiles forceChangesToDisk. If somebody wants to take it... else I might do it during the weekend. Cheers, MArtin

2015-09-06 21:56 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:

...

Do we have a documentation about the format of - class definitions - methods - class comments for the changes file ?

I am asking because some entries are using two separator "! !" and some only one "!" And we have a problem with the "Recover lost changes" browser (https://pharo.fogbugz.com/f/cases/16498/unrecoverable-changes) and I am not sure if this is a bug on the code importer or if it is a bug on how we write changes to the changes file.

nicolai