Tx esteban I agree :) On Fri, Nov 10, 2017 at 7:30 PM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
none of them, Iâm sorry :) I think we can do a lot better with just a bit of effort.
Esteban
On 10 Nov 2017, at 15:26, Stephane Ducasse <stepharo.self@gmail.com> wrote:
I would prefer the last one. Regularity over exception. Now the parser should just be good :)
Stef
On Fri, Nov 10, 2017 at 7:20 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 10 Nov 2017, at 19:12, Stephane Ducasse <stepharo.self@gmail.com> wrote:
Hi sven
Do you have an example because I do not understand your doubling.
Sure,
Bla { bla:bla } [ self isFoo ifTrue: [ #foo ] ifFalse: [ [ "block" #bar ]] ]
would have to become
Bla { bla:bla } [ self isFoo ifTrue: [ #foo ]] ifFalse: [ [ "block" #bar ]]] ]
but this very rare (because usually code is intended).
Nicolas' latest idea is cool too (use as much ['s as needed to avoid conflict). So the example would then become
Bla { bla:bla } [[[ self isFoo ifTrue: [ #foo ] ifFalse: [ [ "block" #bar ]] ]]]
stef
On Fri, Nov 10, 2017 at 11:40 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
Right now, TonelParser>>#methodBody looks a bit dangerous (too complex), and Nicolas already found some breaking cases as well a one possible solution. I was thinking a bit along the same lines, so here is a proposal.
Overall I think we should keep the current readability of Tonel as much as possible (it is very important for diffs), while still being correct - i.e. no method source, even convoluted ones, possibly in different languages should break the encoding.
The problem is how to quickly scan a stream for the closing $] (which occurs as a single character on a line by its own). Especially in Pharo source code, that situation (a conflict between the proper marker and actual source) is very rare, since code is usually intended. But is can occur and must be dealt with.
The proposal is to just double the $] but only on lines that could conflict (which are lines consisting of only $]'s). The code itself becomes quite simple and remains fast enough.
Here is some code to play with (EOL fixed to #cr for demo purposes):
writer := [ :stream :methodSourceString | stream nextPut: $[; cr. methodSourceString linesDo: [ :line | (line notEmpty and: [ line allSatisfy: [ :ch | ch = $] ] ]) ifTrue: [ stream << $] ]. stream << line; cr ]. stream nextPut: $]; cr ].
reader := [ :stream | | line | (stream peekFor: $[) ifTrue: [ stream nextLine ] ifFalse: [ self error ]. String streamContents: [ :out | [ (line := stream nextLine) = ']' ] whileFalse: [ (line notEmpty and: [ line allSatisfy: [ :ch | ch = $] ] ]) ifTrue: [ out << line allButFirst ] ifFalse: [ out << line ]. out cr ] ] ].
encoded := String streamContents: [ :out | writer value: out value: (Collection>>#inject:into:) sourceCode ].
(reader value: encoded readStream) = ((Collection>>#inject:into:) sourceCode, String cr).
escapedEncoded := String streamContents: [ :out | writer value: out value: 'test "Make sure there is no whitespace at line start to force an escape" self isFoo ifTrue: [ #foo ] ifFalse: [ [ "block" #bar ]] ' ].
(reader value: escapedEncoded readStream).
What do you think ? Did I miss any special cases ? Is the price (very occasional insertion of an extra #] in the source code) worth the advantage (a simple, safe scheme).
Sven
PS: Note that several tools line syntax highlighting and double-click next to brackets have problems with weird delimiters such as in the above code ;-)