Thank you Sven for your follow up of the recent JSON/STON/Tirade discussion by providing an STON implementation for Squeak 4.3 and working with Dale to get it to other Smalltalks (github). [1] I think your STON implementation will be very useful for my work. It works fine in Squeak 4.3 updated to the latest change [2]. There is a class STON used as a singleton. I assume this is the equivalent of the JSON object in web browsers, right? --Hannes ----------------------------------------------------------------------------------------------------------------- [1] http://www.mail-archive.com/pharo-project@lists.gforge.inria.fr/msg62706.htm... Example object Dale H. writes Tue, 24 Apr 2012 10:23:18 -0700 Sven, I looked at your sample document in the pdf: TestDomainObject { #created : DateAndTime [ '2012-02-14T16:40:15+01:00' ], #modified : DateAndTime [ '2012-02-14T16:40:18+01:00' ], #integer : 39581, #float : 73.84789359463944, #description : 'This is a test', #color : #green, #tags : [ #two, #beta, #medium ], #bytes : ByteArray [ 'afabfdf61d030f43eb67960c0ae9f39f' ], #boolean : false } ----------------------------------------------------------------------------------------------------------------- [2] Tests in Squeak 4.3 MCHttpRepository location: 'http://ss3.gemstone.com/ss/STON' user: '' password: '' loads fine in a fully updated Squeak-4.3 all 86 tests are green. On 5/5/12, Serge Stinckwich <serge.stinckwich@gmail.com> wrote:

On Sat, May 5, 2012 at 4:02 AM, Sven Van Caekenberghe <sven@beta9.be> wrote:

...

Hi,

STON - Smalltalk Object Notation is a lightweight text-based, human-readable data interchange format for class-based object-oriented languages like Smalltalk. It can be used to serialize domain level objects, either for persistency or network transport. As its name suggests, it is based on JSON (Javascript Object Notation). It adds symbols as a primitive value, class tags for object values and references.

Please read the paper here:

https://github.com/svenvc/ston/blob/master/ston-paper.md

The project lives here:

https://github.com/svenvc/ston

The code can be found here as well:

http://ss3.gemstone.com/ss/STON

This implementation works on Pharo 1.3, 1.4 and 2.0 as well as on Squeak.

Last week Dale Henrich helped me get started with filetree/git (https://github.com/dalehenrich/filetree) and I like it. Check it out! He also ported STON to Gemstone, and is working on Amber and Cuis ports (https://github.com/dalehenrich/ston).

Welcome on board ;-)

Regards, -- Serge Stinckwich UMI UMMISCO 209 (IRD/UPMC), Hanoi, Vietnam Every DSL ends up being Smalltalk http://doesnotunderstand.org/

On 05 May 2012, at 15:58, Sven Van Caekenberghe wrote:

Maybe I should add some invocation examples to the paper to explain the API.

https://github.com/svenvc/ston/blob/master/ston-paper.md#usage Sven

Hey! On 05/08/2012 10:05 AM, Janko Mivšek wrote:

Hi Göran,

One quick note: STON has more chance to be adopted than Tirade simply because of the name. STON is closer to JSON in people's mind, while Tirade, what that means?

Do note that IMHO Tirade does not really compete with STON/JSON. As I think I wrote in the article I tend to think JSON trumphs STON but that Tirade may be more powerful and interesting in some Smalltalk specific cases.

For me it sounds something tyran-like , pretty non-attractive therefore. And because most of us don't have time to go to details, such outside things like name matters. We are also more and more choosing things based on their perception and not reality. More on their outside form than on their real value.

Hmmm, I don't think I agree that the name is all that important for this kind of highly *technical* packages. STON is closer to JSON which is good since it basically is a true superset and tries to fill the exact same niche. But I agree, for "high profile" packages naming is important - like I convinced Nicolas to rename JTalk to something nicer. Tirade was just a funny name based on the "cascade" nature of the format: "a long angry speech or scolding. Synonyms: diatribe, harangue, rant" I mean, don't people have a sense of humor? :)

So, to have bigger adoption rate we need to take care about naming things as well.

Hehe, well... I am sorry Janko, I don't think Tirade is such a bad name and I don't think it would be worth renaming either. Any other more technical thoughts/comments? regards, Göran

On Thu, 10 May 2012, H. Hirzel wrote:

Hello

A question about dynamic arrays

Goran writes that he thinks they were introduced in Squeak?

Göran wrote: "It is simply a dynamic {} array (was introduced originally in Squeak I believe) but with the assumption that the expressions all evaluate to Associations that are limited to a string as key." So it's not about dynamic arrays, but arrays created with the brace syntax. And yes, that was introduced in Squeak. Btw OrderedCollections are dynamic arrays. Levente

Is this correct? If yes when? References? Which other Smalltalk dialects do have them?

Regards Hannes

On 5/8/12, Janko Mivšek <janko.mivsek@eranova.si> wrote: [...snip]

...

Dne 08. 05. 2012 09:55, piše Göran Krampe:

...

Hi guys!

I just posted an article about JSON/STON/Literal arrays/Tirade - and it is probably easiest to read at planet.smalltalk.org (my blog has a rather bad theme I realized):

http://planet.smalltalk.org

...or at my blog:

http://goran.krampe.se/2012/05/08/literal-arrays-vs-json-vs-ston-vs-tirade

Sven - sorry for beating down a bit on STON in that article, nothing personal and I love all the stuff you have done, I just have a hard time placing STON in my toolbox. If you can give me good arguments why I am dead wrong - please do! ;)

And regarding Tirade, feel free to criticize that too of course, it is still not set in stone and all input is appreciated, including telling me it sucks :).

The most interesting bit of the article is perhaps my thoughts on literals in Smalltalk - would be nice to hear what people thinḱ. And oh, sorry for the length of the article - it kinda swelled.

regards, Göran

On 5/10/12, Levente Uzonyi <leves@elte.hu> wrote:

On Thu, 10 May 2012, H. Hirzel wrote:

...

Hello

A question about dynamic arrays

Goran writes that he thinks they were introduced in Squeak?

Göran wrote: "It is simply a dynamic {} array (was introduced originally in Squeak I believe) but with the assumption that the expressions all evaluate to Associations that are limited to a string as key."

So it's not about dynamic arrays, but arrays created with the brace syntax. And yes, that was introduced in Squeak.

Any idea when and for which reason? Where are they used these days? Which other Smalltalk dialects have adopted it?

Btw OrderedCollections are dynamic arrays.

Levente

...

Is this correct? If yes when? References? Which other Smalltalk dialects do have them?

Regards Hannes

On 5/8/12, Janko Mivšek <janko.mivsek@eranova.si> wrote: [...snip]

...

Dne 08. 05. 2012 09:55, piše Göran Krampe:

...

Hi guys!

I just posted an article about JSON/STON/Literal arrays/Tirade - and it is probably easiest to read at planet.smalltalk.org (my blog has a rather bad theme I realized):

http://planet.smalltalk.org

...or at my blog:

http://goran.krampe.se/2012/05/08/literal-arrays-vs-json-vs-ston-vs-tirade

Sven - sorry for beating down a bit on STON in that article, nothing personal and I love all the stuff you have done, I just have a hard time placing STON in my toolbox. If you can give me good arguments why I am dead wrong - please do! ;)

And regarding Tirade, feel free to criticize that too of course, it is still not set in stone and all input is appreciated, including telling me it sucks :).

The most interesting bit of the article is perhaps my thoughts on literals in Smalltalk - would be nice to hear what people thinḱ. And oh, sorry for the length of the article - it kinda swelled.

regards, Göran

On 5/10/12, Bert Freudenberg <bert@freudenbergs.de> wrote:

On 10.05.2012, at 13:25, H. Hirzel wrote:

...

On 5/10/12, Levente Uzonyi <leves@elte.hu> wrote:

...

So it's not about dynamic arrays, but arrays created with the brace syntax. And yes, that was introduced in Squeak.

Any idea when and for which reason?

Dan Ingalls, 1997: "Larry Tesler added the Curly Brace construct to Apple Smalltalk back around 1985, and there it sat in Apple's APDA (Apple Programmer and Developer Association) Smalltalk release for about a decade. Then we resurrected that image as the basis of Squeak, and I've never bothered to tear it out."

More at http://www.cc.gatech.edu/fac/mark.guzdial/squeak/notes/curlybraces.html

- Bert -

Thanks for the link Bert, <citation> David N. Smith wrote: Curly braces are currently used in Squeak to construct runtime-evaluated collections </citation>. The question is -- is there some more documentation than just some clippings from an email correspondence? If I understand you correctly people 'forgot' to take them out and so this syntax stayed. Is it used much? Would it be possible to deprecate this usage and reuse the syntax for implementing literal dictionaries 'JSON style'? --Hannes

GemStone supports the {} array constructor syntax. ----- Original Message ----- | From: "H. Hirzel" <hannes.hirzel@gmail.com> | To: "The general-purpose Squeak developers list" <squeak-dev@lists.squeakfoundation.org> | Cc: Pharo-project@lists.gforge.inria.fr | Sent: Thursday, May 10, 2012 4:25:31 AM | Subject: Re: [squeak-dev] Re: A comparative article (was STON - Smalltalk Object Notation) | | On 5/10/12, Levente Uzonyi <leves@elte.hu> wrote: | > On Thu, 10 May 2012, H. Hirzel wrote: | > | >> Hello | >> | >> A question about dynamic arrays | >> | >> Goran writes that he thinks they were introduced in Squeak? | > | > Göran wrote: | > "It is simply a dynamic {} array (was introduced originally in | > Squeak I | > believe) | > but with the assumption that the expressions all evaluate to | > Associations | > that are limited to a string as key." | > | > So it's not about dynamic arrays, but arrays created with the brace | > syntax. And yes, that was introduced in Squeak. | | Any idea when and for which reason? | | Where are they used these days? | | Which other Smalltalk dialects have adopted it? | | | > | > Btw OrderedCollections are dynamic arrays. | > | > | > Levente | > | >> Is this correct? If yes when? References? | >> Which other Smalltalk dialects do have them? | >> | >> Regards | >> Hannes | >> | >> | >> On 5/8/12, Janko Mivšek <janko.mivsek@eranova.si> wrote: | >> [...snip] | >>> | >>> Dne 08. 05. 2012 09:55, piše Göran Krampe: | >>>> Hi guys! | >>>> | >>>> I just posted an article about JSON/STON/Literal arrays/Tirade - | >>>> and it | >>>> is probably easiest to read at planet.smalltalk.org (my blog has | >>>> a | >>>> rather bad theme I realized): | >>>> | >>>> http://planet.smalltalk.org | >>>> | >>>> ...or at my blog: | >>>> | >>>> http://goran.krampe.se/2012/05/08/literal-arrays-vs-json-vs-ston-vs-tirade | >>>> | >>>> Sven - sorry for beating down a bit on STON in that article, | >>>> nothing | >>>> personal and I love all the stuff you have done, I just have a | >>>> hard | >>>> time | >>>> placing STON in my toolbox. If you can give me good arguments | >>>> why I am | >>>> dead wrong - please do! ;) | >>>> | >>>> And regarding Tirade, feel free to criticize that too of course, | >>>> it is | >>>> still not set in stone and all input is appreciated, including | >>>> telling | >>>> me it sucks :). | >>>> | >>>> The most interesting bit of the article is perhaps my thoughts | >>>> on | >>>> literals in Smalltalk - would be nice to hear what people thinḱ. | >>>> And | >>>> oh, | >>>> sorry for the length of the article - it kinda swelled. | >>>> | >>>> regards, Göran | >> | >> | |

On 8 May 2012 10:07, Sven Van Caekenberghe <sven@beta9.be> wrote:

On 08 May 2012, at 09:55, Göran Krampe wrote:

...

Sven - sorry for beating down a bit on STON in that article, nothing personal and I love all the stuff you have done, I just have a hard time placing STON in my toolbox. If you can give me good arguments why I am dead wrong - please do! ;)

No problem, Göran, discussion are good. No offense taken.

I will try to come back with some more feedback on your post and about Tirade once I have more time to read and think about it. But I have little time this month.

Here are some quick reactions and factual corrections:

- <disclaimer> I don't want to push STON on anybody, I don't want to go on a crusade promoting it.</disclaimer>

- please read the, much extended, paper

       https://github.com/svenvc/ston/blob/master/ston-paper.md

- there are a couple of more examples in the new version of  the paper

- Symbols in STON are not artificially limited, there are two variants

       #foo        #key-value        #FOO/1

       #'foo$bar!'        #''\u00E9l\u00E8ve en Fran\u00E7ais'

- yes, the other primitives are intentionally kept equal to JSON

- STON is already ported and being ported to other Smalltalks

- STON is now backwards compatible with JSON (both nil and null, as well as singe and double quotes work)

- you skipped some of the predefined representations that make life so much easier in STON

       Time['10:32:10']        Date['2012-05-08']        DateAndTime['2012-05-08T10:33:05.134+02:00']        ByteArray['F87800FF12']        Point[10,-10]

- but the key point is indeed the class tag (hence the definition, STON is a lightweight, text-based, human-readable data interchange format for class-based object-oriented languages), your small example

       IntegerArray [1, 2, 3]

       vs

       {"type": "IntegerArray", "data": [1, 2, 3]}

(Your ByteArray example would be ByteArray['010203'] in STON, BTW).

gives the impression that this is a small thing that is easily overcome, but I strongly disagree, and that is the main reason why I want to see how far STON can go. The things is, some people will call the field "type", others might call it "class" or "_class" or whatever, the value might be "Rectangle" or "java.lang.Rectangle" or "geometry$rectangle". This will never be interoperable and falls out of the JSON spec.

JSON on itself is *not* self-describing: you either have to use and agree on annotations like the above, or both parties have to agree on types externally. JSON is *not* a serialization format. Despite all that JSON *is* successful, which is great.

The annotation approach completely breaks down, IMHO, in terms of lightweight & human readable, when the example becomes even slightly more complicated:

       Rectangle { #origin : Point [ -40, -15 ], #corner : Point [ 60, 35 ] }

       vs

       { "type" : "Rectangle", "data" : {                "origin" : { "type" : "Point", "data" : { "x" : -40, "y" : -15 } },                "corner" : { "type" : "Point", "data" : { "x" : 60, "y" : 35 } } }

It's worse than that in two ways: given some type Foo understood by the two ends of the connection, unless you have frozen your wire contract (the shape of things), you need to add versioning. Your types essentially become something like "Foo@1.1" meaning " a version 1.1 Foo". That implies some convention on handling types of differing version. This issue's not limited to JSON: it'll occur in any serialisation mechanism. [1] But I guess the point of []s is to indicate "this is a well-known class whose shape WILL NOT change because its meaning is as per STON, not your crazy dialect where Rectangle has a special unique shape". The second way is one you already know about: JSON is tree-like, not DAG-like, so you can either use a convention for references, or break DAGs into trees by cloning nodes. In the latter case, you then need a way of reconstructing the DAG, for instance by giving the nodes in your graph an identity (like a "_node" : <an integer> field). frank [1] This links back to a topic I've mentioned before: in CLOS you have UPDATE-INSTANCE-FOR-REDEFINED-CLASS (http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/stagenfun_...), but I'll not pursue that issue any further; it's pretty off-topic for the thread.

Granted, Point is treated special in STON, without that special treatment, it would be

       Rectangle { #origin : Point { #x : -40, #y : -15 ], #corner : Point { #x : 60, #y : 35 } }

But let me be absolutely clear: if I want to publish a REST API to talk to the outside world, I would most certainly use JSON (with externally agreed types). For me, STON, is for use between Smalltalk programs (client-server, inter-server, preference, property and configuration files, messaging, meta-info, …). It could work with Java, C#, Objective-C, and others.

Regards,

Sven

-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

(The paper was indeed just updated a couple of days ago). On 08 May 2012, at 13:29, Göran Krampe wrote:

- In general I believe in 99% of the cases the parsing system has to know what it is parsing, how the JSON looks and how it should be mapped onto objects. Making the JSON parser itsy bitsy smarter with type annotations doesn't help me there, I still need to know that I want to instantiate a PDFPage and put this Rectangle into it - but oh, perhaps not as a Rectangle, but perhaps as two points sent into a class side instantiation message or something.

My point being that the type annotations doesn't "buy me" enough.

(Just for the sake of the argument, I don't want to convince you) STON *does* buy you that: as long as we are talking about 'domain level objects' (a vague notion I agree) your Smalltalk object will serialize without any extra effort (see the ZnResponse example in the paper). You can customize some objects if you want to (mostly for readability, sometimes to fix some issue). Object implements #toSton: and #fromSton: by iterating over the instances variables. The issues are: Blocks, Classes and the fact that ephemeral instances are always serialized. Sven

:):):) for the correct definition of "works" I agree :):):) Dale ----- Original Message ----- | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, May 8, 2012 10:46:08 AM | Subject: Re: [Pharo-project] A comparative article (was Re: [squeak-dev] [ANN] STON - Smalltalk Object Notation) | | for me I can understand that people want a format to exchange objects | and that they want to use STON but | I do not understand why we need that to store metadata when a simple | array works. Probably we love to | load our boat with extra readers and writers. | | Stef | | | On May 8, 2012, at 2:05 PM, Sven Van Caekenberghe wrote: | | > (The paper was indeed just updated a couple of days ago). | > | > On 08 May 2012, at 13:29, Göran Krampe wrote: | > | >> - In general I believe in 99% of the cases the parsing system has | >> to know what it is parsing, how the JSON looks and how it should | >> be mapped onto objects. Making the JSON parser itsy bitsy smarter | >> with type annotations doesn't help me there, I still need to know | >> that I want to instantiate a PDFPage and put this Rectangle into | >> it - but oh, perhaps not as a Rectangle, but perhaps as two | >> points sent into a class side instantiation message or something. | >> | >> My point being that the type annotations doesn't "buy me" enough. | > | > (Just for the sake of the argument, I don't want to convince you) | > | > STON *does* buy you that: as long as we are talking about 'domain | > level objects' (a vague notion I agree) your Smalltalk object will | > serialize without any extra effort (see the ZnResponse example in | > the paper). You can customize some objects if you want to (mostly | > for readability, sometimes to fix some issue). | > | > Object implements #toSton: and #fromSton: by iterating over the | > instances variables. | > | > The issues are: Blocks, Classes and the fact that ephemeral | > instances are always serialized. | > | > Sven | > | > | | |

Stef, To me your argument is equivalent to saying "we've got nails why do we need screws"...Where ever you use a screw you should be able to use a nail right? But screws have slightly different characteristics than nails and when those characteristics are called for you use a screw instead of a nail... right? STON, JSON, Tirade and literal arrays all perform the same basic function, but each have slightly different characteristics that make one better for some tasks and another better for other tasks ... right? Dale ----- Original Message ----- | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, May 8, 2012 12:03:18 PM | Subject: Re: [Pharo-project] A comparative article (was Re: [squeak-dev] [ANN] STON - Smalltalk Object Notation) | | | On May 8, 2012, at 8:32 PM, Dale Henrichs wrote: | | > :):):) for the correct definition of "works" I agree :):):) | | sorry but I did not get it funny at all. | But if you laugh even better. So far I do not understand but I'm | probably an idiot understand | why we need dictionary syntax. But again this should be that I'm | totally dull on that topics. | So I will not read mails about this topic anymore. | Too bad may be people could have gained from my idiot views. | | Stef | | > | > Dale | > | > ----- Original Message ----- | > | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | > | To: Pharo-project@lists.gforge.inria.fr | > | Sent: Tuesday, May 8, 2012 10:46:08 AM | > | Subject: Re: [Pharo-project] A comparative article (was Re: | > | [squeak-dev] [ANN] STON - Smalltalk Object Notation) | > | | > | for me I can understand that people want a format to exchange | > | objects | > | and that they want to use STON but | > | I do not understand why we need that to store metadata when a | > | simple | > | array works. Probably we love to | > | load our boat with extra readers and writers. | > | | > | Stef | > | | > | | > | On May 8, 2012, at 2:05 PM, Sven Van Caekenberghe wrote: | > | | > | > (The paper was indeed just updated a couple of days ago). | > | > | > | > On 08 May 2012, at 13:29, Göran Krampe wrote: | > | > | > | >> - In general I believe in 99% of the cases the parsing system | > | >> has | > | >> to know what it is parsing, how the JSON looks and how it | > | >> should | > | >> be mapped onto objects. Making the JSON parser itsy bitsy | > | >> smarter | > | >> with type annotations doesn't help me there, I still need to | > | >> know | > | >> that I want to instantiate a PDFPage and put this Rectangle | > | >> into | > | >> it - but oh, perhaps not as a Rectangle, but perhaps as two | > | >> points sent into a class side instantiation message or | > | >> something. | > | >> | > | >> My point being that the type annotations doesn't "buy me" | > | >> enough. | > | > | > | > (Just for the sake of the argument, I don't want to convince | > | > you) | > | > | > | > STON *does* buy you that: as long as we are talking about | > | > 'domain | > | > level objects' (a vague notion I agree) your Smalltalk object | > | > will | > | > serialize without any extra effort (see the ZnResponse example | > | > in | > | > the paper). You can customize some objects if you want to | > | > (mostly | > | > for readability, sometimes to fix some issue). | > | > | > | > Object implements #toSton: and #fromSton: by iterating over the | > | > instances variables. | > | > | > | > The issues are: Blocks, Classes and the fact that ephemeral | > | > instances are always serialized. | > | > | > | > Sven | > | > | > | > | > | | > | | > | | > | | |

On May 8, 2012, at 8:33 PM, Göran Krampe wrote:

On 05/08/2012 07:46 PM, Stéphane Ducasse wrote:

...

for me I can understand that people want a format to exchange objects and that they want to use STON but I do not understand why we need that to store metadata when a simple array works. Probably we love to load our boat with extra readers and writers.

No, it was just a "trigger" for this discussion/comparison.

BUT... I would like you to at least read the parts in my article which is about improving the literal support we have in Smalltalk/Pharo. The main "problem" with literals that I can see (compared to Tirade) are:

- There is no restricted parser for them (as Tirade is), you need to use Compiler and that is a HUGE security problem. But of course, you are fixing that, right? :)

:) there is no security problem just that gemstone does not have a parser. Because parsing cannot be a security hole this is blind data interpretation that is a problem. This is why Github got problems and I imagine that the code was probably passed in JSON or similar format.

- There is no support for Associations as literals which means there is no good support in literals to express Dictionaries/key-value-pairs (like JSON has). Tirade has that in the form of literal Associations. I think we should add something in this area to Pharo/Smalltalk in general, but for more thoughts on that, please see article.

Ok let me try to understand what I do not get. Do you mean that we cannot distinguish between #((a 13) 'a' 'b' 1 ) and #(a -> 13 'a' 'b' 1 ) why #(Dictionary (a 13) (b 15)) or #(Dictionary ((a 13) (b 15))) is not the same as #(Dictionary a-> 13 b ->15) Now the problem is that a -> 13 is not created at parse time. So I do not get why

regards, Göran

...

...

:) there is no security problem just that gemstone does not have a parser. Because parsing cannot be a security hole this is blind data interpretation that is a problem. This is why Github got problems and I imagine that the code was probably passed in JSON or similar format.

Mmm, well, if you rely on full Compiler to parse literals, it is a security problem. But if you use the technique Igor showed to only parse array literals - then yes, that's safe, I agree. I forgot about his post.

Parsing and compiling + execution are three different actions. Parsing: stream -> tokens* but you know it.

...

...

- There is no support for Associations as literals which means there is no good support in literals to express Dictionaries/key-value-pairs (like JSON has). Tirade has that in the form of literal Associations. I think we should add something in this area to Pharo/Smalltalk in general, but for more thoughts on that, please see article.

Ok let me try to understand what I do not get.

Do you mean that we cannot distinguish between #((a 13) 'a' 'b' 1 )

and #(a -> 13 'a' 'b' 1 )

why #(Dictionary (a 13) (b 15)) or #(Dictionary ((a 13) (b 15)))

is not the same as #(Dictionary a-> 13 b ->15)

Now the problem is that a -> 13 is not created at parse time. So I do not get why

I presume you are describing various conventions/tricks to represent a Dictionary in a literal array, right?

My point is that it would be nice if we actually had proper syntax for Associations (and or Dictionaries).

OK I see. But what I want to say is that then this is not a literal array anymore. Literal means that you only need a parser and no execution to get the information. Parse time. Now I would like to know the use case for a dictionary syntax or association. May be you mean that you want to have the equivalent of { } but for dictionary, why not. Having a syntax for a syntax is not really exciting. Now I think that the key problem is dealing with reference between elements. for that STON can be better because why stopping at dictionary? But then the next question if we go down this road is why don't we add this syntax to smalltalk so that we can create literals objects like in Javascript. I mean without message send because this is exactly that no? What is the difference between Dictionary new add: #flop put: 33 ;yourself and {|% flop : 33 %|} or Person { name : 'stef' } then this is not really Smalltalk anymore and we cannot say that we only have messages and the parser and …. have to be changed…. This is why I asked what is the use case. Finally and I still did not get a clear answer: why there is a need for dictionary syntax to store method meta data. Ok if you need a ***syntax*** then this is another requirement. Stef

...

Finally and I still did not get a clear answer: why there is a need for dictionary syntax to store method meta data.

(I still think we are all discussing different things, and I do understand your point, and I myself don't want to force STON on anybody).

Yes this is not what I meant but I wanted to understand what people really want because so far this is still unclear to me.

But a quick question then, since I know you also know Lisp. Yes I still like its sexpressions style :)

Do you then see array literal syntax as s-expressions ?

I do not know what you mean by that. The problem is that for objects it does not work (this is what we tried to do with selfEvaluating objects - it was to get as close as possible as true => true # (1 2 3) => #(1 2 3) {(10 @20) . (20@ 20)} => {(10 @20) . (20@ 20)} but {Person new} => anOrderedCollection (….)

So that dictionaries/maps are like Lisp plists ?

Yes kind of

Dictionary new at: #x put: 1; at: #y put; 2; yourself.

would then be encoded like

#( x 1 y 2 )

yes or #((x 1) (y 2))

?

Eventually doing a

Dictionary newFromPairs:: #( x 1 y 2 )

? It would certainly work, with the know limitation that one has to know or understand the semantics upfront.

Yessssssss :) And the plus that we do not have to have a new parser. the syntax for dictionary does just that. (ok it can be something big and important) but my question is what is the cost and the use case. Stef

Sven

On 5/8/12, Sven Van Caekenberghe <sven@beta9.be> wrote:

Stéphane,

On 08 May 2012, at 22:59, Stéphane Ducasse wrote:

...

Finally and I still did not get a clear answer: why there is a need for dictionary syntax to store method meta data.

(I still think we are all discussing different things, and I do understand your point, and I myself don't want to force STON on anybody).

But a quick question then, since I know you also know Lisp.

Do you then see array literal syntax as s-expressions ?

Yes, I see them as such and for that I agree with Stephane D. that they may serve to store the meta data. But they are not all that readable and the semantics is not so easy to deduct. However that's a matter of taste.

So that dictionaries/maps are like Lisp plists ?

Yes. And BTW, thank you all for this interesting discussion. We'll see where it leads to. It will help us all to get more insight into the matter of data exchange in a human readable and sustainable way. (i.e. parsers are available in various dialects and languages) --Hannes

Dictionary new at: #x put: 1; at: #y put; 2; yourself.

would then be encoded like

#( x 1 y 2 )

?

Eventually doing a

Dictionary newFromPairs:: #( x 1 y 2 )

?

It would certainly work, with the know limitation that one has to know or understand the semantics upfront.

Sven

On 5/9/12, Göran Krampe <goran@krampe.se> wrote:

Hi all!

First - I am not fighting nor pushing in any direction. I just find the subject interesting.

On 05/08/2012 10:59 PM, Stéphane Ducasse wrote:

...

...

...

:) there is no security problem just that gemstone does not have a parser. Because parsing cannot be a security hole this is blind data interpretation that is a problem. This is why Github got problems and I imagine that the code was probably passed in JSON or similar format.

Mmm, well, if you rely on full Compiler to parse literals, it is a security problem. But if you use the technique Igor showed to only parse array literals - then yes, that's safe, I agree. I forgot about his post.

Parsing and compiling + execution are three different actions. Parsing: stream -> tokens* but you know it.

Of course. What I mean is that depending on how you "parse" a Smalltalk literal you may or may not be susceptible to a security hole. If you just blindly use the full Compiler to do the whole thing - then you have a hole. If you do not, then you don't. Igor showed a neat way to only use the literal array part of the parser - fine, no hole. But if you just use full Compiler - not fine.

[SNIP]

...

...

My point is that it would be nice if we actually had proper syntax for Associations (and or Dictionaries).

OK I see. But what I want to say is that then this is not a literal array anymore. Literal means that you only need a parser and no execution to get the information. Parse time.

But Stephane... please *read* what I write in the article. Let me repeat:

#-> is a message in Smalltalk, not a literal syntax. We all know that.

BUT... there is nothing preventing us from adding support in the Smalltalk parser to handle this as a *literal* syntax that creates an Association. Tirade has support for it and a large part of the article talks about this, but I presume you have not read it.

...

Now I would like to know the use case for a dictionary syntax or association.

There are TONS of use cases. The whole success behind JSON is IMHO based on the fact that it has explicit syntax for *both* arrays and dictionaries (called "objects" in JavaScript). Nested structures of these two building blocks is a very potent combination. And Smalltalk literal arrays only give us ONE of them.

...

May be you mean that you want to have the equivalent of { } but for dictionary, why not.

Sigh. "Maybe"? Ok, things would be MUCH simpler if you actually read the article you know. And no, not really, because {} is a *dynamic* array with expressions in it and I am talking about (primarily) *literal* dictionaries.

...

Having a syntax for a syntax is not really exciting.

What?

...

Now I think that the key problem is dealing with reference between elements. for that STON can be better because why stopping at dictionary? But then the next question if we go down this road is why don't we add this syntax to smalltalk so that we can create literals objects like in Javascript. I mean without message send because this is exactly that no? What is the difference between Dictionary new add: #flop put: 33 ;yourself and {|% flop : 33 %|}

or Person { name : 'stef' }

The difference? Well, the first is an imperative Smalltalk program (fully insecure of course since I presume you would parse and execute it as full Smalltalk) and the second two are declarative literal syntaxes that are safe to parse.

...

then this is not really Smalltalk anymore and we cannot say that we only have messages and the parser and …. have to be changed…. This is why I asked what is the use case.

Your reasoning is very... confusing. What do you mean "only have messages" - we *already* have a whole bunch of literal syntax for creating objects. What makes a Dictionary so special compared to Arrays?

And "not really Smalltalk anymore"... ehm, I give up.

Anyway, my article mainly says that Smalltalk literals go a long way BUT without the support for Dictionaries they fall short of JSON, and that is a bit of a shame I think. But still - the article was mainly food for thought - I am not fighting a battle here, I am just exploring.

...

Finally and I still did not get a clear answer: why there is a need for dictionary syntax to store method meta data.

Can you please stop referring to "method meta data" all the time? This thread is about an article I wrote that you haven't read and it has nothing to do with that use case. Ok? I agree but the discussion started out from the question how to do meta data exchange. Dale H. uses JSON for that at the moment.

I do recommend to everybody to actually _read_ the article http://goran.krampe.se/2012/05/08/literal-arrays-vs-json-vs-ston-vs-tirade/. In the context of meta data exchange I agree with Stephane D. that literal arrays (something like s-expressions) can do the job; however the syntax is not well defined across dialects and as Dale points out with the nails and screws comparison it might not be all that user friendly. In terms of general data exchange I value this discussion of JSON, STON and Tirade. It is very useful for my work. So we actually have two use cases 1) exchange of meta data 2) exchange of general data with literal objects. This leads to the discussion: How are object literals done in Smalltalk? --Hannes

regards, Göran

Hi Stephane and all! On 05/09/2012 08:29 AM, Stéphane Ducasse wrote:

...

...

Parsing and compiling + execution are three different actions. Parsing: stream -> tokens* but you know it.

Of course. What I mean is that depending on how you "parse" a Smalltalk literal you may or may not be susceptible to a security hole. If you just blindly use the full Compiler to do the whole thing - then you have a hole. If you do not, then you don't. Igor showed a neat way to only use the literal array part of the parser - fine, no hole. But if you just use full Compiler - not fine.

yes so using literal array does not bring any security problems. Or we should say stupidly using the compiler introduces security problems and so far I never advocate for that.

Good, agreement. :)

...

[SNIP]

...

...

My point is that it would be nice if we actually had proper syntax for Associations (and or Dictionaries).

OK I see. But what I want to say is that then this is not a literal array anymore. Literal means that you only need a parser and no execution to get the information. Parse time.

But Stephane... please *read* what I write in the article. Let me repeat:

#-> is a message in Smalltalk, not a literal syntax.

Not in my compiler :)

...then why did you write "then this is not a literal array anymore". I was presuming that you meant "because => is not a literal syntax, it is a message". But perhaps you meant something else?

...

We all know that.

BUT... there is nothing preventing us from adding support in the Smalltalk parser to handle this as a *literal* syntax that creates an Association.

and what else?

So people want ANSI compatibility and at the same time new syntax

Ok, regarding changing the syntax of Smalltalk: - Personally I don't care that much for ANSI. It's dead. I think Smalltalk needs to evolve and bloody hell, you have *Traits* in Pharo, that is a HUGE difference compared to a bit of literal changes. ;) - If we *do* decide to "Hey, let's take a look at our literal syntax", then we should probably see if we can harmonize a bit. Again, see my article for some thoughts in that area. I agree that Smalltalk literals are quite powerful but I think they can be improved.

...

Tirade has support for it and a large part of the article talks about this, but I presume you have not read it.

...

Now I would like to know the use case for a dictionary syntax or association.

There are TONS of use cases. The whole success behind JSON is IMHO based on the fact that it has explicit syntax for *both* arrays and dictionaries (called "objects" in JavaScript). Nested structures of these two building blocks is a very potent combination. And Smalltalk literal arrays only give us ONE of them.

Not only arrays and dictionaries. Just plain objects and this is a key difference!!!!! Dictionaries are not objects in Smalltalk so adding dictionary looks to me like a patch.

I don't follow. JavaScript has literal syntax for Arrays and Objects. And yes, "Dictionaries" and Objects are the same in js, although it's limited to only string keys etc. In Smalltalk we have literal syntax for Arrays (three different ones even) but none for Dictionaries. Why do you say that adding syntax for Dictionaries looks like a "patch" but having syntax for Arrays is perfectly fine? What is the distinguishing feature of Array that makes them so different?

...

...

May be you mean that you want to have the equivalent of { } but for dictionary, why not.

Sigh. "Maybe"? Ok, things would be MUCH simpler if you actually read the article you know. And no, not really, because {} is a *dynamic* array with expressions in it and I am talking about (primarily) *literal* dictionaries.

ok I was talking about the syntactic aspect of having { } instead of writing Array new: because this is what you want {|% instead of Dictionary newFrom:

How do you know "what I want" when you don't read the article? Or have you read it now?

...

...

Now I think that the key problem is dealing with reference between elements. for that STON can be better because why stopping at dictionary? But then the next question if we go down this road is why don't we add this syntax to smalltalk so that we can create literals objects like in Javascript. I mean without message send because this is exactly that no? What is the difference between Dictionary new add: #flop put: 33 ;yourself and {|% flop : 33 %|}

or Person { name : 'stef' }

The difference? Well, the first is an imperative Smalltalk program (fully insecure of course since I presume you would parse and execute it as full Smalltalk) and the second two are declarative literal syntaxes that are safe to parse.

You could also say quite limited because you can only put dictionary inside dictionary and literals. so no objects.

Sure, limited. But that is the whole point of literals, that they are limited. And in the case of Smalltalk of course - that they are actually evaluated and constructed at *compile time* and not at run time, that is also a very important aspect.

but in that case why don't we use literal syntax for objects? Why stopping at Dictionary.

This kind of argument is not fruitful.

If you ask me I would remove #() as a syntactic elements because most of the time { } is what you want.

Well... The fact that #() are constructed at compile time is a BIG difference. So I still think both constructs are useful.

...

...

then this is not really Smalltalk anymore and we cannot say that we only have messages and the parser and …. have to be changed…. This is why I asked what is the use case.

Your reasoning is very... confusing. What do you mean "only have messages" - we *already* have a whole bunch of literal syntax for creating objects. What makes a Dictionary so special compared to Arrays?

so Dictionary and Set and why not objects….

Again, not fruitful.

...

And "not really Smalltalk anymore"... ehm, I give up.

Anyway, my article mainly says that Smalltalk literals go a long way BUT without the support for Dictionaries they fall short of JSON, and that is a bit of a shame I think. But still - the article was mainly food for thought - I am not fighting a battle here, I am just exploring.

...

Finally and I still did not get a clear answer: why there is a need for dictionary syntax to store method meta data.

Can you please stop referring to "method meta data" all the time? This thread is about an article I wrote that you haven't read and it has nothing to do with that use case. Ok?

Because I do not care if you want to use STON, JSON XML for your own program this is your choice. I care that I will have to load JSON, STON in core to load or save code and that I have to use a different syntax to look at code in Git.

I know, but I/we are not discussing that here. Ok? I know it was the seed of all these threads but THIS thread is not about that. For me personally this sums it up fairly well: - It would be nice with a strong, heavily supported fast and slick implementation of JSON. If STON can be that (superset of JSON) then I support Sven all the way to the moon! For Smalltalk source code meta data? No, personally I would also prefer literals I guess. Or Tirade :) - Tirade covers a different spectra. People get stuck on its name, for whatever reason, call it "Cascade" if it makes you happy. I think it is quite interesting in its capabilities and if anyone wants to help me evolve it more, please do. It was after all created because I felt the need for a good format for Deltas and JSON failed the test (strings in JSON can't have linefeeds!!!). - Smalltalk should not be set in stone. The literal syntax of Smalltalk has a bit of "warts" and it would be awesome if Pharo (with all its momentum) could go forward and say "Hey, let's take another step forward!" and try to bring it up a notch. That would involve both perhaps simplifying, harmonizing and making it a tad more powerful. This last part is something that got evident to me after working with Tirade (felt the lack of Associations there) and later with Amber (which also felt a strong lacking in the same area when trying to mix with js). I will try to write some kind of separate posting on that subject, and see if it "sticks to the wall". Now... Stef - I like you a lot and if you want to discuss this last part constructively I am all game. But hey, if you just want to shoot down everything we say, without even giving it a second thought or (gasp!) taking the time to read - then I do have better things to do. regards, Göran

I would like to share my quick two cents regarding this subject. 1st cent. I would stick with JSON. Even though I find STON slick, I would not deviate from something that is mainstream and all Smalltalk dialects already support. 2nd cent. A Dictionary literal syntax would be useful too. I would like to have it. It could be something like #{'this' -> 1. 'that' -> 2 }, and keep {1. 2} for dynamic arrays. Regards, -- Esteban. -- View this message in context: http://forum.world.st/ANN-STON-Smalltalk-Object-Notation-tp4609808p4620499.h... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Göran Krampe wrote

I don't want to sound like a broken record - but you do realize that I wrote about this in the article and mentioned this exact syntax?

I know, the article was a tad long - but just go to planet.smalltalk.org and search down to "Sidestory: Adding literal Dictionaries to Smalltalk?".

regards, Göran

I do realize, and read the entire post. I simply didn't want to just say "+1" to everything you state in the blog post, and did a summary of what's interesting for me. :) Regards! -- View this message in context: http://forum.world.st/ANN-STON-Smalltalk-Object-Notation-tp4609808p4620638.h... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

On 5/12/12, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:

got three days meeting this week so crawling to get my brain aligned with fun again… I will read your article. I will also look at STON because I like the object literal syntax too :). Fine...!

(note that I'm not changing my mind - I still do not get why I need dictionary for method meta data -

I agree; just for method meta data it is not needed. I just checked it in Squeak 4.3, Pharo 1.4, Cuis 4.0 and Amber {#foo -> 1. #bar -> 42} as: Dictionary. "fine for Squeak, Pharo, Cuis" {#foo -> 1. #bar -> 42} gives an Array in Amber; #as: is not understood But a literal array parser available in many dialects is still needed then. For the curly brackets syntax for dynamic array there was some discussion this week on the squeak list how it is used. The decision by Dale and others to go for JSON was driven by the availability of a parser for many Smalltalk dialects.

but we are thinking (and the team should discuss for real and not just during lunches) that having a literal object syntax would be cool).

Great! --Hannes

Stef

...

...

I would like to share my quick two cents regarding this subject.

1st cent. I would stick with JSON. Even though I find STON slick, I would not deviate from something that is mainstream and all Smalltalk dialects already support.

2nd cent. A Dictionary literal syntax would be useful too. I would like to have it.

It could be something like #{'this' -> 1. 'that' -> 2 }, and keep {1. 2} for dynamic arrays.

I don't want to sound like a broken record - but you do realize that I wrote about this in the article and mentioned this exact syntax?

I know, the article was a tad long - but just go to planet.smalltalk.org and search down to "Sidestory: Adding literal Dictionaries to Smalltalk?".

regards, Göran

I will read your article I missed it originally and after I was running to catch my train.