[Pharo-project] SimpleLiteralArray Spec
OK, just to get started, let try to write a spec for SimpleLiteralArray, this is based on JSON (and my own STON, not ready for public consumption). It will soon become clear that this is slightly harder than expected, IMO a writer is needed as well. S y n t a x value primitive-value list nil primitive-value number true false symbol string classname list #() #( elements ) elements value value , elements string '' ' chars ' chars char char chars char any-printable-ASCII-character- except-"-or-\ \" \\ \/ \b \f \n \r \t \u four-hex-digits symbol # chars-limited # ' chars ' chars-limited char-limited char-limited chars-limited char-limited a-z A-Z 0-9 - _ . / classname uppercase-alpha-char alphanumeric-char number int int frac int exp int frac exp int digit digit1-9 digits - digit - digit1-9 digits frac . digits exp e digits digits digit digit digits e e e+ e- E E+ E- Questions remain: - is the escaping OK ? - what about symbols without a leading # ? - what should be allowed in unquoted symbols ? - what should be allowed for classnames ? - what about namespaces ? - do we allow infinite precision integers ? - maybe the toplevel should always be an array ? Now: shoot! Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
Sven, I know that having the parser produce dictionaries (i.e., dictionary _syntax_) is very convenient for the consumer of the parsed output ... When javascript evaluates/parses JSON, you will note that you get real objects back instead of dictionaries and you can send messages to those objects, so I would think that rather than inventing a dictionary syntax, you invent a STON-object syntax then we can then send setter/getter style messages to these boys... then we'd have _objects_, arrays, strings, numbers and booleans for the STON format ... a field could be optionally included in the STON object called _class_ which would optionally allow the parser to produce instances of that class with all of the fields filled in ... JSON on steroids! Anyway ... just a thought. Dale ----- Original Message ----- | From: "Sven Van Caekenberghe" <sven@beta9.be> | To: "Pharo Development" <Pharo-project@lists.gforge.inria.fr> | Sent: Tuesday, April 24, 2012 4:26:12 AM | Subject: [Pharo-project] SimpleLiteralArray Spec | | OK, just to get started, let try to write a spec for | SimpleLiteralArray, this is based on JSON (and my own STON, not | ready for public consumption). It will soon become clear that this | is slightly harder than expected, IMO a writer is needed as well. | | S y n t a x | | value | primitive-value | list | nil | primitive-value | number | true | false | symbol | string | classname | list | #() | #( elements ) | elements | value | value , elements | string | '' | ' chars ' | chars | char | char chars | char | any-printable-ASCII-character- | except-"-or-\ | \" | \\ | \/ | \b | \f | \n | \r | \t | \u four-hex-digits | symbol | # chars-limited | # ' chars ' | chars-limited | char-limited | char-limited chars-limited | char-limited | a-z A-Z 0-9 - _ . / | classname | uppercase-alpha-char alphanumeric-char | number | int | int frac | int exp | int frac exp | int | digit | digit1-9 digits | - digit | - digit1-9 digits | frac | . digits | exp | e digits | digits | digit | digit digits | e | e | e+ | e- | E | E+ | E- | | | Questions remain: | | - is the escaping OK ? | - what about symbols without a leading # ? | - what should be allowed in unquoted symbols ? | - what should be allowed for classnames ? | - what about namespaces ? | - do we allow infinite precision integers ? | - maybe the toplevel should always be an array ? | | | Now: shoot! | | | Sven | | | -- | Sven Van Caekenberghe | http://stfx.eu | Smalltalk is the Red Pill | | | | | |
What is STON and STON-object syntax? Norbert Am 24.04.2012 um 18:18 schrieb Dale Henrichs:
Sven,
I know that having the parser produce dictionaries (i.e., dictionary _syntax_) is very convenient for the consumer of the parsed output ...
When javascript evaluates/parses JSON, you will note that you get real objects back instead of dictionaries and you can send messages to those objects, so I would think that rather than inventing a dictionary syntax, you invent a STON-object syntax then we can then send setter/getter style messages to these boys... then we'd have _objects_, arrays, strings, numbers and booleans for the STON format ... a field could be optionally included in the STON object called _class_ which would optionally allow the parser to produce instances of that class with all of the fields filled in ...
JSON on steroids!
Anyway ... just a thought.
Dale
----- Original Message ----- | From: "Sven Van Caekenberghe" <sven@beta9.be> | To: "Pharo Development" <Pharo-project@lists.gforge.inria.fr> | Sent: Tuesday, April 24, 2012 4:26:12 AM | Subject: [Pharo-project] SimpleLiteralArray Spec | | OK, just to get started, let try to write a spec for | SimpleLiteralArray, this is based on JSON (and my own STON, not | ready for public consumption). It will soon become clear that this | is slightly harder than expected, IMO a writer is needed as well. | | S y n t a x | | value | primitive-value | list | nil | primitive-value | number | true | false | symbol | string | classname | list | #() | #( elements ) | elements | value | value , elements | string | '' | ' chars ' | chars | char | char chars | char | any-printable-ASCII-character- | except-"-or-\ | \" | \\ | \/ | \b | \f | \n | \r | \t | \u four-hex-digits | symbol | # chars-limited | # ' chars ' | chars-limited | char-limited | char-limited chars-limited | char-limited | a-z A-Z 0-9 - _ . / | classname | uppercase-alpha-char alphanumeric-char | number | int | int frac | int exp | int frac exp | int | digit | digit1-9 digits | - digit | - digit1-9 digits | frac | . digits | exp | e digits | digits | digit | digit digits | e | e | e+ | e- | E | E+ | E- | | | Questions remain: | | - is the escaping OK ? | - what about symbols without a leading # ? | - what should be allowed in unquoted symbols ? | - what should be allowed for classnames ? | - what about namespaces ? | - do we allow infinite precision integers ? | - maybe the toplevel should always be an array ? | | | Now: shoot! | | | Sven | | | -- | Sven Van Caekenberghe | http://stfx.eu | Smalltalk is the Red Pill | | | | | |
Norbert Hartl wrote
What is STON and STON-object syntax?
Smalltalk Object Notation? http://ss3.gemstone.com/ss/STON.html -- View this message in context: http://forum.world.st/SimpleLiteralArray-Spec-tp4583109p4584177.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Norbert, Dale, et al, On 24 Apr 2012, at 18:43, Norbert Hartl wrote:
What is STON and STON-object syntax?
Arhh, it is not yet finished, I did it for myself, not for trying to convince the world, since that it too hard anyway. This is the work-in-progress paper: The code lives here: http://ss3.gemstone.com/ss/STON It is a slight variation of JSON, adding some Smalltalk concepts, as well as a solution to the class problem. But again, I am not trying to convince anyone to use this, I did it as a proof of concept for myself, and I kind of like it. On the other hand, JSON is so ubiquitous that I wonder a bit, why bother ? Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
Sven, I looked at your sample document in the pdf: TestDomainObject { #created : DateAndTime [ '2012-02-14T16:40:15+01:00' ], #modified : DateAndTime [ '2012-02-14T16:40:18+01:00' ], #integer : 39581, #float : 73.84789359463944, #description : 'This is a test', #color : #green, #tags : [ #two, #beta, #medium ], #bytes : ByteArray [ 'afabfdf61d030f43eb67960c0ae9f39f' ], #boolean : false } And this is very readable (to me)... not as many '[]' as I inferred from reading the tests:). I take back my YAML comparison ... this has the nice flavor of Smalltalk ... Dale ----- Original Message ----- | From: "Sven Van Caekenberghe" <sven@beta9.be> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 10:06:22 AM | Subject: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | Norbert, Dale, et al, | | | On 24 Apr 2012, at 18:43, Norbert Hartl wrote: | | > What is STON and STON-object syntax? | | Arhh, it is not yet finished, I did it for myself, not for trying to | convince the world, since that it too hard anyway. | | This is the work-in-progress paper: | | | | | The code lives here: http://ss3.gemstone.com/ss/STON | | It is a slight variation of JSON, adding some Smalltalk concepts, as | well as a solution to the class problem. | | But again, I am not trying to convince anyone to use this, I did it | as a proof of concept for myself, and I kind of like it. | | On the other hand, JSON is so ubiquitous that I wonder a bit, why | bother ? | | Sven | | | -- | Sven Van Caekenberghe | http://stfx.eu | Smalltalk is the Red Pill | | | | |
Sven, For tODE I am very interested having something with this kind of potential ... shipping "real" ST objects back and forth between Amber and Pharo or GemStone has some real appeal... So Goran, the STON format looks somewhat similar to Tirade ... what would Tirade look like for the same data set? Dale ----- Original Message ----- | From: "Dale Henrichs" <dhenrich@vmware.com> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 10:22:36 AM | Subject: Re: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | Sven, | | I looked at your sample document in the pdf: | | TestDomainObject { | #created : DateAndTime [ '2012-02-14T16:40:15+01:00' ], | #modified : DateAndTime [ '2012-02-14T16:40:18+01:00' ], | #integer : 39581, | #float : 73.84789359463944, | #description : 'This is a test', | #color : #green, | #tags : [ | #two, | #beta, | #medium | ], | #bytes : ByteArray [ 'afabfdf61d030f43eb67960c0ae9f39f' ], | #boolean : false | } | | And this is very readable (to me)... not as many '[]' as I inferred | from reading the tests:). | | I take back my YAML comparison ... this has the nice flavor of | Smalltalk ... | | Dale | | ----- Original Message ----- | | From: "Sven Van Caekenberghe" <sven@beta9.be> | | To: Pharo-project@lists.gforge.inria.fr | | Sent: Tuesday, April 24, 2012 10:06:22 AM | | Subject: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | | | Norbert, Dale, et al, | | | | | | On 24 Apr 2012, at 18:43, Norbert Hartl wrote: | | | | > What is STON and STON-object syntax? | | | | Arhh, it is not yet finished, I did it for myself, not for trying | | to | | convince the world, since that it too hard anyway. | | | | This is the work-in-progress paper: | | | | | | | | | | The code lives here: http://ss3.gemstone.com/ss/STON | | | | It is a slight variation of JSON, adding some Smalltalk concepts, | | as | | well as a solution to the class problem. | | | | But again, I am not trying to convince anyone to use this, I did it | | as a proof of concept for myself, and I kind of like it. | | | | On the other hand, JSON is so ubiquitous that I wonder a bit, why | | bother ? | | | | Sven | | | | | | -- | | Sven Van Caekenberghe | | http://stfx.eu | | Smalltalk is the Red Pill | | | | | | | | | | |
On 04/24/2012 07:28 PM, Dale Henrichs wrote:
Sven,
For tODE I am very interested having something with this kind of potential ... shipping "real" ST objects back and forth between Amber and Pharo or GemStone has some real appeal...
So Goran, the STON format looks somewhat similar to Tirade ... what would Tirade look like for the same data set?
Hmmm, let's see.
| TestDomainObject { | #created : DateAndTime [ '2012-02-14T16:40:15+01:00' ], | #modified : DateAndTime [ '2012-02-14T16:40:18+01:00' ], | #integer : 39581, | #float : 73.84789359463944, | #description : 'This is a test', | #color : #green, | #tags : [ | #two, | #beta, | #medium | ], | #bytes : ByteArray [ 'afabfdf61d030f43eb67960c0ae9f39f' ], | #boolean : false | }
It is an interesting question since Tirade is not meant to be parsed by a generic parser like JSON or STON. Tirade is a series of Smalltalk messages (unary or keyword) with arguments limited to literals (although with some subtle additions to regular Smalltalk literals, like for example Associations). These messages give Tirade semantics and thus we do not normally add "types" to the Tirade input. So the above could be encoded as a single Tirade message like this: created: '2012-02-14T16:40:15+01:00' modified: '2012-02-14T16:40:18+01:00' integer: 39581 float: 73.84789359463944 description: 'This is a test' color: #green tags: #(two beta medium) bytes: 'afabfdf61d030f43eb67960c0ae9f39f' boolean: false. Note that the above syntactically is a regular Smalltalk keyword message with no receiver to the left and ended with a "." (mandatory in Tirade). Also, the tags could also be represented as {#two . #beta . #medium} but although the syntax is like dynamic arrays in Pharo that does not mean expressions are allowed in Tirade - only literals. Normally there would be a subclass of TiradeReader that implements the above message and then does the appropriate instantiations for #created, #modified and #bytes given the string input for those arguments. This method could look something like this (just pseudo code): created: created modified: modified integer: int float: float description: description color: color tags: tags bytes: bytes boolean: boolean result := TestDomainObject new created: (TimeStamp fromString: created); modified: (TimeStamp fromString: modified). result integer: int; float: float; description: description; color: color; tags: tags; bytes: bytes asByteArray; boolean: boolean If one instead uses a plain TiradeParser to parse the above it will of course not have an implementation of the above method and will instead by default produce an OrderedCollection of "messages" where each message is an Association with the keyword selector as key and an array of arguments as the value. So running TiradeParser>>parse: on the above input gives: an OrderedCollection(#created:modified:integer:float:description:color:tags:bytes:boolean:->#('2012-02-14T16:40:15+01:00' '2012-02-14T16:40:18+01:00' 39581 73.84789359463944 'This is a test' #green #(true #beta #medium) 'afabfdf61d030f43eb67960c0ae9f39f' false)) Finally a totally different approach is to encode the above object as a nested array with Associations, thus mimicking JSON/STON and it could look like this: testdomainobject: { #created -> '2012-02-14T16:40:15+01:00'. #modified -> '2012-02-14T16:40:18+01:00'. #integer -> 39581. #float -> 73.84789359463944. #description -> 'This is a test'. #color -> #green. #tags -> #(two beta medium). #bytes -> 'afabfdf61d030f43eb67960c0ae9f39f'. #boolean -> false }. NOTE: The -> syntax is added to Tirade as a "literal syntax" although it is in fact a binary message in Pharo. This will create an Array with Associations in it. Again, if the reader implements the selector #testdomainobject: it will know what to do with the argument. Again, using only TiradeParser it would parse into: an OrderedCollection(#testdomainobject:->{((Array new: 9) at: 1 put: (#created->'2012-02-14T16:40:15+01:00'); at: 2 put: (#modified->'2012-02-14T16:40:18+01:00'); at: 3 put: (#integer->39581); at: 4 put: (#float->73.84789359463944); at: 5 put: (#description->'This is a test'); at: 6 put: (#color->#green); at: 7 put: (#tags->#(true #beta #medium )); at: 8 put: (#bytes->'afabfdf61d030f43eb67960c0ae9f39f'); at: 9 put: (#boolean->false); yourself)}) Oh, funky. Obviously that is how print-it behaves on an Array of Assocations :) Hopefully this gives a better idea of how Tirade works. Again, the three articles on Tirade from my blog explains more details: http://goran.krampe.se/blog/Squeak/Tirade.rdoc http://goran.krampe.se/blog/Squeak/Tirade2.rdoc http://goran.krampe.se/blog/Squeak/Tirade3.rdoc NOTE: The part on "receiver juggling" was way crazy, forget that :) regards, Göran
Okay, so Tirade isn't a generic parser for a well-defined syntax like STON and it looks like it really shines when it comes to doing dynamic interpretation of streams of messages .... now I better understand how Tirade can be applied to deltas or creating executable logs that can be replayed ... At this point in time for disk-based structures that humans and computers are expected to understand it seems that a well-defined syntax is useful. Dale ----- Original Message ----- | From: "Göran Krampe" <goran@krampe.se> | To: pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 3:56:36 PM | Subject: Re: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | On 04/24/2012 07:28 PM, Dale Henrichs wrote: | > Sven, | > | > For tODE I am very interested having something with this kind of | > potential ... shipping "real" ST objects back and forth between | > Amber and Pharo or GemStone has some real appeal... | > | > So Goran, the STON format looks somewhat similar to Tirade ... what | > would Tirade look like for the same data set? | | Hmmm, let's see. | | > | TestDomainObject { | > | #created : DateAndTime [ '2012-02-14T16:40:15+01:00' ], | > | #modified : DateAndTime [ '2012-02-14T16:40:18+01:00' ], | > | #integer : 39581, | > | #float : 73.84789359463944, | > | #description : 'This is a test', | > | #color : #green, | > | #tags : [ | > | #two, | > | #beta, | > | #medium | > | ], | > | #bytes : ByteArray [ 'afabfdf61d030f43eb67960c0ae9f39f' ], | > | #boolean : false | > | } | | It is an interesting question since Tirade is not meant to be parsed | by | a generic parser like JSON or STON. Tirade is a series of Smalltalk | messages (unary or keyword) with arguments limited to literals | (although | with some subtle additions to regular Smalltalk literals, like for | example Associations). | | These messages give Tirade semantics and thus we do not normally add | "types" to the Tirade input. | | So the above could be encoded as a single Tirade message like this: | | created: '2012-02-14T16:40:15+01:00' | modified: '2012-02-14T16:40:18+01:00' | integer: 39581 | float: 73.84789359463944 | description: 'This is a test' | color: #green | tags: #(two beta medium) | bytes: 'afabfdf61d030f43eb67960c0ae9f39f' | boolean: false. | | Note that the above syntactically is a regular Smalltalk keyword | message | with no receiver to the left and ended with a "." (mandatory in | Tirade). | Also, the tags could also be represented as {#two . #beta . #medium} | but | although the syntax is like dynamic arrays in Pharo that does not | mean | expressions are allowed in Tirade - only literals. | | Normally there would be a subclass of TiradeReader that implements | the | above message and then does the appropriate instantiations for | #created, | #modified and #bytes given the string input for those arguments. This | method could look something like this (just pseudo code): | | created: created | modified: modified | integer: int | float: float | description: description | color: color | tags: tags | bytes: bytes | boolean: boolean | | result := TestDomainObject new | created: (TimeStamp fromString: created); | modified: (TimeStamp fromString: modified). | result integer: int; float: float; description: description; | color: color; tags: tags; | bytes: bytes asByteArray; boolean: boolean | | | If one instead uses a plain TiradeParser to parse the above it will | of | course not have an implementation of the above method and will | instead | by default produce an OrderedCollection of "messages" where each | message | is an Association with the keyword selector as key and an array of | arguments as the value. So running TiradeParser>>parse: on the above | input gives: | | an | OrderedCollection(#created:modified:integer:float:description:color:tags:bytes:boolean:->#('2012-02-14T16:40:15+01:00' | '2012-02-14T16:40:18+01:00' 39581 73.84789359463944 'This is a test' | #green #(true #beta #medium) 'afabfdf61d030f43eb67960c0ae9f39f' | false)) | | Finally a totally different approach is to encode the above object as | a | nested array with Associations, thus mimicking JSON/STON and it could | look like this: | | testdomainobject: { | #created -> '2012-02-14T16:40:15+01:00'. | #modified -> '2012-02-14T16:40:18+01:00'. | #integer -> 39581. | #float -> 73.84789359463944. | #description -> 'This is a test'. | #color -> #green. | #tags -> #(two beta medium). | #bytes -> 'afabfdf61d030f43eb67960c0ae9f39f'. | #boolean -> false | }. | | NOTE: The -> syntax is added to Tirade as a "literal syntax" although | it | is in fact a binary message in Pharo. This will create an Array with | Associations in it. | | Again, if the reader implements the selector #testdomainobject: it | will | know what to do with the argument. Again, using only TiradeParser it | would parse into: | | an OrderedCollection(#testdomainobject:->{((Array new: 9) at: 1 put: | (#created->'2012-02-14T16:40:15+01:00'); at: 2 put: | (#modified->'2012-02-14T16:40:18+01:00'); at: 3 put: | (#integer->39581); | at: 4 put: (#float->73.84789359463944); at: 5 put: | (#description->'This | is a test'); at: 6 put: (#color->#green); at: 7 put: (#tags->#(true | #beta #medium )); at: 8 put: | (#bytes->'afabfdf61d030f43eb67960c0ae9f39f'); at: 9 put: | (#boolean->false); yourself)}) | | Oh, funky. Obviously that is how print-it behaves on an Array of | Assocations :) | | Hopefully this gives a better idea of how Tirade works. Again, the | three | articles on Tirade from my blog explains more details: | | http://goran.krampe.se/blog/Squeak/Tirade.rdoc | http://goran.krampe.se/blog/Squeak/Tirade2.rdoc | http://goran.krampe.se/blog/Squeak/Tirade3.rdoc | | NOTE: The part on "receiver juggling" was way crazy, forget that :) | | regards, Göran | |
On 04/25/2012 01:11 AM, Dale Henrichs wrote:
Okay, so Tirade isn't a generic parser for a well-defined syntax like STON
Well... :) I am not sure if you read my whole post but it does do nested arrays and associatons etc so it seems to match STON fairly well, if we disregard the class mechanism of STON. But yes, it is primarily constructed as a "serializer for Smalltalk messages" and not as a "serializer for Smalltalk objects".
and it looks like it really shines when it comes to doing dynamic interpretation of streams of messages ....
Right.
now I better understand how Tirade can be applied to deltas or creating executable logs that can be replayed ...
Yes, although in DeltaStreams I do not use it like that - I use it to serialize objects, but Deltas consists of lots of different Change objects and each such Change object is serialized using a specific Tirade message. Thus the reader simply implements all those messages and creates the appropriate objects. One *very* nice aspect of this is that we do not encode classes in the actual files, the reader decides which class to instantiate. Thus the files are much more robust against change. In that sense I am not that fond of putting classes explicitly in the STON file.
At this point in time for disk-based structures that humans and computers are expected to understand it seems that a well-defined syntax is useful.
Dale
Sure. And at least it was fun to explain a bit about Tirade. And I ended up adding support for literal arrays :) regards, Göran
Agreed, I think there are valid use cases for both STON and Tirade ... there is some overlap and where there is overlap one or the other can be used ... then again each has some unique characteristics ... I'm glad that I've learned more about Tirade in the process, because it is much clearer to me where Tirade can be applied ... Dale ----- Original Message ----- | From: "Göran Krampe" <goran@krampe.se> | To: pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 5:17:26 PM | Subject: Re: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | On 04/25/2012 01:11 AM, Dale Henrichs wrote: | > Okay, so Tirade isn't a generic parser for a well-defined syntax | > like STON | | Well... :) I am not sure if you read my whole post but it does do | nested | arrays and associatons etc so it seems to match STON fairly well, if | we | disregard the class mechanism of STON. | | But yes, it is primarily constructed as a "serializer for Smalltalk | messages" and not as a "serializer for Smalltalk objects". | | > and it looks like it really shines when it comes to doing dynamic | > interpretation of streams of messages .... | | Right. | | > now I better understand how Tirade can be applied to deltas or | > creating executable logs that can be replayed ... | | Yes, although in DeltaStreams I do not use it like that - I use it to | serialize objects, but Deltas consists of lots of different Change | objects and each such Change object is serialized using a specific | Tirade message. Thus the reader simply implements all those messages | and | creates the appropriate objects. | | One *very* nice aspect of this is that we do not encode classes in | the | actual files, the reader decides which class to instantiate. Thus the | files are much more robust against change. | | In that sense I am not that fond of putting classes explicitly in the | STON file. | | > At this point in time for disk-based structures that humans and | > computers are expected to understand it seems that a well-defined | > syntax is useful. | > | > Dale | | Sure. And at least it was fun to explain a bit about Tirade. And I | ended | up adding support for literal arrays :) | | regards, Göran | |
Göran Thank you for this illustration. http://goran.krampe.se/blog/Squeak/Tirade.rdoc I understand that 1) the Tirade parser parses a subset of regular Smalltalk messages. 2) the receiver is not given as it is assumed to be always the Tirade parser object. 3) the Tirade parser acts like a builder for objects? 4) It solves problems JSON and STON do not. I think this idea is very worth to pursue. --Hannes On 4/25/12, Göran Krampe <goran@krampe.se> wrote:
On 04/24/2012 07:28 PM, Dale Henrichs wrote:
Sven,
For tODE I am very interested having something with this kind of potential ... shipping "real" ST objects back and forth between Amber and Pharo or GemStone has some real appeal...
So Goran, the STON format looks somewhat similar to Tirade ... what would Tirade look like for the same data set?
Hmmm, let's see.
| TestDomainObject { | #created : DateAndTime [ '2012-02-14T16:40:15+01:00' ], | #modified : DateAndTime [ '2012-02-14T16:40:18+01:00' ], | #integer : 39581, | #float : 73.84789359463944, | #description : 'This is a test', | #color : #green, | #tags : [ | #two, | #beta, | #medium | ], | #bytes : ByteArray [ 'afabfdf61d030f43eb67960c0ae9f39f' ], | #boolean : false | }
It is an interesting question since Tirade is not meant to be parsed by a generic parser like JSON or STON. Tirade is a series of Smalltalk messages (unary or keyword) with arguments limited to literals (although with some subtle additions to regular Smalltalk literals, like for example Associations).
These messages give Tirade semantics and thus we do not normally add "types" to the Tirade input.
So the above could be encoded as a single Tirade message like this:
created: '2012-02-14T16:40:15+01:00' modified: '2012-02-14T16:40:18+01:00' integer: 39581 float: 73.84789359463944 description: 'This is a test' color: #green tags: #(two beta medium) bytes: 'afabfdf61d030f43eb67960c0ae9f39f' boolean: false.
Note that the above syntactically is a regular Smalltalk keyword message with no receiver to the left and ended with a "." (mandatory in Tirade). Also, the tags could also be represented as {#two . #beta . #medium} but although the syntax is like dynamic arrays in Pharo that does not mean expressions are allowed in Tirade - only literals.
Normally there would be a subclass of TiradeReader that implements the above message and then does the appropriate instantiations for #created, #modified and #bytes given the string input for those arguments. This method could look something like this (just pseudo code):
created: created modified: modified integer: int float: float description: description color: color tags: tags bytes: bytes boolean: boolean
result := TestDomainObject new created: (TimeStamp fromString: created); modified: (TimeStamp fromString: modified). result integer: int; float: float; description: description; color: color; tags: tags; bytes: bytes asByteArray; boolean: boolean
If one instead uses a plain TiradeParser to parse the above it will of course not have an implementation of the above method and will instead by default produce an OrderedCollection of "messages" where each message is an Association with the keyword selector as key and an array of arguments as the value. So running TiradeParser>>parse: on the above input gives:
an OrderedCollection(#created:modified:integer:float:description:color:tags:bytes:boolean:->#('2012-02-14T16:40:15+01:00' '2012-02-14T16:40:18+01:00' 39581 73.84789359463944 'This is a test' #green #(true #beta #medium) 'afabfdf61d030f43eb67960c0ae9f39f' false))
Finally a totally different approach is to encode the above object as a nested array with Associations, thus mimicking JSON/STON and it could look like this:
testdomainobject: { #created -> '2012-02-14T16:40:15+01:00'. #modified -> '2012-02-14T16:40:18+01:00'. #integer -> 39581. #float -> 73.84789359463944. #description -> 'This is a test'. #color -> #green. #tags -> #(two beta medium). #bytes -> 'afabfdf61d030f43eb67960c0ae9f39f'. #boolean -> false }.
NOTE: The -> syntax is added to Tirade as a "literal syntax" although it is in fact a binary message in Pharo. This will create an Array with Associations in it.
Again, if the reader implements the selector #testdomainobject: it will know what to do with the argument. Again, using only TiradeParser it would parse into:
an OrderedCollection(#testdomainobject:->{((Array new: 9) at: 1 put: (#created->'2012-02-14T16:40:15+01:00'); at: 2 put: (#modified->'2012-02-14T16:40:18+01:00'); at: 3 put: (#integer->39581); at: 4 put: (#float->73.84789359463944); at: 5 put: (#description->'This is a test'); at: 6 put: (#color->#green); at: 7 put: (#tags->#(true #beta #medium )); at: 8 put: (#bytes->'afabfdf61d030f43eb67960c0ae9f39f'); at: 9 put: (#boolean->false); yourself)})
Oh, funky. Obviously that is how print-it behaves on an Array of Assocations :)
Hopefully this gives a better idea of how Tirade works. Again, the three articles on Tirade from my blog explains more details:
http://goran.krampe.se/blog/Squeak/Tirade.rdoc http://goran.krampe.se/blog/Squeak/Tirade2.rdoc http://goran.krampe.se/blog/Squeak/Tirade3.rdoc
NOTE: The part on "receiver juggling" was way crazy, forget that :)
regards, Göran
On 04/25/2012 01:25 AM, H. Hirzel wrote:
Göran
Thank you for this illustration.
http://goran.krampe.se/blog/Squeak/Tirade.rdoc
I understand that
1) the Tirade parser parses a subset of regular Smalltalk messages.
Exactly. It doesn't do binary messages. And of course, it doesn't do any expressions.
2) the receiver is not given as it is assumed to be always the Tirade parser object.
Right, but if you use the TiradeReader (which is a bit more experimental) it maintains a stack of objects and will send the next Tirade message to the object on the top of the stack. Then one can register a set of "control messages" and those are always sent to the object first on the stack - which is the TiradeReader itself. As I said, this is an experiment on a generic mechanism for being able to construct complex objects from a series of Tirade messages.
3) the Tirade parser acts like a builder for objects?
The parser actually only parses the Tirade messages, it doesn't try to build anything. But your own subclass of it should implement these messages and typically build something.
4) It solves problems JSON and STON do not.
I think this idea is very worth to pursue.
The elegancy of messaging is quite powerful and makes it easier to construct a file format that is less fragile and has very little structure. regards, Göran
On 24 Apr 2012, at 19:22, Dale Henrichs wrote:
Sven,
I looked at your sample document in the pdf:
TestDomainObject { #created : DateAndTime [ '2012-02-14T16:40:15+01:00' ], #modified : DateAndTime [ '2012-02-14T16:40:18+01:00' ], #integer : 39581, #float : 73.84789359463944, #description : 'This is a test', #color : #green, #tags : [ #two, #beta, #medium ], #bytes : ByteArray [ 'afabfdf61d030f43eb67960c0ae9f39f' ], #boolean : false }
And this is very readable (to me)... not as many '[]' as I inferred from reading the tests:).
I take back my YAML comparison ... this has the nice flavor of Smalltalk ...
Dale
Thanks! The main 'hack' is to extend JSON's list/array [] and map/dictionary {} with a very simple way to instanciate any class, by preceding them with a class name (or alias) and giving the control to instanciate to the class or instance. Furthermore, for Array and Dictionary these are optional. For the concrete representations choosen for say DateAndTime or ByteArray, human readability was the deciding factor. But any of these could be changed without hurting the concept. Note that there are also references and implicit object counting and graph tracing with a defined visit order. In a discussion with Mariano I kind of decided to no longer treat Strings as shareable/referenceable objects. This is not yet reflected in the document but it is in the code. Sven
Sven, Just an off-the-wall thought... If JSON were a subset of STON ... or possibly STON would recognize JSON and parse despite the different syntax...we'd have a nice multipurpose parser/writer that could speak JSON or STON interchangeably... Dale ----- Original Message ----- | From: "Sven Van Caekenberghe" <sven@beta9.be> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 10:35:49 AM | Subject: Re: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | | On 24 Apr 2012, at 19:22, Dale Henrichs wrote: | | > Sven, | > | > I looked at your sample document in the pdf: | > | > TestDomainObject { | > #created : DateAndTime [ '2012-02-14T16:40:15+01:00' ], | > #modified : DateAndTime [ '2012-02-14T16:40:18+01:00' ], | > #integer : 39581, | > #float : 73.84789359463944, | > #description : 'This is a test', | > #color : #green, | > #tags : [ | > #two, | > #beta, | > #medium | > ], | > #bytes : ByteArray [ 'afabfdf61d030f43eb67960c0ae9f39f' ], | > #boolean : false | > } | > | > And this is very readable (to me)... not as many '[]' as I inferred | > from reading the tests:). | > | > I take back my YAML comparison ... this has the nice flavor of | > Smalltalk ... | > | > Dale | | Thanks! | | The main 'hack' is to extend JSON's list/array [] and map/dictionary | {} with a very simple way to instanciate any class, by preceding | them with a class name (or alias) and giving the control to | instanciate to the class or instance. Furthermore, for Array and | Dictionary these are optional. For the concrete representations | choosen for say DateAndTime or ByteArray, human readability was the | deciding factor. But any of these could be changed without hurting | the concept. | | Note that there are also references and implicit object counting and | graph tracing with a defined visit order. In a discussion with | Mariano I kind of decided to no longer treat Strings as | shareable/referenceable objects. This is not yet reflected in the | document but it is in the code. | | Sven |
On 24 Apr 2012, at 21:05, Dale Henrichs wrote:
Sven,
Just an off-the-wall thought... If JSON were a subset of STON ... or possibly STON would recognize JSON and parse despite the different syntax...we'd have a nice multipurpose parser/writer that could speak JSON or STON interchangeably...
Dale
That is a great idea, Dale, I never thought of that ! Apart from the fact that I used single quotes for strings in STON, it already works (and I could enhance the parser to accept both I guess). Here is an example using a JSON REST call to convert IP addresses to countries: *nix Shell: $ curl http://easy.t3-platform.net/rest/geo-ip?address=205.140.197.101 {"latitude":38.0,"address":"205.140.197.101","country":"US","longitude":-97.0} Smalltalk: ZnClient new systemPolicy; beOneShot; url: 'http://easy.t3-platform.net/rest/geo-ip'; queryAt: 'address' put: '205.140.197.101'; accept: ZnMimeType applicationJson; contentReader: [ :entity | STON fromString: (entity contents replaceAll: $" with: $') ]; get. a Dictionary('address'->'205.140.197.101' 'country'->'US' 'latitude'->38.0 'longitude'->-97.0 ) Thx, Sven
On 24 Apr 2012, at 21:40, Sven Van Caekenberghe wrote:
Apart from the fact that I used single quotes for strings in STON, it already works (and I could enhance the parser to accept both I guess).
--- Name: Ston-Core-SvenVanCaekenberghe.27 Author: SvenVanCaekenberghe Time: 24 April 2012, 10:13:07 pm UUID: dd7c16b2-950c-4246-bf60-130de447cddf Ancestors: Ston-Core-SvenVanCaekenberghe.26 extended the parser to allow either single or double quotes as String delimiters which gives us JSON backward compatibilty ! (thanks Dale Henrichs for this great idea). ---
Very cool! Let's see ... export the pharo implementation of STON in Cypress package format (using FileTree), then port STON to other dialects and voila:) the next generation of Cypress can use STON in place of JSON! Dale ----- Original Message ----- | From: "Sven Van Caekenberghe" <sven@beta9.be> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 1:14:08 PM | Subject: Re: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | | On 24 Apr 2012, at 21:40, Sven Van Caekenberghe wrote: | | > Apart from the fact that I used single quotes for strings in STON, | > it already works (and I could enhance the parser to accept both I | > guess). | | --- | Name: Ston-Core-SvenVanCaekenberghe.27 | Author: SvenVanCaekenberghe | Time: 24 April 2012, 10:13:07 pm | UUID: dd7c16b2-950c-4246-bf60-130de447cddf | Ancestors: Ston-Core-SvenVanCaekenberghe.26 | | extended the parser to allow either single or double quotes as String | delimiters which gives us JSON backward compatibilty ! (thanks Dale | Henrichs for this great idea). | --- | | | |
On 4/24/12, Dale Henrichs <dhenrich@vmware.com> wrote:
Very cool!
Let's see ... export the pharo implementation of STON in Cypress package format (using FileTree), then port STON to other dialects and voila:) the next generation of Cypress can use STON in place of JSON!
Dale
Yes, this is promising. First of all STON has a good name 'Smalltalk Object Notation'. People who know JSON will be instantly familiar with the general idea. Sven writes this in the document attached as a PDF at the beginning of this thread. [1] Second STON offers more than JSON shared objects and cycles in the object graph are supported in STON. more data types Third STON seems to be superset of JSON. Sven writes above that he implemented this today in a new version. Fourth It is literal Smalltalk if we accept a dictionary type {} as done in Amber Smalltalk. -- Hannes [1] Sven Van Caekenberghe If you have ever seen JSON (Javascript Object Notation) 1 , you will be instantly familiar with Smaltalk Object Notation. It uses similar primitive values, with the addition of a symbol type. Some details are slightly different though. For object values an optional class tag precedes a representation that is either a list of values or a map of keys and values. Exactly how each class is represented is up to the implementation, although this document suggests some conventions for well known classes.
----- Original Message ----- | From: "Sven Van Caekenberghe" <sven@beta9.be> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 1:14:08 PM | Subject: Re: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | | On 24 Apr 2012, at 21:40, Sven Van Caekenberghe wrote: | | > Apart from the fact that I used single quotes for strings in STON, | > it already works (and I could enhance the parser to accept both I | > guess). | | --- | Name: Ston-Core-SvenVanCaekenberghe.27 | Author: SvenVanCaekenberghe | Time: 24 April 2012, 10:13:07 pm | UUID: dd7c16b2-950c-4246-bf60-130de447cddf | Ancestors: Ston-Core-SvenVanCaekenberghe.26 | | extended the parser to allow either single or double quotes as String | delimiters which gives us JSON backward compatibilty ! (thanks Dale | Henrichs for this great idea). | --- | | | |
Additional context information for people who will come late to this thread: We are discussing a human-readable-format, so neither something like SIXX, nor a binary data exchange format. On 4/25/12, H. Hirzel <hannes.hirzel@gmail.com> wrote:
On 4/24/12, Dale Henrichs <dhenrich@vmware.com> wrote:
Very cool!
Let's see ... export the pharo implementation of STON in Cypress package format (using FileTree), then port STON to other dialects and voila:) the next generation of Cypress can use STON in place of JSON!
Dale
Yes, this is promising.
First of all STON has a good name 'Smalltalk Object Notation'. People who know JSON will be instantly familiar with the general idea. Sven writes this in the document attached as a PDF at the beginning of this thread. [1]
Second STON offers more than JSON shared objects and cycles in the object graph are supported in STON. more data types
Third STON seems to be superset of JSON. Sven writes above that he implemented this today in a new version.
Fourth It is literal Smalltalk if we accept a dictionary type {} as done in Amber Smalltalk.
-- Hannes
[1] Sven Van Caekenberghe If you have ever seen JSON (Javascript Object Notation) 1 , you will be instantly familiar with Smaltalk Object Notation. It uses similar primitive values, with the addition of a symbol type. Some details are slightly different though. For object values an optional class tag precedes a representation that is either a list of values or a map of keys and values. Exactly how each class is represented is up to the implementation, although this document suggests some conventions for well known classes.
----- Original Message ----- | From: "Sven Van Caekenberghe" <sven@beta9.be> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 1:14:08 PM | Subject: Re: [Pharo-project] STON (was Re: SimpleLiteralArray Spec) | | | On 24 Apr 2012, at 21:40, Sven Van Caekenberghe wrote: | | > Apart from the fact that I used single quotes for strings in STON, | > it already works (and I could enhance the parser to accept both I | > guess). | | --- | Name: Ston-Core-SvenVanCaekenberghe.27 | Author: SvenVanCaekenberghe | Time: 24 April 2012, 10:13:07 pm | UUID: dd7c16b2-950c-4246-bf60-130de447cddf | Ancestors: Ston-Core-SvenVanCaekenberghe.26 | | extended the parser to allow either single or double quotes as String | delimiters which gives us JSON backward compatibilty ! (thanks Dale | Henrichs for this great idea). | --- | | | |
Dne 25. 04. 2012 00:20, piše H. Hirzel:
First of all STON has a good name 'Smalltalk Object Notation'. People who know JSON will be instantly familiar with the general idea.
Exactly! -- Janko Mivšek Aida/Web Smalltalk Web Application Server http://www.aidaweb.si
Sven Van Caekenberghe wrote:
Norbert, Dale, et al,
On 24 Apr 2012, at 18:43, Norbert Hartl wrote:
What is STON and STON-object syntax?
Arhh, it is not yet finished, I did it for myself, not for trying to convince the world, since that it too hard anyway.
This is the work-in-progress paper:
------------------------------------------------------------------------
The code lives here: http://ss3.gemstone.com/ss/STON
It is a slight variation of JSON, adding some Smalltalk concepts, as well as a solution to the class problem.
But again, I am not trying to convince anyone to use this, I did it as a proof of concept for myself, and I kind of like it.
On the other hand, JSON is so ubiquitous that I wonder a bit, why bother ?
Sven
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
The human readability is a strong point of JSON & STON, but is there any requirement for "human edit-ability" by which I mean in a text editor? In which case the referencing objects by counting "OrderedCollection [ Point [1, 2], @2, @2 ]" would be fragile, since insertion of an object via text editor would offset all references. Something like "OrderedCollection@1 [ Point@2 [1, 2], @2, @2 ]" would be more robust. However if you expect to never need to edit the STON file in a text editor, this point is mute. The downside of this idea is a bit more cluttered look. However I notice that evaluating #123 in Workspace returns an integer rather than a ByteSymbol. So it would seem that #123 would never be written out in STON format (from Pharo), which means "OrderedCollection#1 [ Point#2 [1, 2], #2, #2 ]" might be a less cluttered option, or perhaps a mix like "OrderedCollection#1 [ Point#2 [1, 2], @2, @2 ]" doesn't detract too much from readability. Actually I think this enhances readability anyhow, since if there are a lot of object references in the STON file, in a text viewer I would have to need to manually count them to follow the references. cheers -ben
Is it just me or is there a problem? Can't get STON from SS3. Doesn't looks like down tough. Phil 2012/4/25 Ben Coman <btc@openinworld.com>
Sven Van Caekenberghe wrote:
Norbert, Dale, et al,
On 24 Apr 2012, at 18:43, Norbert Hartl wrote:
What is STON and STON-object syntax?
Arhh, it is not yet finished, I did it for myself, not for trying to convince the world, since that it too hard anyway.
This is the work-in-progress paper:
------------------------------**------------------------------** ------------
The code lives here: http://ss3.gemstone.com/ss/**STON<http://ss3.gemstone.com/ss/STON>
It is a slight variation of JSON, adding some Smalltalk concepts, as well as a solution to the class problem.
But again, I am not trying to convince anyone to use this, I did it as a proof of concept for myself, and I kind of like it.
On the other hand, JSON is so ubiquitous that I wonder a bit, why bother ?
Sven
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
The human readability is a strong point of JSON & STON, but is there any requirement for "human edit-ability" by which I mean in a text editor? In which case the referencing objects by counting "OrderedCollection [ Point [1, 2], @2, @2 ]" would be fragile, since insertion of an object via text editor would offset all references. Something like "OrderedCollection@1 [ Point@2 [1, 2], @2, @2 ]" would be more robust. However if you expect to never need to edit the STON file in a text editor, this point is mute.
The downside of this idea is a bit more cluttered look. However I notice that evaluating #123 in Workspace returns an integer rather than a ByteSymbol. So it would seem that #123 would never be written out in STON format (from Pharo), which means "OrderedCollection#1 [ Point#2 [1, 2], #2, #2 ]" might be a less cluttered option, or perhaps a mix like "OrderedCollection#1 [ Point#2 [1, 2], @2, @2 ]" doesn't detract too much from readability.
Actually I think this enhances readability anyhow, since if there are a lot of object references in the STON file, in a text viewer I would have to need to manually count them to follow the references.
cheers -ben
-- Philippe Back "Helping you hit the top 3 outcomes you really want to achieve" Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 Mail: phil@highoctane.be| Web: http://philippeback.eu | Blog: http://philippeback.be High Octane SPRL rue cour Boisacq 101 1301 Bierges
On 25 Apr 2012, at 07:46, phil@highoctane.be wrote:
Is it just me or is there a problem?
Can't get STON from SS3. Doesn't looks like down tough.
Apparently the package names themselves have different capitalization: Gofer it url: 'http://ss3.gemstone.com/ss/STON'; package: 'Ston-Core'; package: 'Ston-Tests'; load. Yes, I am confused as well. Sven
Ben, On 25 Apr 2012, at 05:32, Ben Coman wrote:
The human readability is a strong point of JSON & STON, but is there any requirement for "human edit-ability" by which I mean in a text editor? In which case the referencing objects by counting "OrderedCollection [ Point [1, 2], @2, @2 ]" would be fragile, since insertion of an object via text editor would offset all references. Something like "OrderedCollection@1 [ Point@2 [1, 2], @2, @2 ]" would be more robust. However if you expect to never need to edit the STON file in a text editor, this point is mute.
The downside of this idea is a bit more cluttered look. However I notice that evaluating #123 in Workspace returns an integer rather than a ByteSymbol. So it would seem that #123 would never be written out in STON format (from Pharo), which means "OrderedCollection#1 [ Point#2 [1, 2], #2, #2 ]" might be a less cluttered option, or perhaps a mix like "OrderedCollection#1 [ Point#2 [1, 2], @2, @2 ]" doesn't detract too much from readability.
Actually I think this enhances readability anyhow, since if there are a lot of object references in the STON file, in a text viewer I would have to need to manually count them to follow the references.
Thanks for the feedback, I will think about your idea of explicit numbering. However, a format with numbered references will never be very human friendly I guess. If the goal is to have like a config file, I would make sure there are no such references. Sven
Ben Coman wrote:
However I notice that evaluating #123 in Workspace returns an integer rather than a ByteSymbol.
I meant also to poll how compatible the behaviour of #123 evaluating to SmallInteger rather than ByteSymbol is across Smalltalk dialects...
So it would seem that #123 would never be written out in STON format (from Pharo), which means "OrderedCollection#1 [ Point#2 [1, 2], #2, #2 ]" might be a less cluttered option, or perhaps a mix like "OrderedCollection#1 [ Point#2 [1, 2], @2, @2 ]" doesn't detract too much from readability.
Actually I think this enhances readability anyhow, since if there are a lot of object references in the STON file, in a text viewer I would have to need to manually count them to follow the references.
cheers -ben
Ben Coman wrote:
Sven Van Caekenberghe wrote: The human readability is a strong point of JSON & STON, but is there any requirement for "human edit-ability" by which I mean in a text editor? In which case the referencing objects by counting "OrderedCollection [ Point [1, 2], @2, @2 ]" would be fragile, since insertion of an object via text editor would offset all references. Something like "OrderedCollection@1 [ Point@2 [1, 2], @2, @2 ]" would be more robust. However if you expect to never need to edit the STON file in a text editor, this point is mute.
Shouldn't relative numbering help here? Not a bulletproof solution, but for simple changes, it suffices.
The downside of this idea is a bit more cluttered look. However I notice that evaluating #123 in Workspace returns an integer rather than a ByteSymbol. So it would seem that #123 would never be written out in STON format (from Pharo), which means "OrderedCollection#1 [ Point#2 [1, 2], #2, #2 ]" might be a less cluttered option, or perhaps a mix like "OrderedCollection#1 [ Point#2 [1, 2], @2, @2 ]" doesn't detract too much from readability.
Actually I think this enhances readability anyhow, since if there are a lot of object references in the STON file, in a text viewer I would have to need to manually count them to follow the references.
cheers -ben
Herby
Sven, I see that you have already gone beyond a simple literal array style syntax and it looks like you've headed into YAML territory? ... I'm seeing lots of []... the nice thing about YAML is that they've eliminated a lot of the excess syntactic characters which makes it very readable. The real test for readability would be a variety of sample documents... Dale ----- Original Message ----- | From: "Dale Henrichs" <dhenrich@vmware.com> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 9:18:26 AM | Subject: Re: [Pharo-project] SimpleLiteralArray Spec | | Sven, | | I know that having the parser produce dictionaries (i.e., dictionary | _syntax_) is very convenient for the consumer of the parsed output | ... | | When javascript evaluates/parses JSON, you will note that you get | real objects back instead of dictionaries and you can send messages | to those objects, so I would think that rather than inventing a | dictionary syntax, you invent a STON-object syntax then we can then | send setter/getter style messages to these boys... then we'd have | _objects_, arrays, strings, numbers and booleans for the STON format | ... a field could be optionally included in the STON object called | _class_ which would optionally allow the parser to produce instances | of that class with all of the fields filled in ... | | JSON on steroids! | | Anyway ... just a thought. | | Dale | | ----- Original Message ----- | | From: "Sven Van Caekenberghe" <sven@beta9.be> | | To: "Pharo Development" <Pharo-project@lists.gforge.inria.fr> | | Sent: Tuesday, April 24, 2012 4:26:12 AM | | Subject: [Pharo-project] SimpleLiteralArray Spec | | | | OK, just to get started, let try to write a spec for | | SimpleLiteralArray, this is based on JSON (and my own STON, not | | ready for public consumption). It will soon become clear that this | | is slightly harder than expected, IMO a writer is needed as well. | | | | S y n t a x | | | | value | | primitive-value | | list | | nil | | primitive-value | | number | | true | | false | | symbol | | string | | classname | | list | | #() | | #( elements ) | | elements | | value | | value , elements | | string | | '' | | ' chars ' | | chars | | char | | char chars | | char | | any-printable-ASCII-character- | | except-"-or-\ | | \" | | \\ | | \/ | | \b | | \f | | \n | | \r | | \t | | \u four-hex-digits | | symbol | | # chars-limited | | # ' chars ' | | chars-limited | | char-limited | | char-limited chars-limited | | char-limited | | a-z A-Z 0-9 - _ . / | | classname | | uppercase-alpha-char alphanumeric-char | | number | | int | | int frac | | int exp | | int frac exp | | int | | digit | | digit1-9 digits | | - digit | | - digit1-9 digits | | frac | | . digits | | exp | | e digits | | digits | | digit | | digit digits | | e | | e | | e+ | | e- | | E | | E+ | | E- | | | | | | Questions remain: | | | | - is the escaping OK ? | | - what about symbols without a leading # ? | | - what should be allowed in unquoted symbols ? | | - what should be allowed for classnames ? | | - what about namespaces ? | | - do we allow infinite precision integers ? | | - maybe the toplevel should always be an array ? | | | | | | Now: shoot! | | | | | | Sven | | | | | | -- | | Sven Van Caekenberghe | | http://stfx.eu | | Smalltalk is the Red Pill | | | | | | | | | | | | | |
Now I can take a crack at your questions:) ----- Original Message ----- | From: "Sven Van Caekenberghe" <sven@beta9.be> | To: "Pharo Development" <Pharo-project@lists.gforge.inria.fr> | Sent: Tuesday, April 24, 2012 4:26:12 AM | Subject: [Pharo-project] SimpleLiteralArray Spec | ...snip | | Questions remain: | | - is the escaping OK ? | - what about symbols without a leading # ? If you can get rid of extra syntax, that would be good ... | - what should be allowed in unquoted symbols ? | - what should be allowed for classnames ? For Cypress we decided that the name of the directory (className.class) would not necessarily be the real name of the class since the classname rules might vary across different platforms, so something along the same lines might make sense by encoding the real name of the class in a field of the object while preserving the external name ... that way you the external class name could be 'classname' as defined, while the real name of the class could be specified in an optional classname field when the real name can't be mapped to 'classnamed' | - what about namespaces ? For Cypress namespaces are handled by special class attributes , so perhaps there would be an optional class_properties field where additional meta information about the class name resolution can be stored | - do we allow infinite precision integers ? | - maybe the toplevel should always be an array ? I like the idea that the top level can be any legal construct ... | | | Now: shoot! | | | Sven | | | -- | Sven Van Caekenberghe | http://stfx.eu | Smalltalk is the Red Pill | | | | | |
On Apr 24, 2012, at 1:26 PM, Sven Van Caekenberghe wrote:
OK, just to get started, let try to write a spec for SimpleLiteralArray, this is based on JSON (and my own STON, not ready for public consumption). It will soon become clear that this is slightly harder than expected, IMO a writer is needed as well.
S y n t a x
value primitive-value list nil primitive-value number true false symbol string classname
In a literal array a class name is just a symbol #(Foo) first #Foo
list #() #( elements ) elements value value , elements string '' ' chars ' chars char char chars char any-printable-ASCII-character- except-"-or-\ \" \\ \/ \b \f \n \r \t \u four-hex-digits
why do we need that?
symbol # chars-limited # ' chars ' chars-limited char-limited char-limited chars-limited char-limited a-z A-Z 0-9 - _ . / classname uppercase-alpha-char alphanumeric-char
there is not need for classname as element classname is just a pair that the interpretation should handle if necessary
number int int frac int exp int frac exp int digit digit1-9 digits - digit - digit1-9 digits frac . digits exp e digits digits digit digit digits e e e+ e- E E+ E-
Questions remain:
- is the escaping OK ?
I do not know but in literal array there is no escaping. Now if you want to put a cr into a string you probably need that
- what about symbols without a leading # ?
#(foo) -> #(#foo) #(true nil false 1 'n') -> #(true nil false 1 'n')
- what should be allowed in unquoted symbols ? I did not get the question
- what should be allowed for classnames ? Nothing
- what about namespaces ? Nothing no need I do not think that they are in literal arrays. (may be in VW but I did not check)
- do we allow infinite precision integers ? Yes
- maybe the toplevel should always be an array ?
I would say so else this is not a literalArray but a literal.
Now: shoot!
Sven
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
Stef, Which is more important: implementation or functionality? You appear to be fixated on a particular solution and seem to be ignoring the actual problems. Dale ----- Original Message ----- | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 3:13:20 PM | Subject: Re: [Pharo-project] SimpleLiteralArray Spec | | | On Apr 24, 2012, at 1:26 PM, Sven Van Caekenberghe wrote: | | > OK, just to get started, let try to write a spec for | > SimpleLiteralArray, this is based on JSON (and my own STON, not | > ready for public consumption). It will soon become clear that this | > is slightly harder than expected, IMO a writer is needed as well. | > | > S y n t a x | > | > value | > primitive-value | > list | > nil | > primitive-value | > number | > true | > false | > symbol | > string | > classname | | In a literal array a class name is just a symbol | | #(Foo) first | #Foo | | > list | > #() | > #( elements ) | > elements | > value | > value , elements | > string | > '' | > ' chars ' | > chars | > char | > char chars | > char | > any-printable-ASCII-character- | > except-"-or-\ | > \" | > \\ | > \/ | > \b | > \f | > \n | > \r | > \t | > \u four-hex-digits | | why do we need that? | | > symbol | > # chars-limited | > # ' chars ' | > chars-limited | > char-limited | > char-limited chars-limited | > char-limited | > a-z A-Z 0-9 - _ . / | > classname | > uppercase-alpha-char alphanumeric-char | | there is not need for classname as element | classname is just a pair that the interpretation should handle if | necessary | | | > number | > int | > int frac | > int exp | > int frac exp | > int | > digit | > digit1-9 digits | > - digit | > - digit1-9 digits | > frac | > . digits | > exp | > e digits | > digits | > digit | > digit digits | > e | > e | > e+ | > e- | > E | > E+ | > E- | > | > | > Questions remain: | > | > - is the escaping OK ? | | I do not know but in literal array there is no escaping. Now if you | want to put a cr into a string you probably need that | | > - what about symbols without a leading # ? | | #(foo) -> #(#foo) | #(true nil false 1 'n') -> #(true nil false 1 'n') | | > - what should be allowed in unquoted symbols ? | I did not get the question | | > - what should be allowed for classnames ? | Nothing | | > - what about namespaces ? | Nothing no need I do not think that they are in literal arrays. (may | be in VW but I did not check) | | > - do we allow infinite precision integers ? | Yes | | > - maybe the toplevel should always be an array ? | | I would say so else this is not a literalArray but a literal. | | | > | > | > Now: shoot! | > | > | > Sven | > | > | > -- | > Sven Van Caekenberghe | > http://stfx.eu | > Smalltalk is the Red Pill | > | > | > | > | > | | |
On Apr 25, 2012, at 12:17 AM, Dale Henrichs wrote:
Stef,
Which is more important:
implementation or functionality?
functionality: how to store meta data of class or method. So far I do not see why I need ston. Now if you tell me that you need a textual serializer then this is another story and scenario. But this is not the one we need to store meta data of class and method. I'm not that approximate to mix the two. Stef
You appear to be fixated on a particular solution and seem to be ignoring the actual problems.
Dale
----- Original Message ----- | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 3:13:20 PM | Subject: Re: [Pharo-project] SimpleLiteralArray Spec | | | On Apr 24, 2012, at 1:26 PM, Sven Van Caekenberghe wrote: | | > OK, just to get started, let try to write a spec for | > SimpleLiteralArray, this is based on JSON (and my own STON, not | > ready for public consumption). It will soon become clear that this | > is slightly harder than expected, IMO a writer is needed as well. | > | > S y n t a x | > | > value | > primitive-value | > list | > nil | > primitive-value | > number | > true | > false | > symbol | > string | > classname | | In a literal array a class name is just a symbol | | #(Foo) first | #Foo | | > list | > #() | > #( elements ) | > elements | > value | > value , elements | > string | > '' | > ' chars ' | > chars | > char | > char chars | > char | > any-printable-ASCII-character- | > except-"-or-\ | > \" | > \\ | > \/ | > \b | > \f | > \n | > \r | > \t | > \u four-hex-digits | | why do we need that? | | > symbol | > # chars-limited | > # ' chars ' | > chars-limited | > char-limited | > char-limited chars-limited | > char-limited | > a-z A-Z 0-9 - _ . / | > classname | > uppercase-alpha-char alphanumeric-char | | there is not need for classname as element | classname is just a pair that the interpretation should handle if | necessary | | | > number | > int | > int frac | > int exp | > int frac exp | > int | > digit | > digit1-9 digits | > - digit | > - digit1-9 digits | > frac | > . digits | > exp | > e digits | > digits | > digit | > digit digits | > e | > e | > e+ | > e- | > E | > E+ | > E- | > | > | > Questions remain: | > | > - is the escaping OK ? | | I do not know but in literal array there is no escaping. Now if you | want to put a cr into a string you probably need that | | > - what about symbols without a leading # ? | | #(foo) -> #(#foo) | #(true nil false 1 'n') -> #(true nil false 1 'n') | | > - what should be allowed in unquoted symbols ? | I did not get the question | | > - what should be allowed for classnames ? | Nothing | | > - what about namespaces ? | Nothing no need I do not think that they are in literal arrays. (may | be in VW but I did not check) | | > - do we allow infinite precision integers ? | Yes | | > - maybe the toplevel should always be an array ? | | I would say so else this is not a literalArray but a literal. | | | > | > | > Now: shoot! | > | > | > Sven | > | > | > -- | > Sven Van Caekenberghe | > http://stfx.eu | > Smalltalk is the Red Pill | > | > | > | > | > | | |
Stef, I mentioned several weeks ago that the Cypress implementation takes advantage of the fact that the JSON parser produces dictionaries ... this is a bit of functionality that I called out as critical ... STON is the first Smalltalk-centric text format that I've seen that has a nice syntax for producing Dictionaries. Of course, I think it is a bonus that you can use it to represent specific class instances as that can prove to be very useful. Of course I am a Smalltalk programmer, so if you give me STON, instead of meticulously constructing a JSON structure that matches the fields of the classes (for each dialect) I think that we would be inclined to ask the class to create a STON representation and dump that to disk ... So the immediate value of STON would be that we'd be sending messages to objects instead of translating formats .... the messages that we send will end up creating a nice human-readable format ... win-win. Of course for multiple dialects we agreed that we would use the same set of field names for the class attributes so that a dialect could create the class using the fields that it recognized ... so that would imply some level of standardization of the STON field names to be used, but that shouldn't be too hard since there aren't that many fields to monkey with for classes ... Dale ----- Original Message ----- | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | To: Pharo-project@lists.gforge.inria.fr | Sent: Tuesday, April 24, 2012 3:36:47 PM | Subject: Re: [Pharo-project] SimpleLiteralArray Spec | | | On Apr 25, 2012, at 12:17 AM, Dale Henrichs wrote: | | > Stef, | > | > Which is more important: | > | > implementation or functionality? | | functionality: how to store meta data of class or method. So far I do | not see why I need ston. | | Now if you tell me that you need a textual serializer then this is | another story and scenario. | But this is not the one we need to store meta data of class and | method. | | I'm not that approximate to mix the two. | | Stef | > | > You appear to be fixated on a particular solution and seem to be | > ignoring the actual problems. | > | > Dale | > | > ----- Original Message ----- | > | From: "Stéphane Ducasse" <stephane.ducasse@inria.fr> | > | To: Pharo-project@lists.gforge.inria.fr | > | Sent: Tuesday, April 24, 2012 3:13:20 PM | > | Subject: Re: [Pharo-project] SimpleLiteralArray Spec | > | | > | | > | On Apr 24, 2012, at 1:26 PM, Sven Van Caekenberghe wrote: | > | | > | > OK, just to get started, let try to write a spec for | > | > SimpleLiteralArray, this is based on JSON (and my own STON, not | > | > ready for public consumption). It will soon become clear that | > | > this | > | > is slightly harder than expected, IMO a writer is needed as | > | > well. | > | > | > | > S y n t a x | > | > | > | > value | > | > primitive-value | > | > list | > | > nil | > | > primitive-value | > | > number | > | > true | > | > false | > | > symbol | > | > string | > | > classname | > | | > | In a literal array a class name is just a symbol | > | | > | #(Foo) first | > | #Foo | > | | > | > list | > | > #() | > | > #( elements ) | > | > elements | > | > value | > | > value , elements | > | > string | > | > '' | > | > ' chars ' | > | > chars | > | > char | > | > char chars | > | > char | > | > any-printable-ASCII-character- | > | > except-"-or-\ | > | > \" | > | > \\ | > | > \/ | > | > \b | > | > \f | > | > \n | > | > \r | > | > \t | > | > \u four-hex-digits | > | | > | why do we need that? | > | | > | > symbol | > | > # chars-limited | > | > # ' chars ' | > | > chars-limited | > | > char-limited | > | > char-limited chars-limited | > | > char-limited | > | > a-z A-Z 0-9 - _ . / | > | > classname | > | > uppercase-alpha-char alphanumeric-char | > | | > | there is not need for classname as element | > | classname is just a pair that the interpretation should handle if | > | necessary | > | | > | | > | > number | > | > int | > | > int frac | > | > int exp | > | > int frac exp | > | > int | > | > digit | > | > digit1-9 digits | > | > - digit | > | > - digit1-9 digits | > | > frac | > | > . digits | > | > exp | > | > e digits | > | > digits | > | > digit | > | > digit digits | > | > e | > | > e | > | > e+ | > | > e- | > | > E | > | > E+ | > | > E- | > | > | > | > | > | > Questions remain: | > | > | > | > - is the escaping OK ? | > | | > | I do not know but in literal array there is no escaping. Now if | > | you | > | want to put a cr into a string you probably need that | > | | > | > - what about symbols without a leading # ? | > | | > | #(foo) -> #(#foo) | > | #(true nil false 1 'n') -> #(true nil false 1 'n') | > | | > | > - what should be allowed in unquoted symbols ? | > | I did not get the question | > | | > | > - what should be allowed for classnames ? | > | Nothing | > | | > | > - what about namespaces ? | > | Nothing no need I do not think that they are in literal arrays. | > | (may | > | be in VW but I did not check) | > | | > | > - do we allow infinite precision integers ? | > | Yes | > | | > | > - maybe the toplevel should always be an array ? | > | | > | I would say so else this is not a literalArray but a literal. | > | | > | | > | > | > | > | > | > Now: shoot! | > | > | > | > | > | > Sven | > | > | > | > | > | > -- | > | > Sven Van Caekenberghe | > | > http://stfx.eu | > | > Smalltalk is the Red Pill | > | > | > | > | > | > | > | > | > | > | > | | > | | > | | > | | |
participants (11)
-
Ben Coman -
Dale Henrichs -
Göran Krampe -
H. Hirzel -
Herby VojÄÃk -
Janko Mivšek -
Norbert Hartl -
phil＠highoctane.be -
Sean P. DeNigris -
Stéphane Ducasse -
Sven Van Caekenberghe