On 18 Jan 2017, at 16:20, Peter Uhnak <i.uhnak@gmail.com> wrote:
On Wed, Jan 18, 2017 at 03:38:17PM +0100, Sven Van Caekenberghe wrote:
So talking only about the encoding/writing phase, the conclusion would be
10. Generators
A JSON generator produces JSON text. The resulting text MUST strictly conform to the JSON grammar.
I guess the ABNF table is the best reference; but we want to generate the most compact form available (=that still conforms to the syntax).
- not to escape $/
For covenience's sake compact is better * I guess the speciality of \/ is somehow related to JavaScript strings interpreting both '/' and '\/' into '/'?
- escape everything with code points [0,31], using named escapes if they exist, else \uHHHH
yes, the named ones have Pharo equivalent too
s := STON fromString: '"\b\f\n\r\t"'. s asArray "{Character backspace. Character newPage. Character lf. Character cr. Character tab}"
- escape $\ itself
yes
That leaves the question about $' and $".
$' is used in STON as string delimiter, so it has to be escaped.
- escape $'
We already had discussion about $' and JSON http://forum.world.st/STON-doesn-t-produce-valid-JSON-it-shouldn-t-escape-qu...
Yes, I know, that change was OK.
It must not be escaped in JSON, but that raises question about STON being superset of JSON; is such thing achievable given this disparity?
Right now, $" is also escaped. Should that remain the case, or only in JSON compatibility mode (where $" is used as string delimiter) ?
- do not escape $"
In JSON mode, escape $" and not $' then ?
That's how it should be in JSON.
When parsing, all named and other escapes are always accepted, as they are now.
9. Parsers
A JSON parser MUST accept all texts that conform to the JSON grammar. A JSON parser MAY accept non-JSON forms or extensions.
My question about STON and JSON: What is the benefit of STON being superset of JSON? To me it feels like an arbitrary restriction for STON; would STON benefit from dropping this requirement and instead only worry about good smalltalk object representation? (And leave JSON to NeoJSON or something.)
Being a superset means that you get a simple JSON parser (and even limited writer) for free once you install STON (or once it is part of the Pharo image, as it is now). It also means that we can fall back to the JSON spec as a guide in discussion like this one. It also helps people understand what STON is, by analogy but also differences with JSON. So my conclusion would be (while writing), always escape $\ and not $/, in pure STON mode (the default), escape $' and not $", in JSON mode, escape $" and not $'. Those would be the changes. Agreed ?
Peter
On 18 Jan 2017, at 15:25, Peter Uhnak <i.uhnak@gmail.com> wrote:
On Wed, Jan 18, 2017 at 11:11:06AM +0100, Christophe Demarey wrote:
Le 18 janv. 2017 à 09:51, Sven Van Caekenberghe <sven@stfx.eu> a écrit :
Hi Christophe,
STON toString: 'git@github.com:foo/bar.gitâ => ''git@github.com:foo\/bar.gitâ' It used to be ''git@github.com:foo/bar.gitââ.
In other words, it was an implementation error (omission). Note that JSON also has this escape.
Yes and no for JSON.
Only " and \ has to be escaped. Escaping anything else will give it special meaning with the exception of / which will just produce the same thing, because it is a special snowflake. :)
quoted from https://tools.ietf.org/html/rfc7159#section-7:
- Unicode characters may be placed within the quotation marks, except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F). - Alternatively, there are two-character sequence escape representations of some popular characters.
"/" (U+002F) doesn't fall into control character range, but the alternative section does permit it escaping it.
In other words, JSON strings "\/", and "/", and "\u002f" are equivalent.
But JSON itself doesn't require you to escape "/" (just like you are not required to escape "hi" into "\u0068\u0069", although you can).
Note that other systems do not escape / by default:
(Pharo) NeoJSONWriter toString: 'git@github.com:foo/bar.git' => "git@github.com:foo/bar.git"
(JavaScript) JSON.stringify('git@github.com:foo/bar.git') => "git@github.com:foo/bar.git"
(Ruby) require 'json' puts 'git@github.com:foo/bar.git'.to_json => "git@github.com:foo/bar.git"
Peter