Hi, I just published an implementation of Universal Binary JSON (UBJSON) for Pharo. https://github.com/svenvc/NeoUniversalBinaryJSON Universal Binary JSON (UBJSON) is a computer data interchange format. It is a binary form directly imitating JSON, but requiring fewer bytes of data. It aims to achieve the generality of JSON, combined with being easier and more efficient to process than JSON. http://ubjson.org https://en.wikipedia.org/wiki/UBJSON The size/speed/efficiency differences are minor for typical JSON payloads, especially compared with compacted JSON. The implementation is simpler, though, as there is no string escaping and no number parsing. UBJSON is making a larger difference when dealing with arrays containing numbers. Especially with ByteArrays, UBJSON makes a huge difference, since these are essentially stored natively. Sven
Excellent! Thanks Sven for this as well as for all your other contributions to the community. Noury
On 9 Oct 2019, at 14:29, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
I just published an implementation of Universal Binary JSON (UBJSON) for Pharo.
https://github.com/svenvc/NeoUniversalBinaryJSON
Universal Binary JSON (UBJSON) is a computer data interchange format. It is a binary form directly imitating JSON, but requiring fewer bytes of data. It aims to achieve the generality of JSON, combined with being easier and more efficient to process than JSON.
http://ubjson.org https://en.wikipedia.org/wiki/UBJSON
The size/speed/efficiency differences are minor for typical JSON payloads, especially compared with compacted JSON. The implementation is simpler, though, as there is no string escaping and no number parsing.
UBJSON is making a larger difference when dealing with arrays containing numbers. Especially with ByteArrays, UBJSON makes a huge difference, since these are essentially stored natively.
Sven
Sven Excellent. Can the same idea be extended to STON - or is it there already? Peter Kenny -----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Sven Van Caekenberghe Sent: 09 October 2019 13:29 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: [Pharo-users] [ ANN ] Neo Universal Binary JSON Hi, I just published an implementation of Universal Binary JSON (UBJSON) for Pharo. https://github.com/svenvc/NeoUniversalBinaryJSON Universal Binary JSON (UBJSON) is a computer data interchange format. It is a binary form directly imitating JSON, but requiring fewer bytes of data. It aims to achieve the generality of JSON, combined with being easier and more efficient to process than JSON. http://ubjson.org https://en.wikipedia.org/wiki/UBJSON The size/speed/efficiency differences are minor for typical JSON payloads, especially compared with compacted JSON. The implementation is simpler, though, as there is no string escaping and no number parsing. UBJSON is making a larger difference when dealing with arrays containing numbers. Especially with ByteArrays, UBJSON makes a huge difference, since these are essentially stored natively. Sven
On 9 Oct 2019, at 14:49, PBKResearch <peter@pbkresearch.co.uk> wrote:
Sven
Excellent. Can the same idea be extended to STON - or is it there already?
I did not (yet) consider that, I will think about it. However, Pharo contains two serialisation formats out of the box, STON, which is a textual format focusing on domain model objects, and FUEL, which is a fast binary format that can serialise special system objects as well.
Peter Kenny
-----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Sven Van Caekenberghe Sent: 09 October 2019 13:29 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: [Pharo-users] [ ANN ] Neo Universal Binary JSON
Hi,
I just published an implementation of Universal Binary JSON (UBJSON) for Pharo.
https://github.com/svenvc/NeoUniversalBinaryJSON
Universal Binary JSON (UBJSON) is a computer data interchange format. It is a binary form directly imitating JSON, but requiring fewer bytes of data. It aims to achieve the generality of JSON, combined with being easier and more efficient to process than JSON.
http://ubjson.org https://en.wikipedia.org/wiki/UBJSON
The size/speed/efficiency differences are minor for typical JSON payloads, especially compared with compacted JSON. The implementation is simpler, though, as there is no string escaping and no number parsing.
UBJSON is making a larger difference when dealing with arrays containing numbers. Especially with ByteArrays, UBJSON makes a huge difference, since these are essentially stored natively.
Sven
On 9 Oct 2019, at 14:29, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The size/speed/efficiency differences are minor for typical JSON payloads, especially compared with compacted JSON. The implementation is simpler, though, as there is no string escaping and no number parsing.
UBJSON is making a larger difference when dealing with arrays containing numbers. Especially with ByteArrays, UBJSON makes a huge difference, since these are essentially stored natively.
Here are some details/benchmarks. 1. A typical JSON payload data := ZnConstants httpStatusCodes associations collect: [ :each | { #code->each key. #reason->each value } asDictionary ]. [ NeoUBJSONWriter toByteArray: data ] bench. "'11108.114 per second'" bytes := NeoUBJSONWriter toByteArray: data. bytes size. "2290" [ NeoUBJSONReader fromByteArray: bytes ] bench. "'4412.670 per second'" [ NeoJSONWriter toString: data ] bench. "'11542.783 per second'" json := NeoJSONWriter toString: data. json size. "2358" [ NeoJSONReader fromString: json ] bench. "'4814.711 per second'" 2. A 1K integer array data := (1 to: 1024) asArray. [ NeoUBJSONWriter toByteArray: data ] bench. "'6945.444 per second'" bytes := NeoUBJSONWriter toByteArray: data. bytes size. "2822" [ NeoUBJSONReader fromByteArray: bytes ] bench. "'3280.632 per second'" [ NeoJSONWriter toString: data ] bench. "'4523.095 per second'" json := NeoJSONWriter toString: data. json size. "4014" [ NeoJSONReader fromString: json ] bench. "'1253.749 per second'" 3. A 1K byte array data := ByteArray new: 1024 streamContents: [ :out | 1024 timesRepeat: [ out nextPut: 256 atRandom - 1 ] ]. [ NeoUBJSONWriter toByteArray: data ] bench. "'538493.501 per second'" bytes := NeoUBJSONWriter toByteArray: data. bytes size. "1031" [ NeoUBJSONReader fromByteArray: bytes ] bench. "'216269.200 per second'" [ NeoJSONWriter toString: data ] bench. "'4579.084 per second'" json := NeoJSONWriter toString: data. json size. "3686" [ NeoJSONReader fromString: json ] bench. "'1297.362 per second'" Right now, only ByteArray got a highly optimised implementation. In the future, FloatArray and IntegerArray could receive the same treatment. Sven PS: Note that UBJSON is not the same as BSON, nor CBOR, nor MessagePack, which all exist in the same space.
participants (3)
-
Noury Bouraqadi -
PBKResearch -
Sven Van Caekenberghe