On 03 Dec 2012, at 12:43, Sven Van Caekenberghe <sven@stfx.eu> wrote:
There are ZnCharacterReadStream and ZnCharacterWriteStream that add decoding/encoding to binary streams. They could indeed be used as an alternative. But that was not my point.
I reran the examples using a binary file stream to disable the decoding and then using ZnCharacterReadStream to handle the decoding: [ '/tmp/numbers.json' asFileReference readStreamDo: [ :stream | (NeoJSONReader on: stream) next ] ] timeToRun. 3502 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun. 1214 [ StandardFileStream fileNamed: '/tmp/numbers.json' do: [ :stream | (NeoJSONReader on: stream) next ] ] timeToRun 2869 [ StandardFileStream fileNamed: '/tmp/numbers.json' do: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun 1280 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :stream | (NeoJSONReader on: (ZnCharacterReadStream on: stream binary)) next ] ] timeToRun. 1655 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :stream | (NeoJSONReader on: (ZnCharacterReadStream on: stream binary encoding: 'latin1')) next ] ] timeToRun. 1574 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: (ZnCharacterReadStream on: fstream binary) do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun. 1753 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: (ZnCharacterReadStream on: fstream binary encoding: 'latin1') do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun. 1655 This shows several things (for this particular test case): - ZnCharacterReadStream is at least as good as the builtin decoding streams (here twice as good) - UTF-8 (ZnUTF8Encoder) and Latin1 (ZnNullEncoder) decoding are close together (because the input is ASCII) - ZnBufferedReadStream does not help when the stream is pretty good (ZnCharacterReadStream actually uses a 1 character buffer to support peek, which is important to speed up this use case) - this is to be expected: it adds a layer, but its fetch next buffer logic still loops using #next, decoding possibly Unicode characters - the super fast runs (the 2nd and the 4th) are actually cheating because they skip decoding - thanks for that observation ! Sven PS: While doing these explorations, I actually updated ZnUTF8Encoder quite a bit, both when handling ASCII, which is the majority of characters in most cases, as well as in the general case; see the latest version. -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill