Hendrik, Thanks for entering this discussion, it is important territory, for many users. We should get the basics right. On 03 Dec 2012, at 12:10, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
The reduced read times you are seeing, is lack of decoding, not lack of buffering.
I should have been more clear: it is a combination of buffering & decoding (see further).
BufferedReadStream uses readInto:startingAt:count:, which isn't reimplemented for MultiByteFileStream to do proper decoding, and thus inherits the one from StandardFileStream.
If that is so (and it seems to be the case), then that is an important bug, wouldn't you agree ?
The filestreams already do read buffering (albeit with a smaller buffer size), so when Fuel uses binary mode (which skips decoding), the performance is pretty much the same.
Yeah, in binary mode, MultiByteFilestream does not do encoding. Still, IMO that is not the only reason for the difference. However, even when I go straight to MultiByteFilestream's superclass, StandardFileStream, which does not do decoding, the additional buffering helps a lot: [ StandardFileStream fileNamed: '/tmp/numbers.json' do: [ :stream | (NeoJSONReader on: stream) next ] ] timeToRun 2869 [ StandardFileStream fileNamed: '/tmp/numbers.json' do: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun 1280 I said from the start that the access pattern matters: parser code is a sweet spot here. Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill