2010/1/21 Igor Stasenko <siguctua@gmail.com>:

2010/1/21 Stéphane Ducasse <stephane.ducasse@inria.fr>:

...

Hi levente

I was wondering why the bufferedFileStream made the system faster. I mean are querying the same contents on several occasion?

Because disk access is times slower than any "intense" memory access.

Depends how the file access is parameterized... Maybe the OS is buffering behind our back anyway. There is also a cost associated to the primitive (converting file ID into a C FILE*, consistency checks, etc...), and maybe one associated to fread() itself (would be worth testing that with a simple C program). Nicolas

...

Stef

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

-- Best regards, Igor Stasenko AKA sig.

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Thanks. I was wondering if in addition to chunk versus single element reading, the logic of the tools was not fooled and for example accessing two times the source code (may be for identifying selector and the other for method body or something like that). Stef On Jan 21, 2010, at 8:19 PM, Levente Uzonyi wrote:

On Thu, 21 Jan 2010, Stéphane Ducasse wrote:

...

Hi levente

I was wondering why the bufferedFileStream made the system faster. I mean are querying the same contents on several occasion?

As Nicolas said, not all primitives have the same cost. The file primitives cost a lot more than primitive 65 or some smalltalk code. And as Igor said accessing the disk costs a lot more than accessing memory, but that only counts in rare cases.

Yes I imagine that. Marcus in the past did an experience to get all the code in memory and it was fast to do some source code search :)

Here's an example showing that the file primitive costs almost the same if it reads 1 or 2000 bytes:

#(1 2000) collect: [ :bufferSize | (1 to: 5) collect: [ :run | StandardFileStream readOnlyFileNamed: SourceFiles second name do: [ :file | | buffer fileID | fileID := file instVarNamed: #fileID. buffer := String new: bufferSize. Smalltalk garbageCollect. [ 1 to: 100000 do: [ :each | file primSetPosition: fileID to: 0; primRead: fileID into: buffer startingAt: 1 count: bufferSize ] ] timeToRun ] ] ]

===> #( #(458 466 463 466 467) #(482 484 492 483 484))

This means that a primitive send takes ~2.5 microseconds so it can be used ~400000 times/second. If you read one byte/primitive that means 400KB/sec, if you read 2000 bytes/primitive you get 800MB/sec (your disk may be slower of course).

Using the buffer you get the following values: (1 to: 10) collect: [ :run | StandardFileStream readOnlyFileNamed: SourceFiles second name do: [ :file | Smalltalk garbageCollect. [ 1 to: 100000 do: [ :each | file next ] ] timeToRun ] ] ===> #(15 14 12 11 12 12 11 12 12 12)

It takes ~12 milliseconds to read 100KB if you read the bytes one-by-one. That means 8.3MB/sec. This is far from 800MB/sec, but much better than the 400KB/sec.

Why does it matter? Because MultiByteFileStream (which is the default FileStream) can only read (and convert) single bytes.

Ok so this means that reading files could be really speed up by supporting a kind of "larger chunk" reading.

Levente

...

Stef

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project