[Pharo-project] Writing 16 times 1mb to a WriteStream takes 1s. Normal?
Hi guys: |byteArray| byteArray := (ByteArray new: 1024*1024). [ ByteArray streamContents: [:s | 16 timesRepeat: [ s nextPutAll:byteArray ] ] ] timeToRun Gives me around 1 second. Isn't that too much? or is it normal/expected? Anything I could improve? Thanks, -- Mariano http://marianopeck.wordpress.com
Maybe pre-allocate the right size: ByteArray new: byteArray size * 16 streamContents: ... Nicolas 2012/11/10 Mariano Martinez Peck <marianopeck@gmail.com>:
Hi guys:
|byteArray| byteArray := (ByteArray new: 1024*1024). [ ByteArray streamContents: [:s | 16 timesRepeat: [ s nextPutAll:byteArray ] ] ] timeToRun
Gives me around 1 second. Isn't that too much? or is it normal/expected? Anything I could improve?
Thanks,
-- Mariano http://marianopeck.wordpress.com
On Sat, Nov 10, 2012 at 8:18 PM, Nicolas Cellier < nicolas.cellier.aka.nice@gmail.com> wrote:
Maybe pre-allocate the right size:
ByteArray new: byteArray size * 16 streamContents: ...
Well, actually we are trying to do a streaming API for a compressor. So, 1) I don't exactly know the number "16" before hand and 2) doing that is exactly what we are trying to avoid (compress/decompress all together). Instead, we are doing some kind of streaming, compressing in blocks of say 1MB. But I was completely surprised when I saw the results.... So there is nothing easy to improve? Nicolas
2012/11/10 Mariano Martinez Peck <marianopeck@gmail.com>:
Hi guys:
|byteArray| byteArray := (ByteArray new: 1024*1024). [ ByteArray streamContents: [:s | 16 timesRepeat: [ s nextPutAll:byteArray ] ] ] timeToRun
Gives me around 1 second. Isn't that too much? or is it normal/expected? Anything I could improve?
Thanks,
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
But mariano two questions: - did it change something? - don't you have an idea of the chunk you will use for the increment? Stef On Nov 10, 2012, at 8:22 PM, Mariano Martinez Peck wrote:
On Sat, Nov 10, 2012 at 8:18 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote: Maybe pre-allocate the right size:
ByteArray new: byteArray size * 16 streamContents: ...
Well, actually we are trying to do a streaming API for a compressor. So, 1) I don't exactly know the number "16" before hand and 2) doing that is exactly what we are trying to avoid (compress/decompress all together). Instead, we are doing some kind of streaming, compressing in blocks of say 1MB. But I was completely surprised when I saw the results.... So there is nothing easy to improve?
Nicolas
2012/11/10 Mariano Martinez Peck <marianopeck@gmail.com>:
Hi guys:
|byteArray| byteArray := (ByteArray new: 1024*1024). [ ByteArray streamContents: [:s | 16 timesRepeat: [ s nextPutAll:byteArray ] ] ] timeToRun
Gives me around 1 second. Isn't that too much? or is it normal/expected? Anything I could improve?
Thanks,
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
Speed factor is 8x. If you have no idea of required buffer size, you could try to modify growth strategy in your own subclass. The amount of extra free bytes allocated is currently 10 + (oldSize // 4 max: 20). See WriteStream>>#growTo: and its sender #nextPutAll:. Nicolas 2012/11/10 Stéphane Ducasse <stephane.ducasse@inria.fr>:
But mariano two questions: - did it change something? - don't you have an idea of the chunk you will use for the increment?
Stef On Nov 10, 2012, at 8:22 PM, Mariano Martinez Peck wrote:
On Sat, Nov 10, 2012 at 8:18 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote: Maybe pre-allocate the right size:
ByteArray new: byteArray size * 16 streamContents: ...
Well, actually we are trying to do a streaming API for a compressor. So, 1) I don't exactly know the number "16" before hand and 2) doing that is exactly what we are trying to avoid (compress/decompress all together). Instead, we are doing some kind of streaming, compressing in blocks of say 1MB. But I was completely surprised when I saw the results.... So there is nothing easy to improve?
Nicolas
2012/11/10 Mariano Martinez Peck <marianopeck@gmail.com>:
Hi guys:
|byteArray| byteArray := (ByteArray new: 1024*1024). [ ByteArray streamContents: [:s | 16 timesRepeat: [ s nextPutAll:byteArray ] ] ] timeToRun
Gives me around 1 second. Isn't that too much? or is it normal/expected? Anything I could improve?
Thanks,
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
On 10 November 2012 18:33, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Speed factor is 8x. If you have no idea of required buffer size, you could try to modify growth strategy in your own subclass. The amount of extra free bytes allocated is currently 10 + (oldSize // 4 max: 20). See WriteStream>>#growTo: and its sender #nextPutAll:.
yeah , i think for large buffers it much better to use factor x2 growth newsize = oldsize *2
Nicolas
2012/11/10 Stéphane Ducasse <stephane.ducasse@inria.fr>:
But mariano two questions: - did it change something? - don't you have an idea of the chunk you will use for the increment?
Stef On Nov 10, 2012, at 8:22 PM, Mariano Martinez Peck wrote:
On Sat, Nov 10, 2012 at 8:18 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote: Maybe pre-allocate the right size:
ByteArray new: byteArray size * 16 streamContents: ...
Well, actually we are trying to do a streaming API for a compressor. So, 1) I don't exactly know the number "16" before hand and 2) doing that is exactly what we are trying to avoid (compress/decompress all together). Instead, we are doing some kind of streaming, compressing in blocks of say 1MB. But I was completely surprised when I saw the results.... So there is nothing easy to improve?
Nicolas
2012/11/10 Mariano Martinez Peck <marianopeck@gmail.com>:
Hi guys:
|byteArray| byteArray := (ByteArray new: 1024*1024). [ ByteArray streamContents: [:s | 16 timesRepeat: [ s nextPutAll:byteArray ] ] ] timeToRun
Gives me around 1 second. Isn't that too much? or is it normal/expected? Anything I could improve?
Thanks,
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
participants (4)
-
Igor Stasenko -
Mariano Martinez Peck -
Nicolas Cellier -
Stéphane Ducasse