Re: [Pharo-project] about growing streams and allocating buffers
"Philippe Marschall"<kustos@gmx.net> wrote:
preallocation OrderedCollection new: instead of relying on its growing behavior.
Right, but you never know how big the response is going to before actually rendering. Otherwise you could just do Array/String new: beforehand.
I'm not quite clear about the context here so this may not be relevant, but one technique that paid of for us in terms of managing socket buffers was to start recycling them. The idea is that if you generally allocate let's say 32K buffer for every socket you open, rather then throwing the buffer away when you're done with the socket, just put it aside and let the next socket pick up and reuse the same instance. For us the main concern was a server being hammered by transient connection requests. Occasional buffer allocation could be handled by new space scavenger just fine, but a flood of such requests made the buffers spill over to the old space and throwing them away got expensive quickly. Anyway, I realize that your concern is different here, but the same technique might help growing in most common cases. If there's a reasonable upper bound on most responses (e.g. 32K), you could simply preallocate that size always. If the request fits, great, no growing, if not, it will grow. However you'd probably have the same sort of problem as we had above if you simply threw each buffer away. Recycling should help making it manageable. We use the same technique in Xtreams, where we need internal buffers quite frequently. Check out the RecyclingCenter class there if you want. It's a bit more generic because we need to accommodate different collection classes, so you'll probably want something different, but the basics should be similar. One aspect to note is that the recycling pool can be quite small. You don't need to recycle every instance created, you only need to be able to handle the "overflow" under load. The idea is that if you are under load and the number of concurrently used buffers fluctuates between 100 and 110, you don't need the recycling pool size to be 110, but only 10. It only needs to retain the temporarily unused buffers long enough for those to get picked up again. When the load subsides, it's fine to let the remaining 100 buffers GC, they can be allocated again quickly when next wave hits. At the same time it keeps the memory load reasonable in the quiet periods. HTH, Martin
On 04. 01. 2011 04:25, mkobetic@gmail.com wrote:
"Philippe Marschall"<kustos@gmx.net> wrote:
preallocation OrderedCollection new: instead of relying on its growing behavior.
Right, but you never know how big the response is going to before actually rendering. Otherwise you could just do Array/String new: beforehand.
I'm not quite clear about the context here so this may not be relevant, but one technique that paid of for us in terms of managing socket buffers was to start recycling them. The idea is that if you generally allocate let's say 32K buffer for every socket you open, rather then throwing the buffer away when you're done with the socket, just put it aside and let the next socket pick up and reuse the same instance.
Yep, this is a technique used in Swazoo too. When you nextPutAll: to Swazoo's HTTPResponse, you actually write directly to the SwazooBuffer, which has such recyclable collection. Main reason to introduce those buffers was exactly to reduce garbage collection activity to minimum, which is very important when serving (or uploading) large content. Second reason is that network packets must be packaged for network performance into a reasonably big packets and not into a lot of small ones after each nextPutAll:. Best regards Janko -- Janko Mivšek AIDA/Web Smalltalk Web Application Server http://www.aidaweb.si
On 04.01.2011 04:25, mkobetic@gmail.com wrote:
"Philippe Marschall"<kustos@gmx.net> wrote:
preallocation OrderedCollection new: instead of relying on its growing behavior.
Right, but you never know how big the response is going to before actually rendering. Otherwise you could just do Array/String new: beforehand.
I'm not quite clear about the context here so this may not be relevant, but one technique that paid of for us in terms of managing socket buffers was to start recycling them. The idea is that if you generally allocate let's say 32K buffer for every socket you open, rather then throwing the buffer away when you're done with the socket, just put it aside and let the next socket pick up and reuse the same instance. For us the main concern was a server being hammered by transient connection requests. Occasional buffer allocation could be handled by new space scavenger just fine, but a flood of such requests made the buffers spill over to the old space and throwing them away got expensive quickly.
Well yes and no. There are several issues. The first is that the growing policy of #nextPut: and #nextPutAll: is different. I believe #nextPut: has it right. That shows in general where you build a huge collection using Streams, one example is Seaside response generation but there are others. Partially related to that is that allocating huge collections is expensive. So Seaside/the server should reuse a buffer for response rendering. The problem is that this needs server support. If you have a 64k buffer and use 90% of that you don't want to send #contents to this because the server can handle only ByteArray/String. And you don't want the server having to do ByteArray/String conversion. AJP [1] has an experimental implementation of this and very happy with the results. When we have a streaming server all of this is not really a problem because we can/could directly render into the socket buffer. Issue 591 contains some clean up to make this approach more general/portable.
Anyway, I realize that your concern is different here, but the same technique might help growing in most common cases. If there's a reasonable upper bound on most responses (e.g. 32K), you could simply preallocate that size always. If the request fits, great, no growing, if not, it will grow. However you'd probably have the same sort of problem as we had above if you simply threw each buffer away. Recycling should help making it manageable.
We use the same technique in Xtreams, where we need internal buffers quite frequently. Check out the RecyclingCenter class there if you want. It's a bit more generic because we need to accommodate different collection classes, so you'll probably want something different, but the basics should be similar. One aspect to note is that the recycling pool can be quite small. You don't need to recycle every instance created, you only need to be able to handle the "overflow" under load. The idea is that if you are under load and the number of concurrently used buffers fluctuates between 100 and 110, you don't need the recycling pool size to be 110, but only 10. It only needs to retain the temporarily unused buffers long enough for those to get picked up again. When the load subsides, it's fine to let the remaining 100 buffers GC, they can be allocated again quickly when next wave hits. At the same time it keeps the memory load reasonable in the quiet periods.
No offense but such things quickly become like your own memory manager. Wasn't this GC stuff supposed to free (pun) me from this? [1] http://www.squeaksource.com/ajp Cheers Philippe
participants (3)
-
Janko Mivšek -
mkobetic＠gmail.com -
Philippe Marschall