Good, I am happy I could help. It would always be interesting to read a blog post or article describing your application, one day. On 25 Jan 2013, at 18:13, Chris <cpmbailey@btinternet.com> wrote:
On 24/01/2013 12:55, Sven Van Caekenberghe wrote:
On 24 Jan 2013, at 13:37, CHRIS BAILEY <cpmbailey@btinternet.com> wrote:
--- On Thu, 24/1/13, Sven Van Caekenberghe <sven@stfx.eu> wrote:
From: Sven Van Caekenberghe <sven@stfx.eu> Subject: Re: [Pharo-users] Memory Leak with Zodiac? To: "A friendly place where any question about pharo is welcome" <pharo-users@lists.gforge.inria.fr> Date: Thursday, 24 January, 2013, 10:41 Hi Chris,
On 24 Jan 2013, at 11:02, CHRIS BAILEY <cpmbailey@btinternet.com> wrote:
I'm having a little trouble with a possible memory leak with Zodiac. I'm running Pharo 1.4 Summer on Centos 6.3/Fedora 18 for deploy/dev. The plugin is using libssl.so.0.9.8, and I also seem to have another version kicking about which uses libssl.so.6 and am not really sure where I got that from. It may be an older one from the http://code.google.com/p/squeakssl/ site. Both have the same effect, i.e. when running 100 timesRepeat: [ ZnClient new url: 'https://www.google.com'; get ] VIRT and RES memory both go up about 70mb, but just using http doesn't accumulate any. I tried it in Pharo 2.0 which was the same, but the new NBCog doesn't include the libSqueaKSSL.so so I was only copying it over. Has anyone used it successfully in the same environment? I'll try it on Ubuntu when I get a chance. Thanks Chris Thanks for testing this ;-)
Note that although ZnClient is most often used to do just one request, it was also designed to be reused to do multiple request to the same host:port. Even when not taking care of resource cleanup, for casual use #close to the socket will eventually be sent where necessary due to finalization. Helping a bit makes for better performance.
Either one of the following would be better:
100 timesRepeat: [ ZnClient new beOneShot; get: 'https://www.google.com' ].
100 timesRepeat: [ ZnClient new get: 'https://www.google.com'; close ].
| client | client := ZnClient new. 100 timesRepeat: [ client get: 'https://www.google.com' ]. client close.
Now, Zodiac TLS/SSL Streams are more resource intensive: they create ZdcPluginSSLSession instances which represent native native data structures from the OS's SSL libraries. These are only cleaned up when #destroy is sent, which is one by ZdcSecureSocketStream>>#close. I didn't write any explicit code to do this by finalization, so that probably does not happen. Maybe we should experiment with adding a #finalize to ZdcPluginSSLSession as an equivalent to #destroy.
It would be cool if you could test again. Hopefully memory consumption improves. Hi Sven, thanks for the quick reply.
Sorry, I didn't consider that when I reproduced it with the simplest example I could think of! I'm actually using it in the context of a single ZnClient being held open for an entire Seaside session which talks to an external web service. I therefore only send a close when the session ends. What solution would you recommend, as it is quite easy currently to run the image out of memory before the session gets unregistered. That is more a question about Seaside session management, which would be similar to managing database connections. My Seaside knowledge is rusty, but if I remember correctly, you can arrange for a method to be called on your custom Seaside session object whenever a session expires, there you could do some cleanup. Making sure sessions are expired quickly enough is also tuneable in various ways.
What I have done in some projects is to have a single small pool of ZnClients ready to talk to a specific service. Each session then calls out to the pool to access the external service over HTTP(S). Of course, you have to be careful with multithreaded access, but also with timeouts, resource starvation, failures and cleanup, unwanted state sharing or caching, etc..
Or you could just try to be aggressive in closing the HTTP(S) clients after each use. There is not much point in keep those connections open for more than a couple of seconds as the other end will close them anyway. Thanks for that, I've just gone with closing it each time and it works fine now :-) I've had DBXConnections and the http version of ZnClient working fine with one instance for the session, and such that it gets closed at unregister but they don't accumulate any memory when used and left open. Each call was generating about 1mb so it was very easy to run out of memory even with quite strict session cleanup.
Chris