July 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Sockets don't close, 100% CPU

by Janko MivÅ¡ek

S, Henrik Johansen piÅ¡e: >> http://code.google.com/p/pharo/issues/detail?id=4505 >> >> Feel free to comment or contribute. > Holiday starts next week, if weather is bad I hopefully have time for more than yes/no answers ;) Nice to hear that :) And thanks in advance! Janko

July 6, 2011

Re: [Pharo-project] Sockets don't close, 100% CPU

by Sven Van Caekenberghe

On 06 Jul 2011, at 13:09, Henrik Johansen wrote: > > On Jul 6, 2011, at 1:03 56PM, Sven Van Caekenberghe wrote: > >> http://code.google.com/p/pharo/issues/detail?id=4505 >> >> Feel free to comment or contribute. > Holiday starts next week, if weather is bad I hopefully have time for more than yes/no answers ;) > > Cheers, > Henry OK, no sweat! Enjoy your holidays. Sven

July 6, 2011

Re: [Pharo-project] Sockets don't close, 100% CPU

by Henrik Johansen

On Jul 6, 2011, at 1:03 56PM, Sven Van Caekenberghe wrote: > http://code.google.com/p/pharo/issues/detail?id=4505 > > Feel free to comment or contribute. Holiday starts next week, if weather is bad I hopefully have time for more than yes/no answers ;) Cheers, Henry

July 6, 2011

Re: [Pharo-project] Sockets don't close, 100% CPU

by Sven Van Caekenberghe

http://code.google.com/p/pharo/issues/detail?id=4505 Feel free to comment or contribute. On 06 Jul 2011, at 12:47, Henrik Johansen wrote: > > On Jul 6, 2011, at 12:35 17PM, Sven Van Caekenberghe wrote: > >> >> On 06 Jul 2011, at 12:12, Henrik Johansen wrote: >> >>> IIRC, it should be explained in the thread :) >> >> I didn' open all messages ;-) >> >> So #registerExternalObject: and/or #safelyRegisterExternalObject: then. > yes >> Is there already an issue for this, > no >> should we have one, I think so ? > > yes :) > > Cheers, > Henry >

July 6, 2011

Re: [Pharo-project] Sockets don't close, 100% CPU

by Henrik Johansen

On Jul 6, 2011, at 12:35 17PM, Sven Van Caekenberghe wrote: > > On 06 Jul 2011, at 12:12, Henrik Johansen wrote: > >> IIRC, it should be explained in the thread :) > > I didn' open all messages ;-) > > So #registerExternalObject: and/or #safelyRegisterExternalObject: then. yes > Is there already an issue for this, no > should we have one, I think so ? yes :) Cheers, Henry

July 6, 2011

Re: [Pharo-project] Sockets don't close, 100% CPU

by Sven Van Caekenberghe

On 06 Jul 2011, at 12:12, Henrik Johansen wrote: > IIRC, it should be explained in the thread :) I didn' open all messages ;-) So #registerExternalObject: and/or #safelyRegisterExternalObject: then. Is there already an issue for this, should we have one, I think so ? Sven

July 6, 2011

Re: [Pharo-project] Sockets don't close, 100% CPU

by Henrik Johansen

On Jul 6, 2011, at 11:52 29AM, Sven Van Caekenberghe wrote: > > On 06 Jul 2011, at 11:19, Henrik Johansen wrote: > >> http://forum.world.st/Networking-change-in-Pharo-1-2-tp3456097p3461723.html >> should explain it. >> >> TLDR; Changes in the image are required to work with Cog without silently failing to signal semaphores when too many Sockets are open. > > OK, I think I understand more or less. Thanks, Hendrik. > > Where does it fail silently ? I IIRC, it should be explained in the thread :) > MHO that should not happen: the failure should be hard as in 'I am running out of semaphores' or whatever. That would help in debugging situations like this. > > Sven There are two good options: 1) Increase the external table size if possible, and log a warning that some signals may have been lost and you really should increase the size at startup. 2) Raise an error. I think you'd want 1 in a production environment, and 2 in development. Cheers, Henry

July 6, 2011

Re: [Pharo-project] Sockets don't close, 100% CPU

by Sven Van Caekenberghe

On 06 Jul 2011, at 11:19, Henrik Johansen wrote: > http://forum.world.st/Networking-change-in-Pharo-1-2-tp3456097p3461723.html > should explain it. > > TLDR; Changes in the image are required to work with Cog without silently failing to signal semaphores when too many Sockets are open. OK, I think I understand more or less. Thanks, Hendrik. Where does it fail silently ? IMHO that should not happen: the failure should be hard as in 'I am running out of semaphores' or whatever. That would help in debugging situations like this. Sven

July 6, 2011

Re: [Pharo-project] Sockets don't close, 100% CPU

by Janko MivÅ¡ek

S, Sven Van Caekenberghe piÅ¡e: > Janko MivÅ¡ek wrote: >> If I recall others have also problems with sockets in Pharo, so here is >> the my current image, which: >> >> - has 105 Sockets open, mostly waiting >> - has also >105 processes open >> - consumes 100% cpu >> - on aSocket close responds with primitive failure >> - image is workable, I can browse etc. >> >> - Pharo1.2.2a on Linux >> >> Sockets are waiting on Socket>>waitForConnectionFor:ifTimeOut: > Is that number of open sockets and processes by design or not ? If they keep coming back, you must be creating them. Your application has to keep these (expensive) resources under control to start with. This is Swazoo accepting the connections, probably from web spiders. But also promptly closing them, because there are no instances of HTTPConnection hanging around. > Long running Seaside images seem to be fine and they are using sockets and processes a lot. > > Socket have #close and #destroy, there might be a difference. Swazoo uses Sport portability layer and its SpSocket actually call #closeAndDestroy on underlying Socket. Obviously this method don't manage to actually close nor destroy the socket. > I think that part of the cleanup of sockets is delayed (using finalization) and needs some GC work to complete. GC policies might be involved as well as weak datastructures. This could be a way to proceed. How to diagnose that? > I am also interested in knowning the practical upper bound for open sockets and number of processes. >From Henrik pointer it seems that in 1.2 one-click there can be up to 512 semaphores by default. Every socket need three, so up to 170 sockets can be open at once. Semaphore limit can be increased , to say 1024*3 for 1024 sockets by: Smalltalk vmParameterAt: 49 put: 1024*3 Another limit on Linux is namely a default 1024 limit of sockets per a whole machine. This one can also be increased Best regards Janko -- Janko MivÅ¡ek Aida/Web Smalltalk Web Application Server http://www.aidaweb.si

July 6, 2011

[Pharo-project] [Jenkins] State of transition Hudson -> Jenkins

by Marcus Denker

State of transition Hudson -> Jenkins ============================= We are slowly (step by step) moving builds from Hudson to Jenkins. the last days e.g. Coral (the scripting engine), RPackage, Nautilus... https://ci.lille.inria.fr/pharo/ The largest thing to move will be the VM autobuilt... that will be done later. Jean Baptiste Arnaud started with a setup for regression testing. https://ci.lille.inria.fr/pharo/job/Pharo%201.4%20Regression%20Tests/ This means "doing something slow" "run all tests" get new image "doing something other" "run all tests" The idea here is that this will be used for the following -> recompile the image -> decompile and recompile the decompiled code -> pretty print -> condense sources -> condense changes .... On the TODO list we have -> test the compiled VMs by running all tests on all plattforms -> update the One-click build script with the last good VM -> update the VM used to do everything with the self-build vm The goal is of course to have that what we release be build completely without human intervention, after every commit (either VM source or Image level change). In addtion, this setup will be available for others to use. This way students can easily clone the VM source and set up Continuous Integration for their research Fork. This will than make it far far easier for people to invent cool stuff that needs VM changes, like e.g. this: http://scg.unibe.ch/archive/papers/Lien08bBackInTimeDebugging.pdf or http://rmod.lille.inria.fr/archives/papers/Arna10a-Tools2010-ReadOnlyExecut… The problem right now is that all the code is "somewhere" and just picking up where the student left is a huge amount of work... with Git and autbuild, this will get much much better. (and it simplifies the life of the student a lot... autobuild, easy to track changes in the upstream...) JB already successfully builds his Handle VM: https://pharo-ic.lille.inria.fr/hudson/job/Handles/ So we are getting there. Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.

July 6, 2011