October 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Socket timeout terminates COG VM

by John Toohey

I think the main source of the exceptions is in the WAListenerAdapter in the #process: aNativeRequest (Implemented in WAServerAdapter) My app uses Comet so I need a streaming server. I've wrapped an exception handler around this, and am testing now. Overall I think that the socket code, if it is going to detect timeouts, it should retry and the just close and cleanup that socket. If the calling app ever calls on that socket, then an exception should be thrown, and in this case, the app would be expecting it. On Mon, Oct 10, 2011 at 21:34, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu> wrote: > Not knowing what exception this is, I'm not sure whether to agree or argue. Â However, given the importance of error handling to robust network apps, it should get handled. > > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua(a)gmail.com] > Sent: Monday, October 10, 2011 7:05 PM > To: Pharo-project(a)lists.gforge.inria.fr > Subject: Re: [Pharo-project] Socket timeout terminates COG VM > > On 11 October 2011 01:27, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu> wrote: >> John, >> >> No question, the punishment certainly exceeds the crime :) Â Â I was making a general plea for what I really believe is the correct way to build network software. >> > So you prefer leaving an image in some undefined state, potentially > leaking system resources, > because of unhandled exception which nobody cares to handle? > At best what you can have then is, when your critical process will > enter an endless error-producing loop > is just bogging all system resources or hanging an image, without > giving you any idea what's going on. > So , if you prefer such kind of punishment for your crimes, just use > image prior to 1.3 and be happy :) > >> Bill >> >> >> >> >> ________________________________________ >> From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of John Toohey [jt(a)parspro.com] >> Sent: Monday, October 10, 2011 5:24 PM >> To: Pharo-project(a)lists.gforge.inria.fr >> Subject: Re: [Pharo-project] Socket timeout terminates COG VM >> >> The exception is not coming from my code, if it was I could trap it. >> Regardless, isn't shutting the VM down a bit of overkill in this case? >> >> On Mon, Oct 10, 2011 at 17:12, Igor Stasenko <siguctua(a)gmail.com> wrote: >>> On 10 October 2011 23:19, John Toohey <jt(a)parspro.com> wrote: >>>> Hi, >>>> >>>> I've had a problem since 1.0 with socket timeouts throwing exceptions >>>> and invoking the debugger in my headless images. I don't know why a >>>> socket timeout is considered exceptional, but I've gotten used to >>>> using VNC to log into the server and close the debuger windows. >>>> However, I've moved to COG and the latest one-click images, and now >>>> the error terminates the VM. >>>> >>>> My app is a standard Seaside server, using the streaming connector for >>>> Comet support. >>>> >>>> I've attached the PharoDebug.log. It would be appreciated if anyone >>>> could help me with this problem, as I'm at a loss where to even begin. >>>> >>> >>> So, just put an exception handler, and handle this exception in a >>> manner you want >>> and then your image will keep working! :) >>> >>>> >>>> Using Pharo1.3 #13298 on Lucid32 (Ubuntu) >>>> >>>> >>>> -- >>>> ~JT >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Igor Stasenko. >>> >>> >> >> >> >> -- >> ~JT >> >> >> > > > > -- > Best regards, > Igor Stasenko. > > > -- ~JT

Oct. 11, 2011

Re: [Pharo-project] Socket timeout terminates COG VM

by Schwab,Wilhelm K

Not knowing what exception this is, I'm not sure whether to agree or argue. However, given the importance of error handling to robust network apps, it should get handled. ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua(a)gmail.com] Sent: Monday, October 10, 2011 7:05 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] Socket timeout terminates COG VM On 11 October 2011 01:27, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu> wrote: > John, > > No question, the punishment certainly exceeds the crime :) I was making a general plea for what I really believe is the correct way to build network software. > So you prefer leaving an image in some undefined state, potentially leaking system resources, because of unhandled exception which nobody cares to handle? At best what you can have then is, when your critical process will enter an endless error-producing loop is just bogging all system resources or hanging an image, without giving you any idea what's going on. So , if you prefer such kind of punishment for your crimes, just use image prior to 1.3 and be happy :) > Bill > > > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of John Toohey [jt(a)parspro.com] > Sent: Monday, October 10, 2011 5:24 PM > To: Pharo-project(a)lists.gforge.inria.fr > Subject: Re: [Pharo-project] Socket timeout terminates COG VM > > The exception is not coming from my code, if it was I could trap it. > Regardless, isn't shutting the VM down a bit of overkill in this case? > > On Mon, Oct 10, 2011 at 17:12, Igor Stasenko <siguctua(a)gmail.com> wrote: >> On 10 October 2011 23:19, John Toohey <jt(a)parspro.com> wrote: >>> Hi, >>> >>> I've had a problem since 1.0 with socket timeouts throwing exceptions >>> and invoking the debugger in my headless images. I don't know why a >>> socket timeout is considered exceptional, but I've gotten used to >>> using VNC to log into the server and close the debuger windows. >>> However, I've moved to COG and the latest one-click images, and now >>> the error terminates the VM. >>> >>> My app is a standard Seaside server, using the streaming connector for >>> Comet support. >>> >>> I've attached the PharoDebug.log. It would be appreciated if anyone >>> could help me with this problem, as I'm at a loss where to even begin. >>> >> >> So, just put an exception handler, and handle this exception in a >> manner you want >> and then your image will keep working! :) >> >>> >>> Using Pharo1.3 #13298 on Lucid32 (Ubuntu) >>> >>> >>> -- >>> ~JT >>> >> >> >> >> -- >> Best regards, >> Igor Stasenko. >> >> > > > > -- > ~JT > > > -- Best regards, Igor Stasenko.

Oct. 11, 2011

Re: [Pharo-project] When will Fuel file format stabilize?

by Yanni Chiu

On 10/10/11 2:43 PM, Mariano Martinez Peck wrote: > > On Mon, Oct 10, 2011 at 6:50 PM, Yanni Chiu <yanni(a)rogers.com > > For the case where a lot of serialized files need to be converted, > having two versions in memory is almost a necessity; > > > heheheheh how people could live with ReferenceStream for 10 years? > > otherwise, the Fuel code update would have to be applied for each > file being converted. > > > but answer my question: why would you need to update Fuel so frequently ? I don't need to update Fuel frequently. I'm currently "stuck" at some pre-1.6 version of Fuel (some combination of package versions that I managed to get working at the time, using early Fuel releases). I want to migrate to a newer version at some point (to use some new Fuel features), but the hold up is that I would have to migrate about a dozen serialization files - probably manually, since a dozen is roughly at my threshold to automate. However, I expect that the number of files to migrate would be a larger number in the future. At that point I'd automate the: deserialize with old Fuel, update to newer Fuel and serialize again. The migration procedure seems safer to me, if it did not load code in the middle of the process. There's lots of workarounds for this issue, so it's not a high priority. > A separate issue is that at some point I wanted to serialize > something differently, in two different contexts. In one case, > serializing a Pier kernel, I want Fuel to just do its thing. In the > other case, when exporting a portion a Pier kernel, I want to fine > tune (i.e. map to different objects) the objects being > exported/imported. There is a hook to allow the object swap, but the > swap would then take place for the other case of serializing to > whole kernel (where the desired behaviour was for no such mapping). > > > That's an interesting point....Let me see if I understood. Let's say > that you want to exclude certain instance variables of a class. What you > mean is that you want to apply this hook (or another one) in an specific > serialization and not always? because right now with Fuel (and SIXX, and > I guess StOMP also) since you put the hooks in methods of the class, > they are always taken into account. How would the API for the user > should be? you can you specify for an specific serialization run that > certain hook of certain object/class must be applied and some should not ? Yes, you've got the idea. No idea what the API should be for such functionality - probably a serialization context (passed in at the start) that answers configuration parameters to use to pick an object mapping method. > Apart from that, do you have a real example where you need such behavior > ? Beause it looks quite complex/rare scenario ;) My serialization code is not top of mind, but I think the issue I had was that when exporting a portion of a Pier kernel, for importing into another Pier kernel, the security objects should be nil'ed out. However, when using Fuel to serialize the kernel, the security objects must be serialized instead of being nil'ed out. Not sure why I don't currently have a bug - probably, the security is being inherited from the parent structures when they're nil. I'll see the problem once I try to set different security on lower down structures - they'll be lost on restart. -- Yanni

Oct. 11, 2011

Re: [Pharo-project] Pharo Consortium News: let's make it real.

by Alexandre Bergel

> - We are wondering if we could have a academic fee for the institution that want it: > we have to discuss the fee. Any suggestions would be good: I was thinking that 500 Euros would be good for rich countries > and 200 for the other ones. Now tell us. As you can imagine, pulling out some money is not easy for a University. However, what can be done (at least for me), is to allocate money to invite a PhD holder to stay at the Uni of Chile and work on a topic. I should also be able to buy books (why not a special edition of Pharo by Example at 80 euros?) > - We could also have way to pay a license or a way to give money to the consortium. +1 Alexandre > > Now the complex part: > - we would like to collect money from companies and again we will need your help > to understand what is the correct model. > We are thinking: > single person company: starting at 300 Euros > "small company": starting at 2500 Euros > large companies: starting at 5000 Euros. > > We would like to keep the model really simple and transparent. > > Now our goal is to be able to pay an engineer like igor full time because there is so much to do. > But each day we get closer to what we want. > > Stef and Marcus. > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Oct. 11, 2011

Re: [Pharo-project] Socket timeout terminates COG VM

by Igor Stasenko

On 11 October 2011 02:03, John Toohey <jt(a)parspro.com> wrote: > Yep, now even if I fix this, I'll have to move each of my apps to a > separate VM. Dynamic typing and Web apps are not always a perfect > match, and I don't want an error in one bringing everything down. > > I'm guessing that this was not a problem for me before, as the older > images or the squeak VM started the UIManager even in headless mode. > Therefore when I used VNC to check the servers, I would see the > debugger windows. > You could hack the UnhandledError default action to suit your needs. There's many ways to force system to behave like you want. > > On Mon, Oct 10, 2011 at 18:27, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu> wrote: >> John, >> >> No question, the punishment certainly exceeds the crime :) Â Â I was making a general plea for what I really believe is the correct way to build network software. >> >> Bill >> >> >> >> >> ________________________________________ >> From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of John Toohey [jt(a)parspro.com] >> Sent: Monday, October 10, 2011 5:24 PM >> To: Pharo-project(a)lists.gforge.inria.fr >> Subject: Re: [Pharo-project] Socket timeout terminates COG VM >> >> The exception is not coming from my code, if it was I could trap it. >> Regardless, isn't shutting the VM down a bit of overkill in this case? >> >> On Mon, Oct 10, 2011 at 17:12, Igor Stasenko <siguctua(a)gmail.com> wrote: >>> On 10 October 2011 23:19, John Toohey <jt(a)parspro.com> wrote: >>>> Hi, >>>> >>>> I've had a problem since 1.0 with socket timeouts throwing exceptions >>>> and invoking the debugger in my headless images. I don't know why a >>>> socket timeout is considered exceptional, but I've gotten used to >>>> using VNC to log into the server and close the debuger windows. >>>> However, I've moved to COG and the latest one-click images, and now >>>> the error terminates the VM. >>>> >>>> My app is a standard Seaside server, using the streaming connector for >>>> Comet support. >>>> >>>> I've attached the PharoDebug.log. It would be appreciated if anyone >>>> could help me with this problem, as I'm at a loss where to even begin. >>>> >>> >>> So, just put an exception handler, and handle this exception in a >>> manner you want >>> and then your image will keep working! :) >>> >>>> >>>> Using Pharo1.3 #13298 on Lucid32 (Ubuntu) >>>> >>>> >>>> -- >>>> ~JT >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Igor Stasenko. >>> >>> >> >> >> >> -- >> ~JT >> >> >> > > > > -- > ~JT > > -- Best regards, Igor Stasenko.

Oct. 11, 2011

Re: [Pharo-project] Socket timeout terminates COG VM

by Igor Stasenko

On 11 October 2011 01:27, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu> wrote: > John, > > No question, the punishment certainly exceeds the crime :) Â Â I was making a general plea for what I really believe is the correct way to build network software. > So you prefer leaving an image in some undefined state, potentially leaking system resources, because of unhandled exception which nobody cares to handle? At best what you can have then is, when your critical process will enter an endless error-producing loop is just bogging all system resources or hanging an image, without giving you any idea what's going on. So , if you prefer such kind of punishment for your crimes, just use image prior to 1.3 and be happy :) > Bill > > > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of John Toohey [jt(a)parspro.com] > Sent: Monday, October 10, 2011 5:24 PM > To: Pharo-project(a)lists.gforge.inria.fr > Subject: Re: [Pharo-project] Socket timeout terminates COG VM > > The exception is not coming from my code, if it was I could trap it. > Regardless, isn't shutting the VM down a bit of overkill in this case? > > On Mon, Oct 10, 2011 at 17:12, Igor Stasenko <siguctua(a)gmail.com> wrote: >> On 10 October 2011 23:19, John Toohey <jt(a)parspro.com> wrote: >>> Hi, >>> >>> I've had a problem since 1.0 with socket timeouts throwing exceptions >>> and invoking the debugger in my headless images. I don't know why a >>> socket timeout is considered exceptional, but I've gotten used to >>> using VNC to log into the server and close the debuger windows. >>> However, I've moved to COG and the latest one-click images, and now >>> the error terminates the VM. >>> >>> My app is a standard Seaside server, using the streaming connector for >>> Comet support. >>> >>> I've attached the PharoDebug.log. It would be appreciated if anyone >>> could help me with this problem, as I'm at a loss where to even begin. >>> >> >> So, just put an exception handler, and handle this exception in a >> manner you want >> and then your image will keep working! :) >> >>> >>> Using Pharo1.3 #13298 on Lucid32 (Ubuntu) >>> >>> >>> -- >>> ~JT >>> >> >> >> >> -- >> Best regards, >> Igor Stasenko. >> >> > > > > -- > ~JT > > > -- Best regards, Igor Stasenko.

Oct. 10, 2011

Re: [Pharo-project] Socket timeout terminates COG VM

by John Toohey

Yep, now even if I fix this, I'll have to move each of my apps to a separate VM. Dynamic typing and Web apps are not always a perfect match, and I don't want an error in one bringing everything down. I'm guessing that this was not a problem for me before, as the older images or the squeak VM started the UIManager even in headless mode. Therefore when I used VNC to check the servers, I would see the debugger windows. On Mon, Oct 10, 2011 at 18:27, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu> wrote: > John, > > No question, the punishment certainly exceeds the crime :) Â Â I was making a general plea for what I really believe is the correct way to build network software. > > Bill > > > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of John Toohey [jt(a)parspro.com] > Sent: Monday, October 10, 2011 5:24 PM > To: Pharo-project(a)lists.gforge.inria.fr > Subject: Re: [Pharo-project] Socket timeout terminates COG VM > > The exception is not coming from my code, if it was I could trap it. > Regardless, isn't shutting the VM down a bit of overkill in this case? > > On Mon, Oct 10, 2011 at 17:12, Igor Stasenko <siguctua(a)gmail.com> wrote: >> On 10 October 2011 23:19, John Toohey <jt(a)parspro.com> wrote: >>> Hi, >>> >>> I've had a problem since 1.0 with socket timeouts throwing exceptions >>> and invoking the debugger in my headless images. I don't know why a >>> socket timeout is considered exceptional, but I've gotten used to >>> using VNC to log into the server and close the debuger windows. >>> However, I've moved to COG and the latest one-click images, and now >>> the error terminates the VM. >>> >>> My app is a standard Seaside server, using the streaming connector for >>> Comet support. >>> >>> I've attached the PharoDebug.log. It would be appreciated if anyone >>> could help me with this problem, as I'm at a loss where to even begin. >>> >> >> So, just put an exception handler, and handle this exception in a >> manner you want >> and then your image will keep working! :) >> >>> >>> Using Pharo1.3 #13298 on Lucid32 (Ubuntu) >>> >>> >>> -- >>> ~JT >>> >> >> >> >> -- >> Best regards, >> Igor Stasenko. >> >> > > > > -- > ~JT > > > -- ~JT

Oct. 10, 2011

Re: [Pharo-project] Socket timeout terminates COG VM

by John Toohey

Henrik, Could you explain this in a little more detail for me? Are you saying the socket timeout is really a semaphore problem? On Mon, Oct 10, 2011 at 17:58, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > On 10.10.2011 23:28, John Toohey wrote: >> >> I agree. In my own networking code, my sockets handlers on both side >> never timeout. I have a manager/monitor process that I use if I have >> to kill them. > > Then raise Smalltalk vm maxExternalSemaphoresSilently: to an appropriate > number (max amount of socket handlers * 3 + some headroom), save your image, > and you should be set. While it has a max limit of 64k, the socket plugin > code on at least windows will max out at much less than 20k open sockets, as > it uses 3 OS threads per socket. (Around 1k IIRC) > Semaphore timeouts were the old symptoms of running out of this space, now > it gives you an explicit exception instead. > > maxExternalSemaphores: Â is what is raising the error, some info _should_ be > written to wherever crLog: redirects to with a NonInteractiveUIManager > before the image terminates. > > Cheers, > Henry > > -- ~JT

Oct. 10, 2011

Re: [Pharo-project] [squeak-dev] Any serializer can customize serialization strategy?

by HernÃ¡n Morales Durand

Thanks Mariano, That's pretty much what I want, but... why to expose the visitor interface? ;) Just to give a little feedback of what I have in my mind: mySerializer setEngine: ExternalCompressor for: ( MyClass -> #myStringIVar ) or mySerializer setEngine: ExternalCompressor for: incomingStream Now with machines generating daily 6 TBytes of raw data, one of the keys will be data compression. 2011/10/10 Mariano Martinez Peck <marianopeck(a)gmail.com>: > > > On Mon, Oct 10, 2011 at 9:35 PM, HernÃ¡n Morales Durand > <hernan.morales(a)gmail.com> wrote: >> >> Is there any serializer (Fuel/StOMP/Ma object >> serialization/BOSS/SRP/SIXX??) which let me attach an external >> compressor/decompressor for specific Strings to the >> serialization/deserialization process? > > In Fuel you can do: > > String >> fuelSubstitution > ^ ExternalCompressor compress: self > > String >> fuelAccept: aVisitor > ^ (ExternalCompressor shouldCompressString: self) > ifTrue: [aVisitor visitSubstitution: self] > ifFalse: [super fuelAccept: aVisitor] > > But the problem is that if #compress: answers an instance of String, you > should be careful that if #shouldCompressString:Â asnwers true to the > compressed string, then you will end up in an infinitive loop. > > See: http://rmod.lille.inria.fr/web/pier/software/Fuel/Hooks and > FLHookedSubstitutionTest > > I think StOMP should support this as well: > http://stomp.smalltalk-users.jp/home/how-to-use-stomp/hook-methods > > The previous hook you let you replace the original String with its > compression during serialization. Which make sense because if you are going > to compress it is better to do it during serialization. > >> With external I mean it could >> be in a separate binary (I could arrange the binding though). > > I am not sure if I understood. > > > >> >> Thanks >> >> HernÃ¡n >> > > > > -- > Mariano > http://marianopeck.wordpress.com > >

Oct. 10, 2011

Re: [Pharo-project] When will Fuel file format stabilize?

by Schwab,Wilhelm K

Last one (I think<g>): I have an active study with data spanning about six years. I wish I had known then some of the things I know now, but I'm not in a hurry to ditch data simply because they are old - sample size, rare events, and all that good stuff. ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Schwab,Wilhelm K [bschwab(a)anest.ufl.edu] Sent: Monday, October 10, 2011 6:32 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] When will Fuel file format stabilize? The data in question are medical records - they *really* need to load when demanded. ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice(a)gmail.com] Sent: Monday, October 10, 2011 5:30 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] When will Fuel file format stabilize? 2011/10/10 Igor Stasenko <siguctua(a)gmail.com>: > On 11 October 2011 00:19, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu> wrote: >> True, but what about reading data from five years ago? I have ten year old Dolphin STB that I can still load. >> >> > to my experience a 10 years old data has very small chances to be relevant :) > The problem is that if it ever is relevant, you have very small chance to know because you cannot load ;) Nicolas >> >> ________________________________________ >> From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua(a)gmail.com] >> Sent: Monday, October 10, 2011 4:59 PM >> To: Pharo-project(a)lists.gforge.inria.fr >> Subject: Re: [Pharo-project] When will Fuel file format stabilize? >> >> I concur with Mariano. >> In production images, you want probably stick with concrete >> verision(s) of frameworks you using and don't >> follow the bleeding edge. Because if you do, then you spending time on >> coping up with latest and greatest >> versions of these frameworks instead of spending time on developing >> your project :) >> >> -- >> Best regards, >> Igor Stasenko. >> >> >> > > > > -- > Best regards, > Igor Stasenko. > >

Oct. 10, 2011