November 2010 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] #& in Socket >> #waitForSendDoneFor:

by Levente Uzonyi

On Tue, 9 Nov 2010, Philippe Marschall wrote: > Hi > > I have a case where the #& in Socket >> #waitForSendDoneFor: shows up in > a tally (see attachment). I'm able to push about 1 Mbyte/s more when I > replace that with #and:. So my question is this really needed or should > I file a bug and send a patch? I "fixed" this once by for Squeak, but I didn't push this change. Here's the boolean expression: (sendDone := self primSocketSendDone: socketHandle) not and: [ self isConnected and: [ "Connection end and final data can happen fast, so test in this order" Time millisecondClockValue < deadline] ] This refactoring saves one #isConnected send which may not be significant. The appearence of #& in MessageTally may be just false measurement. IIRC with the current VMs the time spent in primitives are added to the next real message send (for performance reasons), which means that the #primSocketConnectionStatus: from #isConnected may be showing up there. Let me know if this change helps. Levente > > This is Pharo 1.1.1 BTW. > > Cheers > Philippe >

Nov. 9, 2010

Re: [Pharo-project] Process-specific state broken and uncomplete

by Igor Stasenko

On 9 November 2010 08:06, Philippe Marschall <kustos(a)gmx.net> wrote: > On 08.11.2010 06:39, Igor Stasenko wrote: >> Hello, >> >> i just found that Squeak images Process class contains unused ivars: >> island env >> >> in Pharo image, env ivar is used for holding a process-specific state, >> which is convenient, so one could use: >> Processor activeProcess environmentAt: put:... >> >> but in Squeak image there is even no such methods. >> >> There are also some caveats with such thing: >> during process termination, Processor activeProcess could point to >> different process, >> and so, accessing to process environment may lead to error(s). >> >> To avoid issues like these, the process termination action (stack >> unwinding) should be always performed only for active process. >> One of a trick i think, is to replace the suspendedContext with own >> context, and then activate given process in order to unwind its stack >> etc. >> >> Another way is to remember environment somewhere else during process >> termination, and use different method to access it i.e. instead of: >> Processor activeProcess environmentAt: >> use >> self processEnvironmentAt: >> >> where #processEnvironmentAt: is implemented in Object class, and can >> handle termination gracefully. >> >> But i think replacing suspendedContext is more generic. >> What you think? > > Not so convinced. Fast thread locals are important and a Symbol lookup > in an IdentityDictionary is pretty fast. And polluting Object with stuff > that doesn't belong there isn't a very convincing alternative to me. > does that means that you prefer to replace suspendedContext? > Cheers > PHilippe > > > -- Best regards, Igor Stasenko AKA sig.

Nov. 9, 2010

[Pharo-project] #& in Socket >> #waitForSendDoneFor:

by Philippe Marschall

Hi I have a case where the #& in Socket >> #waitForSendDoneFor: shows up in a tally (see attachment). I'm able to push about 1 Mbyte/s more when I replace that with #and:. So my question is this really needed or should I file a bug and send a patch? This is Pharo 1.1.1 BTW. Cheers Philippe

Nov. 9, 2010

Re: [Pharo-project] Process-specific state broken and uncomplete

by Philippe Marschall

On 08.11.2010 06:39, Igor Stasenko wrote: > Hello, > > i just found that Squeak images Process class contains unused ivars: > island env > > in Pharo image, env ivar is used for holding a process-specific state, > which is convenient, so one could use: > Processor activeProcess environmentAt: put:... > > but in Squeak image there is even no such methods. > > There are also some caveats with such thing: > during process termination, Processor activeProcess could point to > different process, > and so, accessing to process environment may lead to error(s). > > To avoid issues like these, the process termination action (stack > unwinding) should be always performed only for active process. > One of a trick i think, is to replace the suspendedContext with own > context, and then activate given process in order to unwind its stack > etc. > > Another way is to remember environment somewhere else during process > termination, and use different method to access it i.e. instead of: > Processor activeProcess environmentAt: > use > self processEnvironmentAt: > > where #processEnvironmentAt: is implemented in Object class, and can > handle termination gracefully. > > But i think replacing suspendedContext is more generic. > What you think? Not so convinced. Fast thread locals are important and a Symbol lookup in an IdentityDictionary is pretty fast. And polluting Object with stuff that doesn't belong there isn't a very convincing alternative to me. Cheers PHilippe

Nov. 9, 2010

Re: [Pharo-project] [squeak-dev] Re: Process-specific state broken and uncomplete

by Igor Stasenko

On 9 November 2010 06:42, Levente Uzonyi <leves(a)elte.hu> wrote: > On Tue, 9 Nov 2010, Igor Stasenko wrote: > > snip > >> >> No, you can't. I feel sorry for you :) >> >> A #useSession:during: using ensure, and of course it will work >> correctly, because you not accessing session via >> process-specific storage. This is illustrated by my "good fork" example. > > Erm, no. In both of your examples the session wasn't aquired by the forked > process. It was available via a variable. If you aquire the session _in_ the > forked process _from_ the session store, then the #ensure: block in the > session store will be able to retrieve it. > this doesn't changes anything. The point is that once you accessing _any_ process-specific state in ensure block, you're in trouble. And of course, if you capturing such state before entering ensure block, you are safe. But this not always an option. >> >> now replace #doSomething with this: >> >> [ 1 year asDelay wait ] ensure: [ >> Â self assert: Â (Processor activeProcess environmentAt: #session) notNil ] >> >> and then terminate a process which running your fork. > > Replaced (in theory) and terminated. What will happen? This assertion in > this #ensure: block will fail, but the session will be retrieved properly by > the session store's unwind block. > > So you can use process local variables, but you shouldn't access them during > unwinding (#ensure:, #ifCurtailed:, whatever). This is possible as I showed > it in my example. > i didn't said this is not possible. >> >> In attachment you'll find a test, which shows the problem (i using >> tests from Pharo image). > > The test fails, because you're accessing a process local variable in an > unwind block, which you shouldn't do. > Why i shouldn't? Give me the reasons why. ("because it works that way" is not a good answer ;) It sounds like you feel completely happy with current status quo, and don't want to address that problem. I don't know how else to convince you that this is wrong. Suppose you wrote a library, which using a process-local state, buried somewhere deeply. Users of your library might even be not aware of it, because they will be using a higher-level api. A developer who writes own code, sure thing, thinks that he is free to use #ensure: #ifCurtailed: whereever he needs to, isnt? But with stuff like that, you putting him in the position, that he should be aware of such subtle nuances, and be very careful about using your api inside ensure blocks. This is really beyond my understanding. Why we should not provide a consistent API instead? > > Levente > -- Best regards, Igor Stasenko AKA sig.

Nov. 9, 2010

Re: [Pharo-project] [squeak-dev] Re: Process-specific state broken and uncomplete

by Levente Uzonyi

On Tue, 9 Nov 2010, Igor Stasenko wrote: snip > > No, you can't. I feel sorry for you :) > > A #useSession:during: using ensure, and of course it will work > correctly, because you not accessing session via > process-specific storage. This is illustrated by my "good fork" example. Erm, no. In both of your examples the session wasn't aquired by the forked process. It was available via a variable. If you aquire the session _in_ the forked process _from_ the session store, then the #ensure: block in the session store will be able to retrieve it. > > now replace #doSomething with this: > > [ 1 year asDelay wait ] ensure: [ > self assert: (Processor activeProcess environmentAt: #session) notNil ] > > and then terminate a process which running your fork. Replaced (in theory) and terminated. What will happen? This assertion in this #ensure: block will fail, but the session will be retrieved properly by the session store's unwind block. So you can use process local variables, but you shouldn't access them during unwinding (#ensure:, #ifCurtailed:, whatever). This is possible as I showed it in my example. > > In attachment you'll find a test, which shows the problem (i using > tests from Pharo image). The test fails, because you're accessing a process local variable in an unwind block, which you shouldn't do. Levente > > >> >> Levente >> >> >> snip >> >> > > > > -- > Best regards, > Igor Stasenko AKA sig. >

Nov. 9, 2010

Re: [Pharo-project] Howto measure memory footprints of functions/objects/libraries in Pharo?

by Levente Uzonyi

On Tue, 9 Nov 2010, Guido Stepken wrote: > E.g.: Three different solutions for a Euler problem: > > (1 to: 999) inject: 0 into: [:sum :i | (i \\ 3 = 0) | (i \\ 5 = 0) > ifTrue: [sum + i] > ifFalse: [sum]] > > > (1 to: 999) detectSum: [:i | (i \\ 3 = 0) | (i \\ 5 = 0) ifTrue: [i] ifFalse: > [0]] > > (1 to: 999) detectSum: [:i | i * (#(0 0 1 0 1 1 0 0 1 1 0 1 0 0 1) atWrap: > i)] > > How do i measure the exact memory footprint of each solution? I think you could measure it with the simulator. But these snippets are so simple that they only use at most few kilobytes. > > Pharo claims having reduced memory footprint at many places. Can that be > proofed, e.g. with one comparable demo app, once run in old squeak, once in > Pharo 1.1/1.2? You misunderstood Pharo's claims. It doesn't mean that running code in Pharo will use less memory than in Squeak. It means that the image is smaller, because it contains less stuff. So, there's nothing to be proven. Btw it's funny that you refer to Squeak as "old". > > And no, MessageTally spyOn: does not do right! MessageTally is for measuring runtime, not space. But I guess the GC statistics are reliable, so the allocated memory and time spent in GC is correct even on CogVM. But your code runs too fast for MessageTally. Levente > > Tnx in advance, Guido Stepken > > >

Nov. 9, 2010

Re: [Pharo-project] [squeak-dev] Re: Process-specific state broken and uncomplete

by Igor Stasenko

On 9 November 2010 05:59, Levente Uzonyi <leves(a)elte.hu> wrote: > On Tue, 9 Nov 2010, Igor Stasenko wrote: > > snip > >> Suppose you have a domain object which provides following ways to >> access the session: >> >> session: forkedSession >> Â Processor activeProcess environmentAt: #session put: forkedSession >> >> session >> Â ^ Processor activeProcess environmentAt: #session >> >> >> now, here what i'd like to write: >> >> | session | >> >> session := MySession new . >> session initialize blabla. >> >> "good fork" >> good := [ >> Â self session: session. >> Â [ self process stuff ] ensure: [ session close ] >> ] newProcess. >> >> good resume. >> 1 minute asDelay wait. >> good terminate. >> self assert: (session isClosed). >> >> In the above case, everything is ok: session object is passed in 'session' >> temp >> and visible inside ensure block directly, so, no matter what may >> happen (process termination), >> it is guaranteed that session will be closed at the end. >> >> "bad fork" >> >> session := MySession new . >> session initialize blabla. >> >> bad := [ >> Â self session: session. >> Â [ self process stuff ] ensure: [ self session close ] >> ] newProcess. >> >> bad resume. >> 1 minute asDelay wait. >> bad terminate. >> self assert: (session isClosed). >> >> >> In this case, a session is accessed indirectly, through process specific >> state. >> And test might fail or not, depending if process was already >> terminated before sending #terminate, >> or its still running. >> >> The 'bad fork' illustrates that one should not attempt to access a >> process specific state in ensure blocks, >> because during process termination, stack unwinding is performed by >> other process, which could have different session object, >> or don't have it at all. >> >> This is simple, of course, to implement session finalization without >> using process spefic state. But now consider that session has not only >> #close method, but allows doing something more userful, like: >> >> self session backupStuff. >> [ Â .. do something.. ] ensure: [ self session restoreStuff ]. >> >> And again, same problem: unless you have a direct reference to session >> object, you can't do it. >> Not with using process-specific state, because it is flawed. >> >> In other words, anything which may want to access to a >> process-specific state, like: >> >> [ ... ... ] ensure: [ (Processor activeProcess environmentAt: #foo) >> doSomething ] >> >> have no guarantees to work correctly in all cases. >> >> This defeats a purpose of process-specific state, because you can't >> rely on it for building a reliable stuff. > > Not really. You can store the sessions in a session store which is > responsible for retrieving the session: > > [ > Â Â Â Â sessionStore useSession: sessionKey during: [ :session | > Â Â Â Â Â Â Â Â Processor activeProcess environmentAt: #session put: session. > Â Â Â Â Â Â Â Â self doSomething. > Â Â Â Â Â Â Â Â Processor activeProcess environmentAt: #session put: nil ] ] > fork. > > SessionStore >> useSession: sessionKey during: aBlock > > Â Â Â Â | session | > Â Â Â Â [ > Â Â Â Â Â Â Â Â session := self sessionFor: sessionKey. > Â Â Â Â Â Â Â Â aBlock value: session ] > Â Â Â Â Â Â Â Â ensure: [ > Â Â Â Â Â Â Â Â Â Â Â Â session ifNotNil: [ > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â self releaseSession: session ] ] > > In #doSomething you can access the session via the process. We're using this > pattern in PostgresV3's connection pool. It works pretty well. > No, you can't. I feel sorry for you :) A #useSession:during: using ensure, and of course it will work correctly, because you not accessing session via process-specific storage. This is illustrated by my "good fork" example. now replace #doSomething with this: [ 1 year asDelay wait ] ensure: [ self assert: (Processor activeProcess environmentAt: #session) notNil ] and then terminate a process which running your fork. In attachment you'll find a test, which shows the problem (i using tests from Pharo image). > > Levente > > > snip > > -- Best regards, Igor Stasenko AKA sig.

Nov. 9, 2010

Re: [Pharo-project] [squeak-dev] Re: Process-specific state broken and uncomplete

by Levente Uzonyi

On Tue, 9 Nov 2010, Igor Stasenko wrote: > On 9 November 2010 03:31, Levente Uzonyi <leves(a)elte.hu> wrote: snip >> >> So they integrated the full ProcessSpecific package. Is it a problem that >> you have to load it into Squeak? >> >> > So, its a separate package, except that ivars already there.. Yes, it's on SqueakSource. > Strange choice. Maybe it worth mentioning then in Process comment that > these vars are there for purpose? > Otherwise someone may attempt to remove them :) Good idea. Levente snip

Nov. 9, 2010

Re: [Pharo-project] [squeak-dev] Re: Process-specific state broken and uncomplete

by Igor Stasenko

On 9 November 2010 03:31, Levente Uzonyi <leves(a)elte.hu> wrote: > On Mon, 8 Nov 2010, Igor Stasenko wrote: > >> Hello, >> >> i just found that Squeak images Process class contains unused ivars: >> island env > > IIRC those were added by Matthew Fulmer to support ProcessSpecific _and_ > Cobalt. The idea is that the two can be used together this way, because the > definition of Process doesn't have to be changed. > Sure. >> >> in Pharo image, env ivar is used for holding a process-specific state, >> which is convenient, so one could use: >> Processor activeProcess environmentAt: put:... >> >> but in Squeak image there is even no such methods. > > So they integrated the full ProcessSpecific package. Is it a problem that > you have to load it into Squeak? > > So, its a separate package, except that ivars already there.. Strange choice. Maybe it worth mentioning then in Process comment that these vars are there for purpose? Otherwise someone may attempt to remove them :) > Levente > >> >> There are also some caveats with such thing: >> during process termination, Processor activeProcess could point to >> different process, >> and so, accessing to process environment may lead to error(s). >> >> To avoid issues like these, the process termination action (stack >> unwinding) should be always performed only for active process. >> One of a trick i think, is to replace the suspendedContext with own >> context, and then activate given process in order to unwind its stack >> etc. >> >> Another way is to remember environment somewhere else during process >> termination, and use different method to access it i.e. instead of: >> Processor activeProcess environmentAt: >> use >> self processEnvironmentAt: >> >> where #processEnvironmentAt: is implemented in Object class, and can >> handle termination gracefully. >> >> But i think replacing suspendedContext is more generic. >> What you think? >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> >> > > -- Best regards, Igor Stasenko AKA sig.

Nov. 9, 2010