- Pharo-dev - lists.pharo.org

Re: [Pharo-project] Xtreams port to Squeak - second wave

by Levente Uzonyi

On Sun, 10 Oct 2010, Nicolas Cellier wrote: > Hi again, > I now have ported two more packages > - Xtreams-Transforms > - Xtreams-Substreams > and their tests > > I did not have any portability problem with those... > But that's because I did not handle the Character encoder/decoder. > Consequently, I have 8 tests failing (the Base64 related tests) > Plus 4 other tests failing because of my poor implementation of > #after:do: (forking processes in a SUnit TestCase can't be that > obvious). I added Number >> #milliseconds to Xtreams-VWCompatible, because Squeak has #milliSeconds for some reason and the tests use #milliseconds I also uploaded a new version of Xtreams-CoreTests which doesn't use #after:do:. This change may use Squeak specific stuff, revert it if you don't like it. All tests (except for the base64 related) are passing reliably, no more random failures. Levente > > Now, the easy part of the port (copy/paste) is almost ended. > Once we manage a compatible way to handle pragmas, PEG Parser should > port quite easily too. > > Then, the harder work begins: > - File/Socket/Pipe > - Pointers (in External Heap) > - Character encoding/decoding > - Compression/Decompression > > If you think you can help in any of these, please tell. > > Nicolas > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by David T. Lewis

On Sun, Oct 10, 2010 at 08:16:57PM -0400, Schwab,Wilhelm K wrote: > <snip> > The creation of the redundant executors is a recipe for for a double free and resulting crash. <snip> > Am I missing something? There is no such thing as a "double free and resulting crash". Dave

Oct. 11, 2010

[Pharo-project] Fwd: [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Igor Stasenko

2010/10/11 Levente Uzonyi <leves(a)elte.hu>: > On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> 2010/10/11 Levente Uzonyi <leves(a)elte.hu>: >> On Mon, 11 Oct 2010, Igor Stasenko wrote: >> >>> On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: >>> On Sun, 10 Oct 2010, Igor Stasenko wrote: >>> >>>> A current implementation of this method >>>> >>>> remove: oldObject ifAbsent: exceptionBlock >>>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>>> >>>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>>> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject >>>> ifAbsent: >>>> nil ]) >>>> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >>>> >>>> simply removes a previously registered object from registry and voila. >>>> >>>> Now lets get back to our discussion about multiple finalizers per >>>> object and using them in weak subscriptions etc. >>>> >>>> Suppose i am added a socket to weak registry, >>>> and suppose i am added a weak subscription to it. >>>> >>>> Now, if i do 'socket close' , it tells weak registry to remove it from >>>> list. >>>> And what we'll have: >>>> - socket handle is closed >>>> - socket is wiped from weak registry >>>> - but weak subscription still listed somewhere in a list of >>>> subscriptions >>>> >>>> >>>> My suggestion is, that upon #remove:, >>>> a weak registry should notify all executors that object of interest >>>> are no longer takes part in finalization scheme, >>>> so they should not count on receiving #finalize eventually. >>>> >>>> In other words: >>>> >>>> remove: oldObject ifAbsent: exceptionBlock >>>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>>> >>>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>>> Â Â Â Â ^(self protected: [ | executor | >>>> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: >>>> nil. >>>> Â Â Â Â Â Â executor discardFinalization. >>>> Â Â Â Â ]) >>>> Â Â Â Â ifNil: [ exceptionBlock value ] >>> >>> It's only an issue with the new WeakRegistry implementation, previous >>> implementations don't have such problem. I think changing the method as >>> you >>> suggested, implementing WeakFinalizerItem >> #discardFinalization as >>> "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to >>> ignore the executor if it's nil will fix the problem. Am I right? >>> >>> I don't get how "multiple finalizers per object" is related to this >>> problem >>> at all. >>> >> >> No, you miss the point. >> When you removing object from weak registry, it is important , time to >> time to tell all of its executors, >> that they will no longer receive #finalize, because object is no >> longer a member of weak registry. >> >> If you simply set executor := nil, it does nothing, an executor itself >> did not notified that he won't be needed >> in any possible future. >> So, if your finalization action is to remove some object(s) from the >> list , you'll get your list dirty after object will die, >> because #finalize will be never sent to your executor. >> >> Here the simple test case: >> >> | coll obj someWrapper | >> coll := OrderedCollection new. >> obj := Object new. >> someWrapper := WeakArray with: obj. >> coll add: someWrapper. >> >> obj toFinalizeSend: #remove: to: coll with: someWrapper. >> obj finalizationRegistry remove: obj. >> obj := nil. >> Smalltalk garbageCollect. >> self assert: coll isEmpty >> >> >> the point is, that once you doing a #remove: , >> your finalization scheme is broken. >> >> >> I don't get you. When you remove an object from a WeakRegistry, you tell >> the >> system that you want the object removed and all related executors deleted. >> So the executors should never receive #finalize, they should be thrown >> away. >> Even if there's action to be done with the executors (which is pointless >> IMHO since those will be thrown away, but who knows), doing them is the >> responsibility of the sender of #remove:. >> > > Let us get back to multiple finalizers per object, > when you have two separate places, which adding possibly different > executors for a single object > without knowing about each other (otherwise why would you want to add > two finalizers instead of one). > Now if one decides to do #remove: , > how to ensure that second one won't be left with dirty/invalid state? > > > It's not ensured, and it doesn't have to be. The sender of #remove: takes > all responsibility. It can decide to add some executors back to the registry > if that's necessary. > If you want to keep your executors safe from some other code, which may > #remove: your objects from a WeakRegistry, then you can create a private > WeakRegistry and store your executors there, just like Sockets and > FileStreams do. > Conclusion: weak registry don't have to support multiple finalizers per single object. Instead, if one sees the need to control a finalization of certain object(s) in own way, indepentently fron any other, then he should use a private weak registry (like Sockets and FileStreams do), and register object and its executor there. -- Best regards, Igor Stasenko AKA sig.

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Levente Uzonyi

On Mon, 11 Oct 2010, Igor Stasenko wrote: > 2010/10/11 Levente Uzonyi <leves(a)elte.hu>: > On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: >> On Sun, 10 Oct 2010, Igor Stasenko wrote: >> >>> A current implementation of this method >>> >>> remove: oldObject ifAbsent: exceptionBlock >>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>> >>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >>> nil ]) >>> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >>> >>> simply removes a previously registered object from registry and voila. >>> >>> Now lets get back to our discussion about multiple finalizers per >>> object and using them in weak subscriptions etc. >>> >>> Suppose i am added a socket to weak registry, >>> and suppose i am added a weak subscription to it. >>> >>> Now, if i do 'socket close' , it tells weak registry to remove it from >>> list. >>> And what we'll have: >>> - socket handle is closed >>> - socket is wiped from weak registry >>> - but weak subscription still listed somewhere in a list of subscriptions >>> >>> >>> My suggestion is, that upon #remove:, >>> a weak registry should notify all executors that object of interest >>> are no longer takes part in finalization scheme, >>> so they should not count on receiving #finalize eventually. >>> >>> In other words: >>> >>> remove: oldObject ifAbsent: exceptionBlock >>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>> >>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>> Â Â Â Â ^(self protected: [ | executor | >>> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: >>> nil. >>> Â Â Â Â Â Â executor discardFinalization. >>> Â Â Â Â ]) >>> Â Â Â Â ifNil: [ exceptionBlock value ] >> >> It's only an issue with the new WeakRegistry implementation, previous >> implementations don't have such problem. I think changing the method as >> you >> suggested, implementing WeakFinalizerItem >> #discardFinalization as >> "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to >> ignore the executor if it's nil will fix the problem. Am I right? >> >> I don't get how "multiple finalizers per object" is related to this >> problem >> at all. >> > > No, you miss the point. > When you removing object from weak registry, it is important , time to > time to tell all of its executors, > that they will no longer receive #finalize, because object is no > longer a member of weak registry. > > If you simply set executor := nil, it does nothing, an executor itself > did not notified that he won't be needed > in any possible future. > So, if your finalization action is to remove some object(s) from the > list , you'll get your list dirty after object will die, > because #finalize will be never sent to your executor. > > Here the simple test case: > > | coll obj someWrapper | > coll := OrderedCollection new. > obj := Object new. > someWrapper := WeakArray with: obj. > coll add: someWrapper. > > obj toFinalizeSend: #remove: to: coll with: someWrapper. > obj finalizationRegistry remove: obj. > obj := nil. > Smalltalk garbageCollect. > self assert: coll isEmpty > > > the point is, that once you doing a #remove: , > your finalization scheme is broken. > > > I don't get you. When you remove an object from a WeakRegistry, you tell the > system that you want the object removed and all related executors deleted. > So the executors should never receive #finalize, they should be thrown away. > Even if there's action to be done with the executors (which is pointless > IMHO since those will be thrown away, but who knows), doing them is the > responsibility of the sender of #remove:. > Let us get back to multiple finalizers per object, when you have two separate places, which adding possibly different executors for a single object without knowing about each other (otherwise why would you want to add two finalizers instead of one). Now if one decides to do #remove: , how to ensure that second one won't be left with dirty/invalid state? It's not ensured, and it doesn't have to be. The sender of #remove: takes all responsibility. It can decide to add some executors back to the registry if that's necessary. If you want to keep your executors safe from some other code, which may #remove: your objects from a WeakRegistry, then you can create a private WeakRegistry and store your executors there, just like Sockets and FileStreams do. Levente > > Levente > >> >> Levente >> > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Igor Stasenko

2010/10/11 Levente Uzonyi <leves(a)elte.hu>: > On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: >> On Sun, 10 Oct 2010, Igor Stasenko wrote: >> >>> A current implementation of this method >>> >>> remove: oldObject ifAbsent: exceptionBlock >>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>> >>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >>> nil ]) >>> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >>> >>> simply removes a previously registered object from registry and voila. >>> >>> Now lets get back to our discussion about multiple finalizers per >>> object and using them in weak subscriptions etc. >>> >>> Suppose i am added a socket to weak registry, >>> and suppose i am added a weak subscription to it. >>> >>> Now, if i do 'socket close' , it tells weak registry to remove it from >>> list. >>> And what we'll have: >>> - socket handle is closed >>> - socket is wiped from weak registry >>> - but weak subscription still listed somewhere in a list of subscriptions >>> >>> >>> My suggestion is, that upon #remove:, >>> a weak registry should notify all executors that object of interest >>> are no longer takes part in finalization scheme, >>> so they should not count on receiving #finalize eventually. >>> >>> In other words: >>> >>> remove: oldObject ifAbsent: exceptionBlock >>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>> >>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>> Â Â Â Â ^(self protected: [ | executor | >>> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: >>> nil. >>> Â Â Â Â Â Â executor discardFinalization. >>> Â Â Â Â ]) >>> Â Â Â Â ifNil: [ exceptionBlock value ] >> >> It's only an issue with the new WeakRegistry implementation, previous >> implementations don't have such problem. I think changing the method as >> you >> suggested, implementing WeakFinalizerItem >> #discardFinalization as >> "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to >> ignore the executor if it's nil will fix the problem. Am I right? >> >> I don't get how "multiple finalizers per object" is related to this >> problem >> at all. >> > > No, you miss the point. > When you removing object from weak registry, it is important , time to > time to tell all of its executors, > that they will no longer receive #finalize, because object is no > longer a member of weak registry. > > If you simply set executor := nil, it does nothing, an executor itself > did not notified that he won't be needed > in any possible future. > So, if your finalization action is to remove some object(s) from the > list , you'll get your list dirty after object will die, > because #finalize will be never sent to your executor. > > Here the simple test case: > > | coll obj someWrapper | > coll := OrderedCollection new. > obj := Object new. > someWrapper := WeakArray with: obj. > coll add: someWrapper. > > obj toFinalizeSend: #remove: to: coll with: someWrapper. > obj finalizationRegistry remove: obj. > obj := nil. > Smalltalk garbageCollect. > self assert: coll isEmpty > > > the point is, that once you doing a #remove: , > your finalization scheme is broken. > > > I don't get you. When you remove an object from a WeakRegistry, you tell the > system that you want the object removed and all related executors deleted. > So the executors should never receive #finalize, they should be thrown away. > Even if there's action to be done with the executors (which is pointless > IMHO since those will be thrown away, but who knows), doing them is the > responsibility of the sender of #remove:. > Let us get back to multiple finalizers per object, when you have two separate places, which adding possibly different executors for a single object without knowing about each other (otherwise why would you want to add two finalizers instead of one). Now if one decides to do #remove: , how to ensure that second one won't be left with dirty/invalid state? > > Levente > >> >> Levente >> > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > -- Best regards, Igor Stasenko AKA sig.

Oct. 11, 2010

Re: [Pharo-project] Xtreams : first embryonary port on Squeak

by mkobetic＠gmail.com

"Schwab,Wilhelm K"<bschwab(a)anest.ufl.edu> wrote: > FWIW, SIF might help you: > > http://www.pocketsmalltalk.com/sif/ > > I have put a lot of code through it in the past couple of years. I'd recommend giving the Monticello tools in VW a try before trying to put something together from lower level tools. It's used to keep Seaside in sync in VW, so it should handle most of the obvious obstacles. With the help of your Xtreams-VWCompatibility package on Squeak side and maybe some adjustments in the portable code and maybe moving some of the non-portable code into packages of their own, we should be able to make this largely automatic.

Oct. 11, 2010

Re: [Pharo-project] Xtreams port to Squeak - second wave

by mkobetic＠gmail.com

Hi Nicolas, "Nicolas Cellier"<nicolas.cellier.aka.nice(a)gmail.com> wrote: > Hi again, > I now have ported two more packages > - Xtreams-Transforms > - Xtreams-Substreams > and their tests This is great! I loaded your code into Pharo 1.1 and things seem to be working quite well. There was a complaint about missing SharedQueue2, I just created a dummy subclass of SharedQueue with that name and things seemed to load fine. XTRecyclingCenter seems to be subclass of XTWriteStream, it should be Object subclass, maybe a typo ? > I did not have any portability problem with those... > But that's because I did not handle the Character encoder/decoder. > Consequently, I have 8 tests failing (the Base64 related tests) I was thinking, we could implement just 'encoding: #ascii' quite easily, to make this reasonably usable at least for applications that are fine with that. We're actually contemplating about implementing our own encoders for Xtreams too. The VW ones are tied to the classic streams more than we like. You might have noticed some rather hairy parts in the encoding streams yourself, where we're trying to work around some of the issues it creates. The advantage of reusing the existing encoders was that there are quite a few of those available, so reimplementing all that would be a drag. But we can come up with a scheme where we can reimplement at least the common ones and in VW we can still preserve hooking into the old ones for the rest. I can give that a try on VW side in the meantime so you could get those for free. > Plus 4 other tests failing because of my poor implementation of > #after:do: (forking processes in a SUnit TestCase can't be that > obvious). I looked at this, and I think this is how #after:do: should look like: after: aDelay do: aBlock "Evaluate the argument block delayed after the specified duration." | watchdog | watchdog := [ aDelay wait. aBlock value. ] newProcess. watchdog priority: Processor userInterruptPriority. watchdog resume. This would assume that the 2 tests calling #timeout:server:client: would use a Delay instance instead of a Duration, which I'd be fine with. However making that change doesn't quite get the tests running. It's blowing up with a DNU on the 'output close' bit in #terminate:server:client: with output being nil, which I'm having trouble figuring out. I can't find who could possibly be nilling it out. I'm somewhat struggling finding my way around the Pharo tools. The test seems to otherwise pass but the DNUs from the background process isn't nice. Also when I just click on the test in the TestRunner, I actually get four DNUs, not just one as I would expect. So I'm kinda stuck, not sure how to move forward without help from someone who knows his way around Pharo. I also get odd failure from #testWriteCollectingMultipleBufferSize, which seems to run fine (against collection) when I run the equivalent in a workspace, but strangely fails when running via the #timeout:server:client: construct, i.e. when client and server run in separate processes. Hm, now that I think of it, they sure could fail if something preempts the client, server processes at the right moment. I'll have to rethink that again. > Now, the easy part of the port (copy/paste) is almost ended. > Once we manage a compatible way to handle pragmas, PEG Parser should > port quite easily too. I wouldn't worry about the Parser stuff at this point. > Then, the harder work begins: > - File/Socket/Pipe > - Pointers (in External Heap) I wouldn't worry, about the external heap stuff either. It's neat, but probably not something many people will miss. > - Character encoding/decoding I'll see if I can help with this from the VW side. > - Compression/Decompression Is Zlib linked into the VM in Squeak too ? The compression streams are written directly against the ZLib API, so there aren't any VW specific dependencies other than how those calls are made. Similarly the crypto streams go directly against the EVP API in LibCrypto in OpenSSL, so as long as we can abstract over how those are called, the stream implementation should work as is. > If you think you can help in any of these, please tell. If you could compile a list of changes that you'd like us to adopt on the VW side, I'd certainly look at that. I did read your posts but it's not entirely clear what you'd like to handle on Squeak side and what on VW side. A fileout would be best to avoid any confusion, but a description is fine too. I'm also unclear about the become: discussion. Don't write streams in Squeak become: the underlying collection when they grow it ? Cheers, Martin

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Schwab,Wilhelm K

Sig, Dumb question: how much does an executor need to know about this? It can't strongly bind the outgoing object or weaklings won't let go and finalization will never happen. Whether the guidance to it is in the form of "clean up" or "skip it," it needs to come from the weak registry, finalization framework, or whatever you want to call it. Dolphin handles finalization with support from the VM, allowing the outgoing object to do the finalization itself, often in the form of releasing external resources, so there is no concept of executor, let alone multiple executors. I have been arguing in favor of thread safety for these mechanisms, and I suspect this is another example of why it is needed. Why would one want more than one object ready to destroy an external handle when a single object disappears? The creation of the redundant executors is a recipe for for a double free and resulting crash. I think we should prevent such redundancy, and would be readily able to do so if the registries were thread safe giving #at:ifAbsentPut: the chops to do the job. Am I missing something? Bill ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua(a)gmail.com] Sent: Sunday, October 10, 2010 7:10 PM To: The general-purpose Squeak developers list Cc: Pharo Development Subject: Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: > On Sun, 10 Oct 2010, Igor Stasenko wrote: > >> A current implementation of this method >> >> remove: oldObject ifAbsent: exceptionBlock >> "Remove oldObject as one of the receiver's elements." >> >> oldObject ifNil: [ ^nil ]. >> ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >> nil ]) >> ifNil: [ exceptionBlock value ] >> >> simply removes a previously registered object from registry and voila. >> >> Now lets get back to our discussion about multiple finalizers per >> object and using them in weak subscriptions etc. >> >> Suppose i am added a socket to weak registry, >> and suppose i am added a weak subscription to it. >> >> Now, if i do 'socket close' , it tells weak registry to remove it from >> list. >> And what we'll have: >> - socket handle is closed >> - socket is wiped from weak registry >> - but weak subscription still listed somewhere in a list of subscriptions >> >> >> My suggestion is, that upon #remove:, >> a weak registry should notify all executors that object of interest >> are no longer takes part in finalization scheme, >> so they should not count on receiving #finalize eventually. >> >> In other words: >> >> remove: oldObject ifAbsent: exceptionBlock >> "Remove oldObject as one of the receiver's elements." >> >> oldObject ifNil: [ ^nil ]. >> ^(self protected: [ | executor | >> executor := valueDictionary removeKey: oldObject ifAbsent: nil. >> executor discardFinalization. >> ]) >> ifNil: [ exceptionBlock value ] > > It's only an issue with the new WeakRegistry implementation, previous > implementations don't have such problem. I think changing the method as you > suggested, implementing WeakFinalizerItem >> #discardFinalization as > "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to > ignore the executor if it's nil will fix the problem. Am I right? > > I don't get how "multiple finalizers per object" is related to this problem > at all. > No, you miss the point. When you removing object from weak registry, it is important , time to time to tell all of its executors, that they will no longer receive #finalize, because object is no longer a member of weak registry. If you simply set executor := nil, it does nothing, an executor itself did not notified that he won't be needed in any possible future. So, if your finalization action is to remove some object(s) from the list , you'll get your list dirty after object will die, because #finalize will be never sent to your executor. Here the simple test case: | coll obj someWrapper | coll := OrderedCollection new. obj := Object new. someWrapper := WeakArray with: obj. coll add: someWrapper. obj toFinalizeSend: #remove: to: coll with: someWrapper. obj finalizationRegistry remove: obj. obj := nil. Smalltalk garbageCollect. self assert: coll isEmpty the point is, that once you doing a #remove: , your finalization scheme is broken. > > Levente > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Levente Uzonyi

On Mon, 11 Oct 2010, Igor Stasenko wrote: > On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: > On Sun, 10 Oct 2010, Igor Stasenko wrote: > >> A current implementation of this method >> >> remove: oldObject ifAbsent: exceptionBlock >> Â Â Â Â "Remove oldObject as one of the receiver's elements." >> >> Â Â Â Â oldObject ifNil: [ ^nil ]. >> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >> nil ]) >> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >> >> simply removes a previously registered object from registry and voila. >> >> Now lets get back to our discussion about multiple finalizers per >> object and using them in weak subscriptions etc. >> >> Suppose i am added a socket to weak registry, >> and suppose i am added a weak subscription to it. >> >> Now, if i do 'socket close' , it tells weak registry to remove it from >> list. >> And what we'll have: >> - socket handle is closed >> - socket is wiped from weak registry >> - but weak subscription still listed somewhere in a list of subscriptions >> >> >> My suggestion is, that upon #remove:, >> a weak registry should notify all executors that object of interest >> are no longer takes part in finalization scheme, >> so they should not count on receiving #finalize eventually. >> >> In other words: >> >> remove: oldObject ifAbsent: exceptionBlock >> Â Â Â Â "Remove oldObject as one of the receiver's elements." >> >> Â Â Â Â oldObject ifNil: [ ^nil ]. >> Â Â Â Â ^(self protected: [ | executor | >> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: nil. >> Â Â Â Â Â Â executor discardFinalization. >> Â Â Â Â ]) >> Â Â Â Â ifNil: [ exceptionBlock value ] > > It's only an issue with the new WeakRegistry implementation, previous > implementations don't have such problem. I think changing the method as you > suggested, implementing WeakFinalizerItem >> #discardFinalization as > "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to > ignore the executor if it's nil will fix the problem. Am I right? > > I don't get how "multiple finalizers per object" is related to this problem > at all. > No, you miss the point. When you removing object from weak registry, it is important , time to time to tell all of its executors, that they will no longer receive #finalize, because object is no longer a member of weak registry. If you simply set executor := nil, it does nothing, an executor itself did not notified that he won't be needed in any possible future. So, if your finalization action is to remove some object(s) from the list , you'll get your list dirty after object will die, because #finalize will be never sent to your executor. Here the simple test case: | coll obj someWrapper | coll := OrderedCollection new. obj := Object new. someWrapper := WeakArray with: obj. coll add: someWrapper. obj toFinalizeSend: #remove: to: coll with: someWrapper. obj finalizationRegistry remove: obj. obj := nil. Smalltalk garbageCollect. self assert: coll isEmpty the point is, that once you doing a #remove: , your finalization scheme is broken. I don't get you. When you remove an object from a WeakRegistry, you tell the system that you want the object removed and all related executors deleted. So the executors should never receive #finalize, they should be thrown away. Even if there's action to be done with the executors (which is pointless IMHO since those will be thrown away, but who knows), doing them is the responsibility of the sender of #remove:. Levente > > Levente > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 10, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Igor Stasenko

On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: > On Sun, 10 Oct 2010, Igor Stasenko wrote: > >> A current implementation of this method >> >> remove: oldObject ifAbsent: exceptionBlock >> Â Â Â Â "Remove oldObject as one of the receiver's elements." >> >> Â Â Â Â oldObject ifNil: [ ^nil ]. >> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >> nil ]) >> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >> >> simply removes a previously registered object from registry and voila. >> >> Now lets get back to our discussion about multiple finalizers per >> object and using them in weak subscriptions etc. >> >> Suppose i am added a socket to weak registry, >> and suppose i am added a weak subscription to it. >> >> Now, if i do 'socket close' , it tells weak registry to remove it from >> list. >> And what we'll have: >> - socket handle is closed >> - socket is wiped from weak registry >> - but weak subscription still listed somewhere in a list of subscriptions >> >> >> My suggestion is, that upon #remove:, >> a weak registry should notify all executors that object of interest >> are no longer takes part in finalization scheme, >> so they should not count on receiving #finalize eventually. >> >> In other words: >> >> remove: oldObject ifAbsent: exceptionBlock >> Â Â Â Â "Remove oldObject as one of the receiver's elements." >> >> Â Â Â Â oldObject ifNil: [ ^nil ]. >> Â Â Â Â ^(self protected: [ | executor | >> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: nil. >> Â Â Â Â Â Â executor discardFinalization. >> Â Â Â Â ]) >> Â Â Â Â ifNil: [ exceptionBlock value ] > > It's only an issue with the new WeakRegistry implementation, previous > implementations don't have such problem. I think changing the method as you > suggested, implementing WeakFinalizerItem >> #discardFinalization as > "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to > ignore the executor if it's nil will fix the problem. Am I right? > > I don't get how "multiple finalizers per object" is related to this problem > at all. > No, you miss the point. When you removing object from weak registry, it is important , time to time to tell all of its executors, that they will no longer receive #finalize, because object is no longer a member of weak registry. If you simply set executor := nil, it does nothing, an executor itself did not notified that he won't be needed in any possible future. So, if your finalization action is to remove some object(s) from the list , you'll get your list dirty after object will die, because #finalize will be never sent to your executor. Here the simple test case: | coll obj someWrapper | coll := OrderedCollection new. obj := Object new. someWrapper := WeakArray with: obj. coll add: someWrapper. obj toFinalizeSend: #remove: to: coll with: someWrapper. obj finalizationRegistry remove: obj. obj := nil. Smalltalk garbageCollect. self assert: coll isEmpty the point is, that once you doing a #remove: , your finalization scheme is broken. > > Levente > -- Best regards, Igor Stasenko AKA sig.

Oct. 10, 2010