October 2010 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] [vwnc] Xtreams port to Squeak - second wave

by Michael Lucas-Smith

On Oct 10, 2010, at 5:18 PM, mkobetic(a)gmail.com wrote: > Hi Nicolas, > > "Nicolas Cellier"<nicolas.cellier.aka.nice(a)gmail.com> wrote: >> Hi again, >> I now have ported two more packages >> - Xtreams-Transforms >> - Xtreams-Substreams >> and their tests If you get Interpreting and Marshaling going we'll have ST-ST communication between Pharo<->Squeak<->VisualWorks with a fast binary protocol. I suspect some trickery will be required to deal with namespaces, as the protocol assumes 'full names' for classes when matching between the ends. Michael

Oct. 11, 2010

Re: [Pharo-project] Xtreams port to Squeak - second wave

by Levente Uzonyi

On Sun, 10 Oct 2010, mkobetic(a)gmail.com wrote: > Hi Nicolas, > > "Nicolas Cellier"<nicolas.cellier.aka.nice(a)gmail.com> wrote: >> Hi again, >> I now have ported two more packages >> - Xtreams-Transforms >> - Xtreams-Substreams >> and their tests > > This is great! I loaded your code into Pharo 1.1 and things seem to be working quite well. There was a complaint about missing SharedQueue2, I just created a dummy subclass of SharedQueue with that name and things seemed to load fine. > XTRecyclingCenter seems to be subclass of XTWriteStream, it should be Object subclass, maybe a typo ? > >> I did not have any portability problem with those... >> But that's because I did not handle the Character encoder/decoder. >> Consequently, I have 8 tests failing (the Base64 related tests) > > I was thinking, we could implement just 'encoding: #ascii' quite easily, to make this reasonably usable at least for applications that are fine with that. We're actually contemplating about implementing our own encoders for Xtreams too. The VW ones are tied to the classic streams more than we like. You might have noticed some rather hairy parts in the encoding streams yourself, where we're trying to work around some of the issues it creates. The advantage of reusing the existing encoders was that there are quite a few of those available, so reimplementing all that would be a drag. But we can come up with a scheme where we can reimplement at least the common ones and in VW we can still preserve hooking into the old ones for the rest. I can give that a try on VW side in the meantime so you could get those for free. > >> Plus 4 other tests failing because of my poor implementation of >> #after:do: (forking processes in a SUnit TestCase can't be that >> obvious). > > I looked at this, and I think this is how #after:do: should look like: > > after: aDelay do: aBlock > "Evaluate the argument block delayed after the specified duration." > > | watchdog | > watchdog := [ > aDelay wait. > aBlock value. > ] newProcess. > watchdog priority: Processor userInterruptPriority. > watchdog resume. > > This would assume that the 2 tests calling #timeout:server:client: would use a Delay instance instead of a Duration, which I'd be fine with. However making that change doesn't quite get the tests running. It's blowing up with a DNU on the 'output close' bit in #terminate:server:client: with output being nil, which I'm having trouble figuring out. I can't find who could possibly be nilling it out. I'm somewhat struggling finding my way around the Pharo tools. The test seems to otherwise pass but the DNUs from the background process isn't nice. Also when I just click on the test in the TestRunner, I actually get four DNUs, not just one as I would expect. So I'm kinda stuck, not sure how to move forward without help from someone who knows his way around Pharo. > I also get odd failure from #testWriteCollectingMultipleBufferSize, which seems to run fine (against collection) when I run the equivalent in a workspace, but strangely fails when running via the #timeout:server:client: construct, i.e. when client and server run in separate processes. Hm, now that I think of it, they sure could fail if something preempts the client, server processes at the right moment. I'll have to rethink that again. These problems should be solved with the latest version of CoreTests. > >> Now, the easy part of the port (copy/paste) is almost ended. >> Once we manage a compatible way to handle pragmas, PEG Parser should >> port quite easily too. > > I wouldn't worry about the Parser stuff at this point. > >> Then, the harder work begins: >> - File/Socket/Pipe >> - Pointers (in External Heap) > > I wouldn't worry, about the external heap stuff either. It's neat, but probably not something many people will miss. > >> - Character encoding/decoding > > I'll see if I can help with this from the VW side. > >> - Compression/Decompression > > Is Zlib linked into the VM in Squeak too ? The compression streams are written directly against the ZLib API, so there aren't any VW specific dependencies other than how those calls are made. Similarly the crypto streams go directly against the EVP API in LibCrypto in OpenSSL, so as long as we can abstract over how those are called, the stream implementation should work as is. > >> If you think you can help in any of these, please tell. > > If you could compile a list of changes that you'd like us to adopt on the VW side, I'd certainly look at that. I did read your posts but it's not entirely clear what you'd like to handle on Squeak side and what on VW side. A fileout would be best to avoid any confusion, but a description is fine too. > > I'm also unclear about the become: discussion. Don't write streams in Squeak become: the underlying collection when they grow it ? No. Squeak uses direct pointers, so it doesn't have an object table, therefore #become: is very expensive. See the Storage Management section of http://ftp.squeak.org/docs/OOPSLA.Squeak.html for details. Levente > > Cheers, > > Martin > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 11, 2010

Re: [Pharo-project] Xtreams port to Squeak - second wave

by Levente Uzonyi

On Sun, 10 Oct 2010, Nicolas Cellier wrote: > Hi again, > I now have ported two more packages > - Xtreams-Transforms > - Xtreams-Substreams > and their tests > > I did not have any portability problem with those... > But that's because I did not handle the Character encoder/decoder. > Consequently, I have 8 tests failing (the Base64 related tests) > Plus 4 other tests failing because of my poor implementation of > #after:do: (forking processes in a SUnit TestCase can't be that > obvious). I added Number >> #milliseconds to Xtreams-VWCompatible, because Squeak has #milliSeconds for some reason and the tests use #milliseconds I also uploaded a new version of Xtreams-CoreTests which doesn't use #after:do:. This change may use Squeak specific stuff, revert it if you don't like it. All tests (except for the base64 related) are passing reliably, no more random failures. Levente > > Now, the easy part of the port (copy/paste) is almost ended. > Once we manage a compatible way to handle pragmas, PEG Parser should > port quite easily too. > > Then, the harder work begins: > - File/Socket/Pipe > - Pointers (in External Heap) > - Character encoding/decoding > - Compression/Decompression > > If you think you can help in any of these, please tell. > > Nicolas > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by David T. Lewis

On Sun, Oct 10, 2010 at 08:16:57PM -0400, Schwab,Wilhelm K wrote: > <snip> > The creation of the redundant executors is a recipe for for a double free and resulting crash. <snip> > Am I missing something? There is no such thing as a "double free and resulting crash". Dave

Oct. 11, 2010

[Pharo-project] Fwd: [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Igor Stasenko

2010/10/11 Levente Uzonyi <leves(a)elte.hu>: > On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> 2010/10/11 Levente Uzonyi <leves(a)elte.hu>: >> On Mon, 11 Oct 2010, Igor Stasenko wrote: >> >>> On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: >>> On Sun, 10 Oct 2010, Igor Stasenko wrote: >>> >>>> A current implementation of this method >>>> >>>> remove: oldObject ifAbsent: exceptionBlock >>>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>>> >>>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>>> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject >>>> ifAbsent: >>>> nil ]) >>>> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >>>> >>>> simply removes a previously registered object from registry and voila. >>>> >>>> Now lets get back to our discussion about multiple finalizers per >>>> object and using them in weak subscriptions etc. >>>> >>>> Suppose i am added a socket to weak registry, >>>> and suppose i am added a weak subscription to it. >>>> >>>> Now, if i do 'socket close' , it tells weak registry to remove it from >>>> list. >>>> And what we'll have: >>>> - socket handle is closed >>>> - socket is wiped from weak registry >>>> - but weak subscription still listed somewhere in a list of >>>> subscriptions >>>> >>>> >>>> My suggestion is, that upon #remove:, >>>> a weak registry should notify all executors that object of interest >>>> are no longer takes part in finalization scheme, >>>> so they should not count on receiving #finalize eventually. >>>> >>>> In other words: >>>> >>>> remove: oldObject ifAbsent: exceptionBlock >>>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>>> >>>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>>> Â Â Â Â ^(self protected: [ | executor | >>>> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: >>>> nil. >>>> Â Â Â Â Â Â executor discardFinalization. >>>> Â Â Â Â ]) >>>> Â Â Â Â ifNil: [ exceptionBlock value ] >>> >>> It's only an issue with the new WeakRegistry implementation, previous >>> implementations don't have such problem. I think changing the method as >>> you >>> suggested, implementing WeakFinalizerItem >> #discardFinalization as >>> "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to >>> ignore the executor if it's nil will fix the problem. Am I right? >>> >>> I don't get how "multiple finalizers per object" is related to this >>> problem >>> at all. >>> >> >> No, you miss the point. >> When you removing object from weak registry, it is important , time to >> time to tell all of its executors, >> that they will no longer receive #finalize, because object is no >> longer a member of weak registry. >> >> If you simply set executor := nil, it does nothing, an executor itself >> did not notified that he won't be needed >> in any possible future. >> So, if your finalization action is to remove some object(s) from the >> list , you'll get your list dirty after object will die, >> because #finalize will be never sent to your executor. >> >> Here the simple test case: >> >> | coll obj someWrapper | >> coll := OrderedCollection new. >> obj := Object new. >> someWrapper := WeakArray with: obj. >> coll add: someWrapper. >> >> obj toFinalizeSend: #remove: to: coll with: someWrapper. >> obj finalizationRegistry remove: obj. >> obj := nil. >> Smalltalk garbageCollect. >> self assert: coll isEmpty >> >> >> the point is, that once you doing a #remove: , >> your finalization scheme is broken. >> >> >> I don't get you. When you remove an object from a WeakRegistry, you tell >> the >> system that you want the object removed and all related executors deleted. >> So the executors should never receive #finalize, they should be thrown >> away. >> Even if there's action to be done with the executors (which is pointless >> IMHO since those will be thrown away, but who knows), doing them is the >> responsibility of the sender of #remove:. >> > > Let us get back to multiple finalizers per object, > when you have two separate places, which adding possibly different > executors for a single object > without knowing about each other (otherwise why would you want to add > two finalizers instead of one). > Now if one decides to do #remove: , > how to ensure that second one won't be left with dirty/invalid state? > > > It's not ensured, and it doesn't have to be. The sender of #remove: takes > all responsibility. It can decide to add some executors back to the registry > if that's necessary. > If you want to keep your executors safe from some other code, which may > #remove: your objects from a WeakRegistry, then you can create a private > WeakRegistry and store your executors there, just like Sockets and > FileStreams do. > Conclusion: weak registry don't have to support multiple finalizers per single object. Instead, if one sees the need to control a finalization of certain object(s) in own way, indepentently fron any other, then he should use a private weak registry (like Sockets and FileStreams do), and register object and its executor there. -- Best regards, Igor Stasenko AKA sig.

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Levente Uzonyi

On Mon, 11 Oct 2010, Igor Stasenko wrote: > 2010/10/11 Levente Uzonyi <leves(a)elte.hu>: > On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: >> On Sun, 10 Oct 2010, Igor Stasenko wrote: >> >>> A current implementation of this method >>> >>> remove: oldObject ifAbsent: exceptionBlock >>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>> >>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >>> nil ]) >>> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >>> >>> simply removes a previously registered object from registry and voila. >>> >>> Now lets get back to our discussion about multiple finalizers per >>> object and using them in weak subscriptions etc. >>> >>> Suppose i am added a socket to weak registry, >>> and suppose i am added a weak subscription to it. >>> >>> Now, if i do 'socket close' , it tells weak registry to remove it from >>> list. >>> And what we'll have: >>> - socket handle is closed >>> - socket is wiped from weak registry >>> - but weak subscription still listed somewhere in a list of subscriptions >>> >>> >>> My suggestion is, that upon #remove:, >>> a weak registry should notify all executors that object of interest >>> are no longer takes part in finalization scheme, >>> so they should not count on receiving #finalize eventually. >>> >>> In other words: >>> >>> remove: oldObject ifAbsent: exceptionBlock >>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>> >>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>> Â Â Â Â ^(self protected: [ | executor | >>> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: >>> nil. >>> Â Â Â Â Â Â executor discardFinalization. >>> Â Â Â Â ]) >>> Â Â Â Â ifNil: [ exceptionBlock value ] >> >> It's only an issue with the new WeakRegistry implementation, previous >> implementations don't have such problem. I think changing the method as >> you >> suggested, implementing WeakFinalizerItem >> #discardFinalization as >> "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to >> ignore the executor if it's nil will fix the problem. Am I right? >> >> I don't get how "multiple finalizers per object" is related to this >> problem >> at all. >> > > No, you miss the point. > When you removing object from weak registry, it is important , time to > time to tell all of its executors, > that they will no longer receive #finalize, because object is no > longer a member of weak registry. > > If you simply set executor := nil, it does nothing, an executor itself > did not notified that he won't be needed > in any possible future. > So, if your finalization action is to remove some object(s) from the > list , you'll get your list dirty after object will die, > because #finalize will be never sent to your executor. > > Here the simple test case: > > | coll obj someWrapper | > coll := OrderedCollection new. > obj := Object new. > someWrapper := WeakArray with: obj. > coll add: someWrapper. > > obj toFinalizeSend: #remove: to: coll with: someWrapper. > obj finalizationRegistry remove: obj. > obj := nil. > Smalltalk garbageCollect. > self assert: coll isEmpty > > > the point is, that once you doing a #remove: , > your finalization scheme is broken. > > > I don't get you. When you remove an object from a WeakRegistry, you tell the > system that you want the object removed and all related executors deleted. > So the executors should never receive #finalize, they should be thrown away. > Even if there's action to be done with the executors (which is pointless > IMHO since those will be thrown away, but who knows), doing them is the > responsibility of the sender of #remove:. > Let us get back to multiple finalizers per object, when you have two separate places, which adding possibly different executors for a single object without knowing about each other (otherwise why would you want to add two finalizers instead of one). Now if one decides to do #remove: , how to ensure that second one won't be left with dirty/invalid state? It's not ensured, and it doesn't have to be. The sender of #remove: takes all responsibility. It can decide to add some executors back to the registry if that's necessary. If you want to keep your executors safe from some other code, which may #remove: your objects from a WeakRegistry, then you can create a private WeakRegistry and store your executors there, just like Sockets and FileStreams do. Levente > > Levente > >> >> Levente >> > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Igor Stasenko

2010/10/11 Levente Uzonyi <leves(a)elte.hu>: > On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: >> On Sun, 10 Oct 2010, Igor Stasenko wrote: >> >>> A current implementation of this method >>> >>> remove: oldObject ifAbsent: exceptionBlock >>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>> >>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >>> nil ]) >>> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >>> >>> simply removes a previously registered object from registry and voila. >>> >>> Now lets get back to our discussion about multiple finalizers per >>> object and using them in weak subscriptions etc. >>> >>> Suppose i am added a socket to weak registry, >>> and suppose i am added a weak subscription to it. >>> >>> Now, if i do 'socket close' , it tells weak registry to remove it from >>> list. >>> And what we'll have: >>> - socket handle is closed >>> - socket is wiped from weak registry >>> - but weak subscription still listed somewhere in a list of subscriptions >>> >>> >>> My suggestion is, that upon #remove:, >>> a weak registry should notify all executors that object of interest >>> are no longer takes part in finalization scheme, >>> so they should not count on receiving #finalize eventually. >>> >>> In other words: >>> >>> remove: oldObject ifAbsent: exceptionBlock >>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>> >>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>> Â Â Â Â ^(self protected: [ | executor | >>> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: >>> nil. >>> Â Â Â Â Â Â executor discardFinalization. >>> Â Â Â Â ]) >>> Â Â Â Â ifNil: [ exceptionBlock value ] >> >> It's only an issue with the new WeakRegistry implementation, previous >> implementations don't have such problem. I think changing the method as >> you >> suggested, implementing WeakFinalizerItem >> #discardFinalization as >> "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to >> ignore the executor if it's nil will fix the problem. Am I right? >> >> I don't get how "multiple finalizers per object" is related to this >> problem >> at all. >> > > No, you miss the point. > When you removing object from weak registry, it is important , time to > time to tell all of its executors, > that they will no longer receive #finalize, because object is no > longer a member of weak registry. > > If you simply set executor := nil, it does nothing, an executor itself > did not notified that he won't be needed > in any possible future. > So, if your finalization action is to remove some object(s) from the > list , you'll get your list dirty after object will die, > because #finalize will be never sent to your executor. > > Here the simple test case: > > | coll obj someWrapper | > coll := OrderedCollection new. > obj := Object new. > someWrapper := WeakArray with: obj. > coll add: someWrapper. > > obj toFinalizeSend: #remove: to: coll with: someWrapper. > obj finalizationRegistry remove: obj. > obj := nil. > Smalltalk garbageCollect. > self assert: coll isEmpty > > > the point is, that once you doing a #remove: , > your finalization scheme is broken. > > > I don't get you. When you remove an object from a WeakRegistry, you tell the > system that you want the object removed and all related executors deleted. > So the executors should never receive #finalize, they should be thrown away. > Even if there's action to be done with the executors (which is pointless > IMHO since those will be thrown away, but who knows), doing them is the > responsibility of the sender of #remove:. > Let us get back to multiple finalizers per object, when you have two separate places, which adding possibly different executors for a single object without knowing about each other (otherwise why would you want to add two finalizers instead of one). Now if one decides to do #remove: , how to ensure that second one won't be left with dirty/invalid state? > > Levente > >> >> Levente >> > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > -- Best regards, Igor Stasenko AKA sig.

Oct. 11, 2010

Re: [Pharo-project] Xtreams : first embryonary port on Squeak

by mkobetic＠gmail.com

"Schwab,Wilhelm K"<bschwab(a)anest.ufl.edu> wrote: > FWIW, SIF might help you: > > http://www.pocketsmalltalk.com/sif/ > > I have put a lot of code through it in the past couple of years. I'd recommend giving the Monticello tools in VW a try before trying to put something together from lower level tools. It's used to keep Seaside in sync in VW, so it should handle most of the obvious obstacles. With the help of your Xtreams-VWCompatibility package on Squeak side and maybe some adjustments in the portable code and maybe moving some of the non-portable code into packages of their own, we should be able to make this largely automatic.

Oct. 11, 2010

Re: [Pharo-project] Xtreams port to Squeak - second wave

by mkobetic＠gmail.com

Hi Nicolas, "Nicolas Cellier"<nicolas.cellier.aka.nice(a)gmail.com> wrote: > Hi again, > I now have ported two more packages > - Xtreams-Transforms > - Xtreams-Substreams > and their tests This is great! I loaded your code into Pharo 1.1 and things seem to be working quite well. There was a complaint about missing SharedQueue2, I just created a dummy subclass of SharedQueue with that name and things seemed to load fine. XTRecyclingCenter seems to be subclass of XTWriteStream, it should be Object subclass, maybe a typo ? > I did not have any portability problem with those... > But that's because I did not handle the Character encoder/decoder. > Consequently, I have 8 tests failing (the Base64 related tests) I was thinking, we could implement just 'encoding: #ascii' quite easily, to make this reasonably usable at least for applications that are fine with that. We're actually contemplating about implementing our own encoders for Xtreams too. The VW ones are tied to the classic streams more than we like. You might have noticed some rather hairy parts in the encoding streams yourself, where we're trying to work around some of the issues it creates. The advantage of reusing the existing encoders was that there are quite a few of those available, so reimplementing all that would be a drag. But we can come up with a scheme where we can reimplement at least the common ones and in VW we can still preserve hooking into the old ones for the rest. I can give that a try on VW side in the meantime so you could get those for free. > Plus 4 other tests failing because of my poor implementation of > #after:do: (forking processes in a SUnit TestCase can't be that > obvious). I looked at this, and I think this is how #after:do: should look like: after: aDelay do: aBlock "Evaluate the argument block delayed after the specified duration." | watchdog | watchdog := [ aDelay wait. aBlock value. ] newProcess. watchdog priority: Processor userInterruptPriority. watchdog resume. This would assume that the 2 tests calling #timeout:server:client: would use a Delay instance instead of a Duration, which I'd be fine with. However making that change doesn't quite get the tests running. It's blowing up with a DNU on the 'output close' bit in #terminate:server:client: with output being nil, which I'm having trouble figuring out. I can't find who could possibly be nilling it out. I'm somewhat struggling finding my way around the Pharo tools. The test seems to otherwise pass but the DNUs from the background process isn't nice. Also when I just click on the test in the TestRunner, I actually get four DNUs, not just one as I would expect. So I'm kinda stuck, not sure how to move forward without help from someone who knows his way around Pharo. I also get odd failure from #testWriteCollectingMultipleBufferSize, which seems to run fine (against collection) when I run the equivalent in a workspace, but strangely fails when running via the #timeout:server:client: construct, i.e. when client and server run in separate processes. Hm, now that I think of it, they sure could fail if something preempts the client, server processes at the right moment. I'll have to rethink that again. > Now, the easy part of the port (copy/paste) is almost ended. > Once we manage a compatible way to handle pragmas, PEG Parser should > port quite easily too. I wouldn't worry about the Parser stuff at this point. > Then, the harder work begins: > - File/Socket/Pipe > - Pointers (in External Heap) I wouldn't worry, about the external heap stuff either. It's neat, but probably not something many people will miss. > - Character encoding/decoding I'll see if I can help with this from the VW side. > - Compression/Decompression Is Zlib linked into the VM in Squeak too ? The compression streams are written directly against the ZLib API, so there aren't any VW specific dependencies other than how those calls are made. Similarly the crypto streams go directly against the EVP API in LibCrypto in OpenSSL, so as long as we can abstract over how those are called, the stream implementation should work as is. > If you think you can help in any of these, please tell. If you could compile a list of changes that you'd like us to adopt on the VW side, I'd certainly look at that. I did read your posts but it's not entirely clear what you'd like to handle on Squeak side and what on VW side. A fileout would be best to avoid any confusion, but a description is fine too. I'm also unclear about the become: discussion. Don't write streams in Squeak become: the underlying collection when they grow it ? Cheers, Martin

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Schwab,Wilhelm K

Sig, Dumb question: how much does an executor need to know about this? It can't strongly bind the outgoing object or weaklings won't let go and finalization will never happen. Whether the guidance to it is in the form of "clean up" or "skip it," it needs to come from the weak registry, finalization framework, or whatever you want to call it. Dolphin handles finalization with support from the VM, allowing the outgoing object to do the finalization itself, often in the form of releasing external resources, so there is no concept of executor, let alone multiple executors. I have been arguing in favor of thread safety for these mechanisms, and I suspect this is another example of why it is needed. Why would one want more than one object ready to destroy an external handle when a single object disappears? The creation of the redundant executors is a recipe for for a double free and resulting crash. I think we should prevent such redundancy, and would be readily able to do so if the registries were thread safe giving #at:ifAbsentPut: the chops to do the job. Am I missing something? Bill ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua(a)gmail.com] Sent: Sunday, October 10, 2010 7:10 PM To: The general-purpose Squeak developers list Cc: Pharo Development Subject: Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: > On Sun, 10 Oct 2010, Igor Stasenko wrote: > >> A current implementation of this method >> >> remove: oldObject ifAbsent: exceptionBlock >> "Remove oldObject as one of the receiver's elements." >> >> oldObject ifNil: [ ^nil ]. >> ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >> nil ]) >> ifNil: [ exceptionBlock value ] >> >> simply removes a previously registered object from registry and voila. >> >> Now lets get back to our discussion about multiple finalizers per >> object and using them in weak subscriptions etc. >> >> Suppose i am added a socket to weak registry, >> and suppose i am added a weak subscription to it. >> >> Now, if i do 'socket close' , it tells weak registry to remove it from >> list. >> And what we'll have: >> - socket handle is closed >> - socket is wiped from weak registry >> - but weak subscription still listed somewhere in a list of subscriptions >> >> >> My suggestion is, that upon #remove:, >> a weak registry should notify all executors that object of interest >> are no longer takes part in finalization scheme, >> so they should not count on receiving #finalize eventually. >> >> In other words: >> >> remove: oldObject ifAbsent: exceptionBlock >> "Remove oldObject as one of the receiver's elements." >> >> oldObject ifNil: [ ^nil ]. >> ^(self protected: [ | executor | >> executor := valueDictionary removeKey: oldObject ifAbsent: nil. >> executor discardFinalization. >> ]) >> ifNil: [ exceptionBlock value ] > > It's only an issue with the new WeakRegistry implementation, previous > implementations don't have such problem. I think changing the method as you > suggested, implementing WeakFinalizerItem >> #discardFinalization as > "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to > ignore the executor if it's nil will fix the problem. Am I right? > > I don't get how "multiple finalizers per object" is related to this problem > at all. > No, you miss the point. When you removing object from weak registry, it is important , time to time to tell all of its executors, that they will no longer receive #finalize, because object is no longer a member of weak registry. If you simply set executor := nil, it does nothing, an executor itself did not notified that he won't be needed in any possible future. So, if your finalization action is to remove some object(s) from the list , you'll get your list dirty after object will die, because #finalize will be never sent to your executor. Here the simple test case: | coll obj someWrapper | coll := OrderedCollection new. obj := Object new. someWrapper := WeakArray with: obj. coll add: someWrapper. obj toFinalizeSend: #remove: to: coll with: someWrapper. obj finalizationRegistry remove: obj. obj := nil. Smalltalk garbageCollect. self assert: coll isEmpty the point is, that once you doing a #remove: , your finalization scheme is broken. > > Levente > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Levente Uzonyi

On Mon, 11 Oct 2010, Igor Stasenko wrote: > On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: > On Sun, 10 Oct 2010, Igor Stasenko wrote: > >> A current implementation of this method >> >> remove: oldObject ifAbsent: exceptionBlock >> Â Â Â Â "Remove oldObject as one of the receiver's elements." >> >> Â Â Â Â oldObject ifNil: [ ^nil ]. >> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >> nil ]) >> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >> >> simply removes a previously registered object from registry and voila. >> >> Now lets get back to our discussion about multiple finalizers per >> object and using them in weak subscriptions etc. >> >> Suppose i am added a socket to weak registry, >> and suppose i am added a weak subscription to it. >> >> Now, if i do 'socket close' , it tells weak registry to remove it from >> list. >> And what we'll have: >> - socket handle is closed >> - socket is wiped from weak registry >> - but weak subscription still listed somewhere in a list of subscriptions >> >> >> My suggestion is, that upon #remove:, >> a weak registry should notify all executors that object of interest >> are no longer takes part in finalization scheme, >> so they should not count on receiving #finalize eventually. >> >> In other words: >> >> remove: oldObject ifAbsent: exceptionBlock >> Â Â Â Â "Remove oldObject as one of the receiver's elements." >> >> Â Â Â Â oldObject ifNil: [ ^nil ]. >> Â Â Â Â ^(self protected: [ | executor | >> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: nil. >> Â Â Â Â Â Â executor discardFinalization. >> Â Â Â Â ]) >> Â Â Â Â ifNil: [ exceptionBlock value ] > > It's only an issue with the new WeakRegistry implementation, previous > implementations don't have such problem. I think changing the method as you > suggested, implementing WeakFinalizerItem >> #discardFinalization as > "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to > ignore the executor if it's nil will fix the problem. Am I right? > > I don't get how "multiple finalizers per object" is related to this problem > at all. > No, you miss the point. When you removing object from weak registry, it is important , time to time to tell all of its executors, that they will no longer receive #finalize, because object is no longer a member of weak registry. If you simply set executor := nil, it does nothing, an executor itself did not notified that he won't be needed in any possible future. So, if your finalization action is to remove some object(s) from the list , you'll get your list dirty after object will die, because #finalize will be never sent to your executor. Here the simple test case: | coll obj someWrapper | coll := OrderedCollection new. obj := Object new. someWrapper := WeakArray with: obj. coll add: someWrapper. obj toFinalizeSend: #remove: to: coll with: someWrapper. obj finalizationRegistry remove: obj. obj := nil. Smalltalk garbageCollect. self assert: coll isEmpty the point is, that once you doing a #remove: , your finalization scheme is broken. I don't get you. When you remove an object from a WeakRegistry, you tell the system that you want the object removed and all related executors deleted. So the executors should never receive #finalize, they should be thrown away. Even if there's action to be done with the executors (which is pointless IMHO since those will be thrown away, but who knows), doing them is the responsibility of the sender of #remove:. Levente > > Levente > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 10, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Igor Stasenko

On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: > On Sun, 10 Oct 2010, Igor Stasenko wrote: > >> A current implementation of this method >> >> remove: oldObject ifAbsent: exceptionBlock >> Â Â Â Â "Remove oldObject as one of the receiver's elements." >> >> Â Â Â Â oldObject ifNil: [ ^nil ]. >> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: >> nil ]) >> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >> >> simply removes a previously registered object from registry and voila. >> >> Now lets get back to our discussion about multiple finalizers per >> object and using them in weak subscriptions etc. >> >> Suppose i am added a socket to weak registry, >> and suppose i am added a weak subscription to it. >> >> Now, if i do 'socket close' , it tells weak registry to remove it from >> list. >> And what we'll have: >> - socket handle is closed >> - socket is wiped from weak registry >> - but weak subscription still listed somewhere in a list of subscriptions >> >> >> My suggestion is, that upon #remove:, >> a weak registry should notify all executors that object of interest >> are no longer takes part in finalization scheme, >> so they should not count on receiving #finalize eventually. >> >> In other words: >> >> remove: oldObject ifAbsent: exceptionBlock >> Â Â Â Â "Remove oldObject as one of the receiver's elements." >> >> Â Â Â Â oldObject ifNil: [ ^nil ]. >> Â Â Â Â ^(self protected: [ | executor | >> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: nil. >> Â Â Â Â Â Â executor discardFinalization. >> Â Â Â Â ]) >> Â Â Â Â ifNil: [ exceptionBlock value ] > > It's only an issue with the new WeakRegistry implementation, previous > implementations don't have such problem. I think changing the method as you > suggested, implementing WeakFinalizerItem >> #discardFinalization as > "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to > ignore the executor if it's nil will fix the problem. Am I right? > > I don't get how "multiple finalizers per object" is related to this problem > at all. > No, you miss the point. When you removing object from weak registry, it is important , time to time to tell all of its executors, that they will no longer receive #finalize, because object is no longer a member of weak registry. If you simply set executor := nil, it does nothing, an executor itself did not notified that he won't be needed in any possible future. So, if your finalization action is to remove some object(s) from the list , you'll get your list dirty after object will die, because #finalize will be never sent to your executor. Here the simple test case: | coll obj someWrapper | coll := OrderedCollection new. obj := Object new. someWrapper := WeakArray with: obj. coll add: someWrapper. obj toFinalizeSend: #remove: to: coll with: someWrapper. obj finalizationRegistry remove: obj. obj := nil. Smalltalk garbageCollect. self assert: coll isEmpty the point is, that once you doing a #remove: , your finalization scheme is broken. > > Levente > -- Best regards, Igor Stasenko AKA sig.

Oct. 10, 2010

Re: [Pharo-project] [squeak-dev] Compiler pedantic about ifNotNil: argument

by Igor Stasenko

On 10 October 2010 23:25, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > > On Oct 10, 2010, at 3:07 PM, Igor Stasenko wrote: > >> On 10 October 2010 15:39, Nicolas Cellier >> <nicolas.cellier.aka.nice(a)gmail.com> wrote: >>> The compiler uselessly insist on #ifNotNil: argument being a zero/one arg block. >>> Thus we cannot write this xtream sentence >>> >>> Â Â process ifNotNil: #terminate. >>> >>> When the argument is not a block, Compiler should avoid inlining and >>> just send a normal message. >>> >> +1 >> >> i am also missing: >> >> someThing ifTrue: 1 ifFalse: 0 > > I hate this idea. > Let the compiler doing optimizations and please do not mix implementation and language design. > The ***semantics*** of iftrue: is to get a thunk (in scheme parlance) something whose evaluation is blocked. > 1 is not that! > hmm.. what is wrong with sending real #ifTrue:ifFalse: message to boolean, if compiler can't inline it? or you think a following is a good way to write a code: blockA := [self foo ]. blockB := [ self bar ]. self zork ifTrue: [ blockA value] ifFalse: [ blockB value ] where you can just write: self zork ifTrue: blockA ifFalse: blockB ? In any way, passing anything else than block literal as argument should work. This is smalltalk, not C with static types. > Now the compiler could be smart and inline whatever. > > Stef > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

Oct. 10, 2010

Re: [Pharo-project] R: R: R: Problem with inspector

by Frederic Pluquet

It's really strange. It works perfectly here. Step by step: - Download the last image of Pharo - Go to Monticello and load the last package of NewInspector - Go to System > Settings > Debugging > NewInspector - Check the options - Open a new inspector on an object The new children must appear. FrÃ©d ---- FrÃ©dÃ©ric Pluquet UniversitÃ© Libre de Bruxelles (ULB) Assistant http://www.ulb.ac.be/di/fpluquet 2010/10/10 morod <morod(a)tiscali.it> > Thanks for your interest, I have done what was suggested, but nothing has > changed. > > > > Morod > > > > *Da:* pharo-project-bounces(a)lists.gforge.inria.fr [mailto: > pharo-project-bounces(a)lists.gforge.inria.fr] *Per conto di *Frederic > Pluquet > *Inviato:* domenica 10 ottobre 2010 17:00 > > *A:* Pharo-project(a)lists.gforge.inria.fr > *Oggetto:* Re: [Pharo-project] R: R: Problem with inspector > > > > If you load the last version of NewInspector (posted this morning), you go > to the Settings Browser (System > Settings in the World menu), expand > "Debugging" > "NewInspector" and check the options you want. > > > > FrÃ©d > > ---- > FrÃ©dÃ©ric Pluquet > UniversitÃ© Libre de Bruxelles (ULB) > Assistant > http://www.ulb.ac.be/di/fpluquet > > 2010/10/10 morod <morod(a)tiscali.it> > > Hello, > > > > How can I switch from the two different versions of tool âInspectorâ > > > > Best regards > > > > Morod > > > > *Da:* pharo-project-bounces(a)lists.gforge.inria.fr [mailto: > pharo-project-bounces(a)lists.gforge.inria.fr] *Per conto di *Frederic > Pluquet > *Inviato:* sabato 9 ottobre 2010 14:08 > > > *A:* Pharo-project(a)lists.gforge.inria.fr > > *Oggetto:* Re: [Pharo-project] R: Problem with inspector > > > > Hello, > > > > It's the NewInspector on both pictures but two different versions of this > tool. In the old version, the class and the methods were displayed for each > inspected object. In the current version, there are not displayed (because > the community prefers less children for each object). > > > > FrÃ©d > > ---- > FrÃ©dÃ©ric Pluquet > UniversitÃ© Libre de Bruxelles (ULB) > Assistant > http://www.ulb.ac.be/di/fpluquet > > 2010/10/8 morod <morod(a)tiscali.it> > > Hi Mariano > > > > I tried it with multiple images and the behavior is the same with all the > images. > > > > Sent the sample is made with: > > Pharo-1.1-11411 > > Latest update: #11411 > > > > The first one is my image and the second is taken from the book âPharo by > Exampleâ > > > > Thanks again > > > > Morod > > > > *Da:* pharo-project-bounces(a)lists.gforge.inria.fr [mailto: > pharo-project-bounces(a)lists.gforge.inria.fr] *Per conto di *Mariano > Martinez Peck > *Inviato:* venerdÃ¬ 8 ottobre 2010 21:43 > *A:* Pharo-project(a)lists.gforge.inria.fr > *Oggetto:* Re: [Pharo-project] Problem with inspector > > > > Hi morod, and welcome to Pharo! > > Which Pharo image are you exactly using? (the name of the zip, the image > file or go to System -> About) The first one is your image, right ? and > the second one? where is it from ? > > The difference is that the first image is using the "old" traditional > inspector, while the second picture is using another inspector, which is > called "NewInspector", which provides more information. > > Cheers > > Mariano > > 2010/10/8 morod <morod(a)tiscali.it> > > Hi > > iâm new to smalltalk and Pharo and my inspector show > > > > *Errore. Il nome file non Ã¨ specificato.* > > > > instead of > > > > *Errore. Il nome file non Ã¨ specificato.* > > Why? Thanks > > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 10, 2010

Re: [Pharo-project] Getting my head around low space

by John M McIntosh

On 2010-10-10, at 2:00 PM, Max Leske wrote: > I had a first failure with 1 GB. I used the 4.2.5b1U VM. Setting the memory to anything above 1 GB would lead to an error message. I'm now trying the 5.7b1 VM with 3.5 GB. > > Max So what is the error message? -- =========================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================

Oct. 10, 2010

Re: [Pharo-project] Getting my head around low space

by Max Leske

I had a first failure with 1 GB. I used the 4.2.5b1U VM. Setting the memory to anything above 1 GB would lead to an error message. I'm now trying the 5.7b1 VM with 3.5 GB. Max On 10.10.2010, at 06:51, John M McIntosh wrote: > It's fairly clear: " the Mac VM has an option -memory" > no -mmap > > use it or set the info.plist setting of SqueakMaxHeapSize. > You can in theory set values up to 4GB on a 4.2.x VM or 5.7.x > Might make lights dim and core dump on a 5.8.x VM, please try... > Obviously you have to actually use the 4GB of memory. > > However I suspect asking for anything close to or over 5.8GB on a 4.2.x and 5.8.x VM won't fly. > Should work with a 5.7.x VM > > > On 2010-10-09, at 8:14 PM, Levente Uzonyi wrote: > >> On Sat, 9 Oct 2010, Max Leske wrote: >> >>> I'm actually working on a mac, so that shouldn't be the problem. >>> So your proposition to use -mmap would be unix specific? Is the option available on mac? >> >> I don't have a Mac, but you can easily check it by starting an image with "-mmap 64m", then allocating a 64MB array in the image. If it fails, then the switch works, otherwise not. >> >> >> Levente >> >>> >>> Thanks, >>> Max >>> >>> >>> On 09.10.2010, at 21:34, Levente Uzonyi wrote: >>> >>>> On Sat, 9 Oct 2010, Adrian Lienhard wrote: >>>> >>>>> The unix VM and apparently also the Mac VM has an option -memory that allows you to set a heap size: >>>>> >>>>> [adrian:/Applications/Squeak 4.2.3beta1U.app/Contents/MacOS] ./Squeak\ VM\ Opt -help >>>>> Usage: ./Squeak VM Opt [<option>...] [<imageName> [<argument>...]] >>>>> ./Squeak VM Opt [<option>...] -- [<argument>...] >>>>> >>>>> Common <option>s: >>>>> -help print this help message, then exit >>>>> -memory <size>[mk] use fixed heap size (added to image size) >>>>> -pathenc <enc> set encoding for pathnames (default: macintosh) >>>>> -headless run in headless (no window) mode (default: false) >>>>> >>>>> Notes: >>>>> <imageName> defaults to `Squeak.image'. >>>>> If `-memory' is not specified then the heap will grow dynamically. >>>>> <argument>s are ignored, but are processed by the Squeak image. >>>>> The first <argument> normally names a Squeak `script' to execute. >>>>> Precede <arguments> by `--' to use default image. >>>> >>>> This is ok for Mac, but the unix VM takes different arguments. >>>> >>>> >>>> Levente >>>> >>>>> >>>>> >>>>> HTH, >>>>> Adrian >>>>> >>>>> On Oct 9, 2010, at 18:01 , Max Leske wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> I'm having trouble with low space warnings in Pharo. Is there a source on how much memory the image can get from the OS? Is there maybe a way to up that limit? I am loading a lot of data into the image and the limit at the moment seems to be around 500 MB. >>>>>> >>>>>> Thanks for any ideas. >>>>>> >>>>>> Cheers, >>>>>> Max >>>>>> _______________________________________________ >>>>>> Pharo-project mailing list >>>>>> Pharo-project(a)lists.gforge.inria.fr >>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pharo-project mailing list >>>>> Pharo-project(a)lists.gforge.inria.fr >>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>>> >>>> >>>> _______________________________________________ >>>> Pharo-project mailing list >>>> Pharo-project(a)lists.gforge.inria.fr >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > -- > =========================================================================== > John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 > Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com > =========================================================================== > > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 10, 2010

Re: [Pharo-project] WeakRegistry>>remove: - when you'll be in trouble

by Levente Uzonyi

On Sun, 10 Oct 2010, Igor Stasenko wrote: > A current implementation of this method > > remove: oldObject ifAbsent: exceptionBlock > "Remove oldObject as one of the receiver's elements." > > oldObject ifNil: [ ^nil ]. > ^(self protected: [ valueDictionary removeKey: oldObject ifAbsent: nil ]) > ifNil: [ exceptionBlock value ] > > simply removes a previously registered object from registry and voila. > > Now lets get back to our discussion about multiple finalizers per > object and using them in weak subscriptions etc. > > Suppose i am added a socket to weak registry, > and suppose i am added a weak subscription to it. > > Now, if i do 'socket close' , it tells weak registry to remove it from list. > And what we'll have: > - socket handle is closed > - socket is wiped from weak registry > - but weak subscription still listed somewhere in a list of subscriptions > > > My suggestion is, that upon #remove:, > a weak registry should notify all executors that object of interest > are no longer takes part in finalization scheme, > so they should not count on receiving #finalize eventually. > > In other words: > > remove: oldObject ifAbsent: exceptionBlock > "Remove oldObject as one of the receiver's elements." > > oldObject ifNil: [ ^nil ]. > ^(self protected: [ | executor | > executor := valueDictionary removeKey: oldObject ifAbsent: nil. > executor discardFinalization. > ]) > ifNil: [ exceptionBlock value ] It's only an issue with the new WeakRegistry implementation, previous implementations don't have such problem. I think changing the method as you suggested, implementing WeakFinalizerItem >> #discardFinalization as "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to ignore the executor if it's nil will fix the problem. Am I right? I don't get how "multiple finalizers per object" is related to this problem at all. Levente > > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 10, 2010

Re: [Pharo-project] [squeak-dev] Compiler pedantic about ifNotNil: argument

by StÃ©phane Ducasse

On Oct 10, 2010, at 3:07 PM, Igor Stasenko wrote: > On 10 October 2010 15:39, Nicolas Cellier > <nicolas.cellier.aka.nice(a)gmail.com> wrote: >> The compiler uselessly insist on #ifNotNil: argument being a zero/one arg block. >> Thus we cannot write this xtream sentence >> >> process ifNotNil: #terminate. >> >> When the argument is not a block, Compiler should avoid inlining and >> just send a normal message. >> > +1 > > i am also missing: > > someThing ifTrue: 1 ifFalse: 0 I hate this idea. Let the compiler doing optimizations and please do not mix implementation and language design. The ***semantics*** of iftrue: is to get a thunk (in scheme parlance) something whose evaluation is blocked. 1 is not that! Now the compiler could be smart and inline whatever. Stef

Oct. 10, 2010

[Pharo-project] Xtreams port to Squeak - second wave

by Nicolas Cellier

Hi again, I now have ported two more packages - Xtreams-Transforms - Xtreams-Substreams and their tests I did not have any portability problem with those... But that's because I did not handle the Character encoder/decoder. Consequently, I have 8 tests failing (the Base64 related tests) Plus 4 other tests failing because of my poor implementation of #after:do: (forking processes in a SUnit TestCase can't be that obvious). Now, the easy part of the port (copy/paste) is almost ended. Once we manage a compatible way to handle pragmas, PEG Parser should port quite easily too. Then, the harder work begins: - File/Socket/Pipe - Pointers (in External Heap) - Character encoding/decoding - Compression/Decompression If you think you can help in any of these, please tell. Nicolas

Oct. 10, 2010

Re: [Pharo-project] Smalltalk Interchange Format [was Xtreams : first embryonary port on Squeak]

by Schwab,Wilhelm K

Nicolas, Can you get sifall.zip? That's what I use. Sadly I am not sure about the license, so while I have no worries about directing you to the source, I am not in a hurry to host it myself. It's old, but I was impressed at how well it worked at getting code out of Dolphin. Pharo/MC acts a little badly about some of the *xzy category names that result, but nothing so bad that you can't fix them with a text editor, the regex package, or by browsing/editing offending names. Bill ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice(a)gmail.com] Sent: Sunday, October 10, 2010 2:48 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] Smalltalk Interchange Format [was Xtreams : first embryonary port on Squeak] Thanks Bill, some of the link on this page seem dead though... Nicolas 2010/10/10 Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>: > FWIW, SIF might help you: > > http://www.pocketsmalltalk.com/sif/ > > I have put a lot of code through it in the past couple of years. > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice(a)gmail.com] > Sent: Saturday, October 09, 2010 6:33 PM > To: The general-purpose Squeak developers list; Pharo Development; vwnc(a)cs.uiuc.edu > Subject: [Pharo-project] Xtreams : first embryonary port on Squeak > > So, I started a quick port of VW Xtreams to Squeak this evening. > Only the Xtreams-Core and the trivial Xtreams-Terminals (no > file/pipe/socket/pointer). > You'll find code at http://www.squeaksource.com/XTream. > My previous experimental Xtream project has been renamed SqueaXTream > to reduce confusion, but still sits in the same project. > Please find my report below. > > cheers > > Nicolas > > --------------------- REPORT --------------------- > > THE CONTENTS: > > This first port is made of 3 packages > - Xtreams-Core > - Xtreams-Terminals > - Xtreams-VWCompatible > Plus 2 tests > - Xtreams-CoreTests > - Xtreams-TerminalsTests > Please notice I removed one hyphen from test package names for MC compatibility. > > THE PROCEDURE: > > I didn't use FileOut30 not any more advanced tool, but just a simple > copy/paste strategy. > This gave me a chance to do a fast review of code. > For porting the future evolutions of VW Xtreams, more advanced tools > will be necessary, the manual approach does not scale. > > THE MODIFICATIONS: > > The changes I made are quite restricted, which indicates that Xtreams > is not that hard to port. > Main changes are due to lack of namespace in Squeak: > - I added a XT prefix to every Xtreams classes (no Namespace in Squeak) > - I added XTPools, a SharedPool for holding the single Xtreams > namespace variable DefaultBufferSize > - I transformed class SharedVariables into class variables and added > proper class side initialization for these in XTWriteStream > > For the rest, I tried to not change Xtreams contents. but rather did > implement VW compatible messages when possible in the > Xtreams-VWCompatible package when they did not exist in Squeak. > These message are a full rewrite and are also released under MIT license. > The few exceptions are: > - I modified originator -> self originator in XTIncomplete. > - I changed the senders of #newInFixedSpace: to rather send an #error: > 'not implemented in Squeak' in XTRecyclingBuffer > - I replaced #waitIfCurtailedSignal/#signal with a #critical: section > in XTRecyclingBuffer > - I replaced Core.Timer after:do: with self after:do: in Xtreams-CoreTests > > A DISCUSSION OF TECHNICAL DETAILS: > > I first attempted to replace #growToAtLeast: with #grownBy: because > Squeak become: is notoriously inefficient. > But obviously, this did not work, the tests rely on destination > preserving its identity. > This clearly is going to degrade micro benchmarks when the destination > collection capacity is not well adjusted in advance. > I think we should discuss this particular point. Is identity > preservation absolutely required, or just convenient ? > > THE STATUS OF TESTS: > > Tests do not all pass. There seems to be a bug in Squeak > #replace:from:to:with: when the replacement is the collection itself, > moved to the right (this is with a COG VM). > > THE FUTURE: > > I will probably continue porting with reduced activity, so I invite > any interested person to help. Right now, the project is world > writeable, code commited here falling under MIT license. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 10, 2010

Re: [Pharo-project] [OT] database choices

by Philippe Marschall

On 10.10.2010 20:38, Schwab,Wilhelm K wrote: > Just learned about this one: > > http://mariadb.org/index.html > > It sounds like the old guard of MySQL is coalescing around it. It's up to you to decide whether that is true, and if so, whether it is a good thing. It's funny because for years Oracle has been making the only usable MySQL engine: InnoDB. Cheers Philippe

Oct. 10, 2010

Re: [Pharo-project] Help request on Xtreams tests with forked process [ was Xtreams : first embryonary port on Squeak ]

by Schwab,Wilhelm K

Nicolas, I'm told there is no such thing as a free lunch<g>, but if you tell me what you want the menu to look like, I might have something that would be useful. Bill ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice(a)gmail.com] Sent: Sunday, October 10, 2010 2:45 PM To: Pharo-project(a)lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Help request on Xtreams tests with forked process [ was Xtreams : first embryonary port on Squeak ] Hi Bill, I see, you're helping me to help myself :) What I was looking at was more like a free lunch ;) Nicolas 2010/10/10 Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>: > Nicolas, > > #fork in a test? Do you mean like the code below? The thread running the test waits on something signaled toward the end of the forked thread. > > If you want a watchdog timer, you could fork another thread that waits on a delay, sets an error flag (best to use a shared queue or a mutex to avoid weirdness) and then signals the test semaphore. You might also want to use #ifCurtailed: to set an error flag and wrap that with #ensure: to signal the semaphore. You don't necessarily need to be perfect about signaling the semaphore because failure to do so is not much different from your test going into an infinite loop (actually more benign because the stack isn't growing); you'll realize it, break, and figure out what didn't happen or at least "that it didn't" and try again forewarned. > > Does that help? > > Bill > > > testLocalServerAndClient > "11-09 - try a server and a client in the same image." > > | server port client serverSocket serverSocketStream text thread done read | > > port := 4900. > done := Semaphore new. > > "11-09 - gag the prompter below" > serverSocket := nil. > > server := ConnectionQueue portNumber:port queueLength:5. > [ > client := SocketStream openConnectionToHostNamed:'localhost' port:port. > [ serverSocket isNil ] whileTrue:[ > serverSocket := server getConnectionOrNil. > ]. > > serverSocketStream := SocketStream on:serverSocket. > text := 'Long live Tux!'. > serverSocketStream nextPutAll:text; flush. > > "11-09 - this hung up w/o the thread. > 11-09 - as you were; the problem was a lack of #flush on the send side, so there was > nothing for the client side to read." > thread := [ > read := client nextMany:text size. > done signal. > ] forkAt:Processor userBackgroundPriority. > thread name:'READ'. > > done wait. > self should:[ read = text ]. > Transcript nextPutAll:'Just to make the point, we read: '; nextPutAll:read; cr; flush. > > ] ensure:[ > server destroy. > serverSocketStream close. > client close. > ]. > > "11-09 - no prompts" > serverSocket := serverSocket. > client := client. > serverSocketStream := serverSocketStream. > thread := thread. > > " > NetworkSmokeTestCase prod:#testLocalServerAndClient. > " > > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice(a)gmail.com] > Sent: Sunday, October 10, 2010 7:05 AM > To: The general-purpose Squeak developers list > Cc: Pharo Development > Subject: Re: [Pharo-project] [squeak-dev] Xtreams : first embryonary port on Squeak > > 2010/10/10 Levente Uzonyi <leves(a)elte.hu>: >> On Sun, 10 Oct 2010, Nicolas Cellier wrote: >> >>> >>> THE STATUS OF TESTS: >>> >>> Tests do not all pass. There seems to be a bug in Squeak >>> #replace:from:to:with: when the replacement is the collection itself, >>> moved to the right (this is with a COG VM). >> >> #replace:from:to:with: is not supposed to work for moves. IIRC the primitive >> version uses memcpy (or strncpy), so moving to the left works just because >> the undelying C function supports it. I used to (ab)use this feature though. >> >> >> Levente >> > > OK, since we use memcpy rather than memmove in Squeak VM, I > implemented the trivial work around - move through a motionBuffer > temporary copy. > > I now have a single test failing, > XTCollectionWriteStream>>#testReadWriteLargeAmount and this must be > due to my #after:do: simplistic implementation. > Messing with #fork in SUnit TestCase is tricky, and I need help on this point. > > Nicolas > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 10, 2010

Re: [Pharo-project] Smalltalk Interchange Format [was Xtreams : first embryonary port on Squeak]

by Nicolas Cellier

Thanks Bill, some of the link on this page seem dead though... Nicolas 2010/10/10 Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>: > FWIW, SIF might help you: > > Â http://www.pocketsmalltalk.com/sif/ > > I have put a lot of code through it in the past couple of years. > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice(a)gmail.com] > Sent: Saturday, October 09, 2010 6:33 PM > To: The general-purpose Squeak developers list; Pharo Development; vwnc(a)cs.uiuc.edu > Subject: [Pharo-project] Xtreams : first embryonary port on Squeak > > So, I started a quick port of VW Xtreams to Squeak this evening. > Only the Xtreams-Core and the trivial Xtreams-Terminals (no > file/pipe/socket/pointer). > You'll find code at http://www.squeaksource.com/XTream. > My previous experimental Xtream project has been renamed SqueaXTream > to reduce confusion, but still sits in the same project. > Please find my report below. > > cheers > > Nicolas > > --------------------- REPORT Â --------------------- > > THE CONTENTS: > > This first port is made of 3 packages > - Xtreams-Core > - Xtreams-Terminals > - Xtreams-VWCompatible > Plus 2 tests > - Xtreams-CoreTests > - Xtreams-TerminalsTests > Please notice I removed one hyphen from test package names for MC compatibility. > > THE PROCEDURE: > > I didn't use FileOut30 not any more advanced tool, but just a simple > copy/paste strategy. > This gave me a chance to do a fast review of code. > For porting the future evolutions of VW Xtreams, more advanced tools > will be necessary, the manual approach does not scale. > > THE MODIFICATIONS: > > The changes I made are quite restricted, which indicates that Xtreams > is not that hard to port. > Main changes are due to lack of namespace in Squeak: > - I added a XT prefix to every Xtreams classes (no Namespace in Squeak) > - I added XTPools, a SharedPool for holding the single Xtreams > namespace variable DefaultBufferSize > - I transformed class SharedVariables into class variables and added > proper class side initialization for these in XTWriteStream > > For the rest, I tried to not change Xtreams contents. but rather did > implement VW compatible messages when possible in the > Xtreams-VWCompatible package when they did not exist in Squeak. > These message are a full rewrite and are also released under MIT license. > The few exceptions are: > - I modified originator -> self originator in XTIncomplete. > - I changed the senders of #newInFixedSpace: to rather send an #error: > 'not implemented in Squeak' in XTRecyclingBuffer > - I replaced #waitIfCurtailedSignal/#signal with a #critical: section > in XTRecyclingBuffer > - I replaced Â Core.Timer after:do: with self after:do: in Xtreams-CoreTests > > A DISCUSSION OF TECHNICAL DETAILS: > > I first attempted to replace #growToAtLeast: with #grownBy: because > Squeak become: is notoriously inefficient. > But obviously, this did not work, the tests rely on destination > preserving its identity. > This clearly is going to degrade micro benchmarks when the destination > collection capacity is not well adjusted in advance. > I think we should discuss this particular point. Is identity > preservation absolutely required, or just convenient ? > > THE STATUS OF TESTS: > > Tests do not all pass. There seems to be a bug in Squeak > #replace:from:to:with: when the replacement is the collection itself, > moved to the right (this is with a COG VM). > > THE FUTURE: > > I will probably continue porting with reduced activity, so I invite > any interested person to help. Right now, the project is world > writeable, code commited here falling under MIT license. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 10, 2010

Re: [Pharo-project] Help request on Xtreams tests with forked process [ was Xtreams : first embryonary port on Squeak ]

by Nicolas Cellier

Hi Bill, I see, you're helping me to help myself :) What I was looking at was more like a free lunch ;) Nicolas 2010/10/10 Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>: > Nicolas, > > #fork in a test? Â Do you mean like the code below? Â The thread running the test waits on something signaled toward the end of the forked thread. > > If you want a watchdog timer, you could fork another thread that waits on a delay, sets an error flag (best to use a shared queue or a mutex to avoid weirdness) and then signals the test semaphore. Â You might also want to use #ifCurtailed: to set an error flag and wrap that with #ensure: to signal the semaphore. Â You don't necessarily need to be perfect about signaling the semaphore because failure to do so is not much different from your test going into an infinite loop (actually more benign because the stack isn't growing); you'll realize it, break, and figure out what didn't happen or at least "that it didn't" and try again forewarned. > > Does that help? > > Bill > > > testLocalServerAndClient > Â Â Â Â "11-09 - try a server and a client in the same image." > > Â Â Â Â Â Â Â Â | server port client serverSocket serverSocketStream text thread done read | > > Â Â Â Â port := 4900. > Â Â Â Â done := Semaphore new. > > Â Â Â Â "11-09 - gag the prompter below" > Â Â Â Â serverSocket := nil. > > Â Â Â Â server := ConnectionQueue portNumber:port queueLength:5. > Â Â Â Â [ > Â Â Â Â Â Â Â Â client := SocketStream openConnectionToHostNamed:'localhost' port:port. > Â Â Â Â Â Â Â Â [ serverSocket isNil ] whileTrue:[ > Â Â Â Â Â Â Â Â Â Â Â Â serverSocket := server getConnectionOrNil. > Â Â Â Â Â Â Â Â ]. > > Â Â Â Â Â Â Â Â serverSocketStream := SocketStream on:serverSocket. > Â Â Â Â Â Â Â Â text := 'Long live Tux!'. > Â Â Â Â Â Â Â Â serverSocketStream nextPutAll:text; flush. > > Â Â Â Â Â Â Â Â "11-09 - this hung up w/o the thread. > Â Â Â Â Â Â Â Â 11-09 - as you were; the problem was a lack of #flush on the send side, so there was > Â Â Â Â Â Â Â Â nothing for the client side to read." > Â Â Â Â Â Â Â Â thread := [ > Â Â Â Â Â Â Â Â Â Â Â Â read := client nextMany:text size. > Â Â Â Â Â Â Â Â Â Â Â Â done signal. > Â Â Â Â Â Â Â Â ] forkAt:Processor userBackgroundPriority. > Â Â Â Â Â Â Â Â thread name:'READ'. > > Â Â Â Â Â Â Â Â done wait. > Â Â Â Â Â Â Â Â self should:[ read = text ]. > Â Â Â Â Â Â Â Â Transcript nextPutAll:'Just to make the point, we read: '; nextPutAll:read; cr; flush. > > Â Â Â Â ] ensure:[ > Â Â Â Â Â Â Â Â server destroy. > Â Â Â Â Â Â Â Â serverSocketStream close. > Â Â Â Â Â Â Â Â client close. > Â Â Â Â ]. > > Â Â Â Â "11-09 - no prompts" > Â Â Â Â serverSocket := serverSocket. > Â Â Â Â client := client. > Â Â Â Â serverSocketStream := serverSocketStream. > Â Â Â Â thread := thread. > > Â Â Â Â " > Â Â Â Â Â Â Â Â NetworkSmokeTestCase prod:#testLocalServerAndClient. > Â Â Â Â " > > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice(a)gmail.com] > Sent: Sunday, October 10, 2010 7:05 AM > To: The general-purpose Squeak developers list > Cc: Pharo Development > Subject: Re: [Pharo-project] [squeak-dev] Xtreams : first embryonary port on Â Â Squeak > > 2010/10/10 Levente Uzonyi <leves(a)elte.hu>: >> On Sun, 10 Oct 2010, Nicolas Cellier wrote: >> >>> >>> THE STATUS OF TESTS: >>> >>> Tests do not all pass. There seems to be a bug in Squeak >>> #replace:from:to:with: when the replacement is the collection itself, >>> moved to the right (this is with a COG VM). >> >> #replace:from:to:with: is not supposed to work for moves. IIRC the primitive >> version uses memcpy (or strncpy), so moving to the left works just because >> the undelying C function supports it. I used to (ab)use this feature though. >> >> >> Levente >> > > OK, since we use memcpy rather than memmove in Squeak VM, I > implemented the trivial work around - move through a motionBuffer > temporary copy. > > I now have a single test failing, > XTCollectionWriteStream>>#testReadWriteLargeAmount and this must be > due to my #after:do: simplistic implementation. > Messing with #fork in SUnit TestCase is tricky, and I need help on this point. > > Nicolas > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 10, 2010

[Pharo-project] [OT] database choices

by Schwab,Wilhelm K

Just learned about this one: http://mariadb.org/index.html It sounds like the old guard of MySQL is coalescing around it. It's up to you to decide whether that is true, and if so, whether it is a good thing.

Oct. 10, 2010