October 2010 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Another fixes to finalization (was Re: [update 1.2] #12161 - #12172)

by Igor Stasenko

2010/10/4 Levente Uzonyi <leves(a)elte.hu>: > On Mon, 4 Oct 2010, Igor Stasenko wrote: > >> On 4 October 2010 22:51, Henrik Sperre Johansen > > <henrik.s.johansen(a)veloxit.no> wrote: >> >> Â On 04.10.2010 21:47, Igor Stasenko wrote: >>> >>> On 4 October 2010 22:09, StÃ©phane Ducasse<stephane.ducasse(a)inria.fr> >>> Â wrote: >>>> >>>> so let us know in the bug entry what is the conclusion :) >>>> >>> I think someone should verify my benchmarks i.e. >>> >>> [ self loadsomething ] timeToRun >>> >>> before and after patch. >>> >>> And conclusion is better be written by Henrik, because he's having >>> concerns about speed, >>> while i don't. :) >> >> I already have: >> >> http://code.google.com/p/pharo/issues/detail?id=1628#c13 >> > emm.. wait. > WeakKeyDict should not delete associations with nil-ed keys > automatically, because otherwise > you won't be able to use it in weak finalization scheme. > > There is two distinct use cases of weak-key dicts: > a) for weak finalization > b) for attaching some extra info(value) per object(key), which can be > automatically discarded once object become garbage > > so, while in case (b) you can mercilessly kill/reuse associations with > nil keys, once they discovered > in case (a) you should preserve them until there is explicit request > from outside to finalize values. > > The need in (a), apparently prohibits from using (b) in most efficient > manner. > So, i think, the solution would be to introduce a specialized weak-key > dicts, which can work much better for (b). > > > Squeak's implementation works well in both cases. > > Trust me, scanning a whole dict to detect nil keys is not what can be called 'well' :) My point is, that these two scenarios should use different classes/implementors each optimized for own case. Moreover, case (b) comes from the inability to dynamically add extra slot(s) to object, which you can easily do in Self or JavaScript, but can't do in smalltalk. > Levente > >> Cheers, >> Henry >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > -- -- Best regards, Igor Stasenko AKA sig.

Oct. 4, 2010

Re: [Pharo-project] Another fixes to finalization (was Re: [update 1.2] #12161 - #12172)

by Henrik Sperre Johansen

Sorry for the duplications in last mail, there comes a point where I just think "screw it", stop editing, and just press send :) Cheers, Henry

Oct. 4, 2010

Re: [Pharo-project] Another fixes to finalization (was Re: [update 1.2] #12161 - #12172)

by Henrik Sperre Johansen

On 04.10.2010 22:26, Levente Uzonyi wrote: > On Mon, 4 Oct 2010, Igor Stasenko wrote: > >> On 4 October 2010 22:51, Henrik Sperre Johansen > <henrik.s.johansen(a)veloxit.no> wrote: >> On 04.10.2010 21:47, Igor Stasenko wrote: >>> >>> On 4 October 2010 22:09, StÃ©phane Ducasse<stephane.ducasse(a)inria.fr> >>> wrote: >>>> >>>> so let us know in the bug entry what is the conclusion :) >>>> >>> I think someone should verify my benchmarks i.e. >>> >>> [ self loadsomething ] timeToRun >>> >>> before and after patch. >>> >>> And conclusion is better be written by Henrik, because he's having >>> concerns about speed, >>> while i don't. :) >> >> I already have: >> >> http://code.google.com/p/pharo/issues/detail?id=1628#c13 >> > emm.. wait. > WeakKeyDict should not delete associations with nil-ed keys > automatically, because otherwise > you won't be able to use it in weak finalization scheme. It could in 1.0, without anyone noticing. #rehash truncated duplicate nil keys, thus if it was triggered (f.ex. by a manual removal, or adding enough to cause growth) in a thread with priority higher than finalization process (ie ran after GC but before finalization thread), the nil keys could be truncated before finalization being run. An unlikely scenario, I admit, which I guess is why noone encountered it. #grow suffered the same problem. (well, actually, it didn't add any of the nil keyed associations) > > There is two distinct use cases of weak-key dicts: > a) for weak finalization > b) for attaching some extra info(value) per object(key), which can be > automatically discarded once object become garbage > > so, while in case (b) you can mercilessly kill/reuse associations with > nil keys, once they discovered > in case (a) you should preserve them until there is explicit request > from outside to finalize values. Yes, that's the point I've been trying to make these past couple of months :) > > The need in (a), apparently prohibits from using (b) in most efficient > manner. > So, i think, the solution would be to introduce a specialized weak-key > dicts, which can work much better for (b). The difference between a and b, and the fact Pharo 1.1's implementation severly screws with the performance if b) (and honestly, isn't that good for a) either) is the point I've been trying to make for months now... In Pharo 1.1 we've moved to the other extreme compared to 1.0, nil keys are never removed. b) always works, but a) HAS to be registered to work satisfactory at all. It was changed to do an extra step of keeping finalized assocs in a special state where they can be simply replaced without rehashing after finalization instead of doing rehash, but that's not really the major result of the change. > > > Squeak's implementation works well in both cases. > > > Levente The way I read it (correct me if I'm wrong), the solution in Squeak was adding a finalizer inst var to WeakKeyDictionary, that way you can distinguish the two cases and handle them appropriately. (in addition there were speed improvements for rehashing, growing, etc.) I looked for another solution which would keep WeakKeyDictionary oblivious to whether or not it was being finalized or not and could not find one, thus I think it is worth porting. Keeping the (pardon my fren.. err, norwegian) POS that is currectly in 1.1, or reverting to the 1.0 version for 1.2, is not a good alternative. Cheers, Henry

Oct. 4, 2010

Re: [Pharo-project] Ocean (was Re: Less plugins, more Smalltalk code! (was: Re: Pharo 1.1.1 OneClickCogVm))

by Igor Stasenko

I think that this project would be good test-bed for testing a non-blocking FFI callouts. On 4 October 2010 23:23, Levente Uzonyi <leves(a)elte.hu> wrote: > On Mon, 4 Oct 2010, Sven Van Caekenberghe wrote: > >> Noury, >> >> One of the other aspects that I find important is elementary efficiency. >> >> A lot of stream related code in Smalltalk is not efficient, the actual >> data is copied around like crazy, turning it from a stream to a collection >> and then back into a stream multiple times. For Zinc HTTP Components I made >> a lot of effort to make it possible to read data from a socket stream in >> true streaming fashion (instead of just returning a byte array). Now, the >> idea was then that for example JPEGReadWriter>>#nextImage would work >> transparaently on that raw stream. Sadly, the code in JPEGReadWriter and >> friends just reads everything into an array before it starts to work ! >> >> Doing a #nextPutAll: <some byte array> should really try never to copy the >> array. Similary, a #next: <count> into: <some byte array> should similary >> write directly into the given array. What I see in the current SocketStream >> is <censored/> ;-) > > This is why we implemented our own SocketStream in PostgresV3. > >> >> But OK, maybe this is easier said than done. The step from Smalltalk to >> system call should also not copy if that is possible, but then FFI and GC >> come into play. > > I wonder if there's a language, that provides socket access via FFI. > > > Levente > >> >> Anyway, good luck, I am willing to try using Ocean as soon as it becomes >> available. >> >> Sven >> >> On 04 Oct 2010, at 10:59, Noury Bouraqadi wrote: >> >>> >>> On 3 oct. 2010, at 16:28, Schwab,Wilhelm K wrote: >>> >>>> For Ocean to succeed, you either have to work really hard to make >>>> non-blocking calls (I'm thinking of SSL sockets), or you need the ability to >>>> call functions on OS threads. Â The calls need to block their calling Process >>>> and let the other Processes run as though nothing is happening. >>>> >>> You're right. >>> Another option is having a multi-threaded VM. Eliot seem to work on it. >>> So far we're working on cleaning up the design of basic functionalities >>> before going on. >>> Then, we'll try to find a workaround for non-blocking calls. >>> All ideas are welcome :-) >>> >>> BTW, some people (you included) proposed to help for the Ocean project. >>> Luc and I want to thank you and we are happy that others are willing to >>> help. >>> We thought about the best way to integrate others help. >>> Our conclusion is that we should first do alone (luc and me) the >>> reference design, before integrating other developers. >>> So, we all can work on solid foudations. >>> >>> Noury >>> >>> >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

Oct. 4, 2010

Re: [Pharo-project] Another fixes to finalization (was Re: [update 1.2] #12161 - #12172)

by Levente Uzonyi

On Mon, 4 Oct 2010, Igor Stasenko wrote: > On 4 October 2010 22:51, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > Â On 04.10.2010 21:47, Igor Stasenko wrote: >> >> On 4 October 2010 22:09, StÃ©phane Ducasse<stephane.ducasse(a)inria.fr> >> Â wrote: >>> >>> so let us know in the bug entry what is the conclusion :) >>> >> I think someone should verify my benchmarks i.e. >> >> [ self loadsomething ] timeToRun >> >> before and after patch. >> >> And conclusion is better be written by Henrik, because he's having >> concerns about speed, >> while i don't. :) > > I already have: > > http://code.google.com/p/pharo/issues/detail?id=1628#c13 > emm.. wait. WeakKeyDict should not delete associations with nil-ed keys automatically, because otherwise you won't be able to use it in weak finalization scheme. There is two distinct use cases of weak-key dicts: a) for weak finalization b) for attaching some extra info(value) per object(key), which can be automatically discarded once object become garbage so, while in case (b) you can mercilessly kill/reuse associations with nil keys, once they discovered in case (a) you should preserve them until there is explicit request from outside to finalize values. The need in (a), apparently prohibits from using (b) in most efficient manner. So, i think, the solution would be to introduce a specialized weak-key dicts, which can work much better for (b). Squeak's implementation works well in both cases. Levente > Cheers, > Henry > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 4, 2010

Re: [Pharo-project] Ocean (was Re: Less plugins, more Smalltalk code! (was: Re: Pharo 1.1.1 OneClickCogVm))

by Levente Uzonyi

On Mon, 4 Oct 2010, Sven Van Caekenberghe wrote: > Noury, > > One of the other aspects that I find important is elementary efficiency. > > A lot of stream related code in Smalltalk is not efficient, the actual data is copied around like crazy, turning it from a stream to a collection and then back into a stream multiple times. For Zinc HTTP Components I made a lot of effort to make it possible to read data from a socket stream in true streaming fashion (instead of just returning a byte array). Now, the idea was then that for example JPEGReadWriter>>#nextImage would work transparaently on that raw stream. Sadly, the code in JPEGReadWriter and friends just reads everything into an array before it starts to work ! > > Doing a #nextPutAll: <some byte array> should really try never to copy the array. Similary, a #next: <count> into: <some byte array> should similary write directly into the given array. What I see in the current SocketStream is <censored/> ;-) This is why we implemented our own SocketStream in PostgresV3. > > But OK, maybe this is easier said than done. The step from Smalltalk to system call should also not copy if that is possible, but then FFI and GC come into play. I wonder if there's a language, that provides socket access via FFI. Levente > > Anyway, good luck, I am willing to try using Ocean as soon as it becomes available. > > Sven > > On 04 Oct 2010, at 10:59, Noury Bouraqadi wrote: > >> >> On 3 oct. 2010, at 16:28, Schwab,Wilhelm K wrote: >> >>> For Ocean to succeed, you either have to work really hard to make non-blocking calls (I'm thinking of SSL sockets), or you need the ability to call functions on OS threads. The calls need to block their calling Process and let the other Processes run as though nothing is happening. >>> >> You're right. >> Another option is having a multi-threaded VM. Eliot seem to work on it. >> So far we're working on cleaning up the design of basic functionalities before going on. >> Then, we'll try to find a workaround for non-blocking calls. >> All ideas are welcome :-) >> >> BTW, some people (you included) proposed to help for the Ocean project. >> Luc and I want to thank you and we are happy that others are willing to help. >> We thought about the best way to integrate others help. >> Our conclusion is that we should first do alone (luc and me) the reference design, before integrating other developers. >> So, we all can work on solid foudations. >> >> Noury >> >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 4, 2010

Re: [Pharo-project] Newbie question: Connection to SQLite DB

by nullPointer

Mariano Martinez Peck wrote: > > >> Anything, I need OpenDBX for use Sqlite3 project? ( >> http://squeaksource.com/@7SFD7f008dKbC-Yt/rKzITl0- >> http://squeaksource.com/@7SFD7f008dKbC-Yt/rKzITl0- ) I donÂ´t need use >> glorp for the work; is for a very dummy DB. >> >> > I don't get this links....what did you want to link? > Is a project on SqueakSource for connect to Sqlite DBs. Excuse me for the bad link. http://www.squeaksource.com/SQLite.html That libs work without SqueakDBX, but obviously needs FFI. Works on Windows without problems, but in MACOSX donÂ´t found any table on DB... :( Many thanks for the help. -- View this message in context: http://forum.world.st/Newbie-question-Connection-to-SQLite-DB-tp2954195p295… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Oct. 4, 2010

Re: [Pharo-project] Another fixes to finalization (was Re: [update 1.2] #12161 - #12172)

by Igor Stasenko

On 4 October 2010 22:51, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > Â On 04.10.2010 21:47, Igor Stasenko wrote: >> >> On 4 October 2010 22:09, StÃ©phane Ducasse<stephane.ducasse(a)inria.fr> >> Â wrote: >>> >>> so let us know in the bug entry what is the conclusion :) >>> >> I think someone should verify my benchmarks i.e. >> >> [ self loadsomething ] timeToRun >> >> before and after patch. >> >> And conclusion is better be written by Henrik, because he's having >> concerns about speed, >> while i don't. :) > > I already have: > > http://code.google.com/p/pharo/issues/detail?id=1628#c13 > emm.. wait. WeakKeyDict should not delete associations with nil-ed keys automatically, because otherwise you won't be able to use it in weak finalization scheme. There is two distinct use cases of weak-key dicts: a) for weak finalization b) for attaching some extra info(value) per object(key), which can be automatically discarded once object become garbage so, while in case (b) you can mercilessly kill/reuse associations with nil keys, once they discovered in case (a) you should preserve them until there is explicit request from outside to finalize values. The need in (a), apparently prohibits from using (b) in most efficient manner. So, i think, the solution would be to introduce a specialized weak-key dicts, which can work much better for (b). > Cheers, > Henry > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Best regards, Igor Stasenko AKA sig.

Oct. 4, 2010

Re: [Pharo-project] Another fixes to finalization (was Re: [update 1.2] #12161 - #12172)

by Henrik Sperre Johansen

On 04.10.2010 21:47, Igor Stasenko wrote: > On 4 October 2010 22:09, StÃ©phane Ducasse<stephane.ducasse(a)inria.fr> wrote: >> so let us know in the bug entry what is the conclusion :) >> > I think someone should verify my benchmarks i.e. > > [ self loadsomething ] timeToRun > > before and after patch. > > And conclusion is better be written by Henrik, because he's having > concerns about speed, > while i don't. :) I already have: http://code.google.com/p/pharo/issues/detail?id=1628#c13 Cheers, Henry

Oct. 4, 2010

Re: [Pharo-project] XMLRPC Project - Progress Report

by StÃ©phane Ducasse

:) >> Tell us when pharo core can be a user. >> Because we want to clean the mess. > > I know you are anxious to do so, and it will happen one day, for sure. Not anxious :) Happily eager :) > > There is still some technical work to do, but the most important thing is that we need as much users as possible to shake out the bugs. The code has to prove itself for others (with different encodings/languages/networks). > > Sven > > PS: As I said before, I am using it myself everyday, working in a Pharo 1.1 image were all HTTPSocket stuff is routed to Zn, so far so good. I also replaced all my other HTTP client uses by using Zn directly, and that works well too. We're getting closer. Excellent. I want to have nice code so that I can learn something.

Oct. 4, 2010