October 2009 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Thread-safe collections

by Igor Stasenko

2009/10/23 Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>: > > Sig, > > What about things like managing connections to remote hosts? Â Do I have a socket going over there? Â No?? Â Let's make one. Â Sounds like a great job for a shared dictionary. > it depends on your model, what you need to manage and how. If your goal is to communicate with remote hosts, where are sockets come from is not really relevant to you. Mainly you just getting a reference to object, which provides some facilities to you. How that object manages a collection of sockets, or does it manages them at all is not your concern.. And i wouldn't say that having a thread-safe collections is a big win for managing connections comparing to other possible implementations which not require any thread-safety for collections. > Bill > > > > -----Original Message----- > From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko > Sent: Thursday, October 22, 2009 8:34 PM > To: Pharo-project(a)lists.gforge.inria.fr > Subject: Re: [Pharo-project] Thread-safe collections > > 2009/10/23 Martin McClure <martin(a)hand2mouse.com>: >> >> It's probably possible to write a hashed collection that is >> thread-safe and semaphore-free for both read and write, but unless >> there's a very specific requirement for that it's probably simpler, >> faster, and more reliable to just use a semaphore. >> > +1. > If you traveling into the field of concurrency, you have to change the mindset. > You should pick different abstractions for that. And collections is one which you should leave to grow in single-threaded woods. > Or.. of course.. you can attempt to make them thread-safe.. But it will be clumpsy , rigid and very error prone. > >> Regards, >> >> -Martin >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

Oct. 23, 2009

Re: [Pharo-project] Starting out: db and gui recommendation

by Schwab,Wilhelm K

Rick, I recommend the red pill. FWIW, I am using ODBC with reasonably good results, but need to give SquakDBX a try. There are also some native interfaces lurking around, and they might be worth a look. Polymorph is probably a good choice for a GUI. See TEasilyThemed (a trait used in IIRC ComposableMorph and StandardWindow). There is room for improvement, but it is remarkably powerful once you dig into it. A local browser accessing Seaside is an option. Some have suggested that as an ideal GUI framework, which, well,..., let's say that I disagree - that is not to say that you should not use it. We need a good GUI framework, and Polymorph is leading the pack right now. Bill ________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Rick Morrison Sent: Thursday, October 22, 2009 9:31 PM To: pharo-project(a)lists.gforge.inria.fr Subject: [Pharo-project] Starting out: db and gui recommendation I am a Java & .Net application programmer who has flirted with Smalltalk for a long time without doing the "deep dive" to really understand it. Pharo and Seaside are making me think that everything is coming together now for another try. That and some micro-ISV ambitions to give me a real reason to write some code outside of work. For my type of project I really need a db package and a gui package. That leads to the questions: 1. Right now, is SqeakDBX my best choice for a db interface? Sqlite is my db of choice for the project I have in mind. 2. For the UI, I've been reading about the choices and I'm having a hard time finding a currently supported project. Do I go with polymorph by default? For the UI, what about "self-hosting" a Seaside instance and doing the UI in the browser? Ie, the app starts a Seaside server and opens a browser window pointed at the apps main window (home page). Can that be done? Is it a good idea? I hope you can give me some direction. I like the direction of Pharo and Seaside is just plain cool. I want to do this to get my head outside the coding box I'm kind of stuck in. And have some fun. Thanks for any help, Rick

Oct. 23, 2009

Re: [Pharo-project] Thread-safe collections

by Schwab,Wilhelm K

Sig, What about things like managing connections to remote hosts? Do I have a socket going over there? No?? Let's make one. Sounds like a great job for a shared dictionary. Bill -----Original Message----- From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko Sent: Thursday, October 22, 2009 8:34 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] Thread-safe collections 2009/10/23 Martin McClure <martin(a)hand2mouse.com>: > > It's probably possible to write a hashed collection that is > thread-safe and semaphore-free for both read and write, but unless > there's a very specific requirement for that it's probably simpler, > faster, and more reliable to just use a semaphore. > +1. If you traveling into the field of concurrency, you have to change the mindset. You should pick different abstractions for that. And collections is one which you should leave to grow in single-threaded woods. Or.. of course.. you can attempt to make them thread-safe.. But it will be clumpsy , rigid and very error prone. > Regards, > > -Martin > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 23, 2009

[Pharo-project] Starting out: db and gui recommendation

by Rick Morrison

I am a Java & .Net application programmer who has flirted with Smalltalk for a long time without doing the "deep dive" to really understand it. Pharo and Seaside are making me think that everything is coming together now for another try. That and some micro-ISV ambitions to give me a real reason to write some code outside of work. For my type of project I really need a db package and a gui package. That leads to the questions: 1. Right now, is SqeakDBX my best choice for a db interface? Sqlite is my db of choice for the project I have in mind. 2. For the UI, I've been reading about the choices and I'm having a hard time finding a currently supported project. Do I go with polymorph by default? For the UI, what about "self-hosting" a Seaside instance and doing the UI in the browser? Ie, the app starts a Seaside server and opens a browser window pointed at the apps main window (home page). Can that be done? Is it a good idea? I hope you can give me some direction. I like the direction of Pharo and Seaside is just plain cool. I want to do this to get my head outside the coding box I'm kind of stuck in. And have some fun. Thanks for any help, Rick

Oct. 23, 2009

Re: [Pharo-project] Thread-safe collections

by Schwab,Wilhelm K

Sig, You appear to be assuming that all such access is in (big) loops. That's not always the case; sometimes, a little #at:ifAbsentPut: here or there can conflict in ugly ways, and something that gets the blocking right is non-trivial and very useful. Bill -----Original Message----- From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko Sent: Thursday, October 22, 2009 7:38 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] Thread-safe collections 2009/10/23 Michael van der Gulik <mikevdg(a)gmail.com>: > > > On Fri, Oct 23, 2009 at 11:23 AM, Igor Stasenko <siguctua(a)gmail.com> wrote: >> >> 2009/10/23 Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>: >> > Hello all, >> > >> > I just looked around for thread-safe collections, and found nothing >> > that looks like a shared dictionary. Â The squeak-dev archives >> > contain the obligatory newbie posts (wanting all collections to be >> > thread safe) and the expected replies, some of which are overly dismissive of the idea. >> > Â Something that protects things like #at:ifAbsentPut: and friends >> > is _really_ useful. Â Am I missing an existing implementation? >> > >> >> Imo, there's only one shared collection which is useful - shared >> queue. :) If you need more kinds of thread-safe collections, it >> probably indicating that the concurrent code (or model) which you >> designed is in pretty bad shape. >> >> Just compare two following snippets: >> >> 1 to: 100 do: [:i | Â collection threadSafeAt: i put: something]. >> >> and: >> >> semaphore critical: [ 1 to: 100 do: [:i | Â collection at: i put: >> something] ]. >> >> What you think, which one is better? >> > > A thread-safe collection package could have non-blocking > implementations of most methods. Writing an object reference into an > array should be an atomic operation and I assume most collections use Arrays for their implementation. > well, #basicAt:put: using a primitive, so its already atomic operation, given than VM doesn't running the code in multiple native threads. So, there is no point in changing something for Arrays, and therefore no point to invent a non-blocking.. its already meets our demands. Lets take a look at collections which doing something more sophisticated at #at:put:, like OrderedCollection or Dictionary. What kind of non-blocking we can introduce here? Dictionary may need to rehash the array due to growth.. OrderedCollection may also need to grow its storage (if you using #add: ) and such operations is far from being atomic. So, what kind of non-blocking you can invent here? The only choice is to use blocking, and hence we are getting back to example above.. what is more efficient: 1 to: 100 do: [:i | collection threadSafeAt: i put: something]. or: semaphore critical: [ 1 to: 100 do: [:i | collection at: i put: something] ]. > Gulik. > > -- > http://gulik.pbwiki.com/ > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 23, 2009

Re: [Pharo-project] Thread-safe collections

by Igor Stasenko

2009/10/23 Martin McClure <martin(a)hand2mouse.com>: > > It's probably possible to write a hashed collection that is thread-safe > and semaphore-free for both read and write, but unless there's a very > specific requirement for that it's probably simpler, faster, and more > reliable to just use a semaphore. > +1. If you traveling into the field of concurrency, you have to change the mindset. You should pick different abstractions for that. And collections is one which you should leave to grow in single-threaded woods. Or.. of course.. you can attempt to make them thread-safe.. But it will be clumpsy , rigid and very error prone. > Regards, > > -Martin > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

Oct. 23, 2009

Re: [Pharo-project] Thread-safe collections

by Martin McClure

Michael van der Gulik wrote: > A thread-safe collection package could have non-blocking implementations > of most methods. Writing an object reference into an array should be an > atomic operation and I assume most collections use Arrays for their > implementation. Write to an array is atomic, but the sequence of 1. Look in array to see if entry is used, and if unused 2. Store into array is not atomic, and hashed collections like Set, Dictionary, etc. require this sequence, usually with additional non-atomic operations on write, and sometimes on read. It's difficult to make a collection that is non-blocking and thread-safe for writing, except for collections that are inherently safe, like Array. A few months ago I did implement a hashed collection (essentially a variant of IdentityDictionary) that requires semaphore access for write, but not for read. This increases the code complexity somewhat, and the code fragility quite a bit. One code change by someone who does not understand all of the invariants that must be observed, and it's no longer thread-safe. The collection is believed to be thread-safe, but testing all of the code branches is very difficult, so it has not been thoroughly tested. (Actually, I know of one design bug that I need to fix, but I believe I know how to fix it.) This collection is rather special-purpose; we had a particular requirement for being able to read it even when debugging the process that was writing to it, otherwise I would not have bothered. It's probably possible to write a hashed collection that is thread-safe and semaphore-free for both read and write, but unless there's a very specific requirement for that it's probably simpler, faster, and more reliable to just use a semaphore. Regards, -Martin

Oct. 23, 2009

Re: [Pharo-project] Thread-safe collections

by Michael van der Gulik

On Fri, Oct 23, 2009 at 1:38 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: > 2009/10/23 Michael van der Gulik <mikevdg(a)gmail.com>: > > A thread-safe collection package could have non-blocking implementations > of > > most methods. Writing an object reference into an array should be an > atomic > > operation and I assume most collections use Arrays for their > implementation. > > > well, #basicAt:put: using a primitive, so its already atomic > operation, given than VM doesn't running the code in > multiple native threads. > > So, there is no point in changing something for Arrays, and therefore > no point to invent a non-blocking.. its already meets our demands. > > Lets take a look at collections which doing something more > sophisticated at #at:put:, like OrderedCollection or Dictionary. > What kind of non-blocking we can introduce here? > Dictionary may need to rehash the array due to growth.. > OrderedCollection may also need to grow its storage (if you using > #add: ) and such operations is far from being atomic. So, what kind of > non-blocking you can invent here? > Checkmate; I can't think of any way of implementing a thread-safe growing/shrinking collection using only available atomic operations. There might be a way. I'll sleep on it. Gulik. -- http://gulik.pbwiki.com/

Oct. 23, 2009

Re: [Pharo-project] Thread-safe collections

by Igor Stasenko

2009/10/23 Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>: > Sig, > > One can always construct examples in which protecting the entire loop is the better option. Â That's not the scenario of interest. Â Just as shared queues are useful, shared sets and dictionaries have value for sporadic access scenarios. Â Doing it your way, everyone writes their own incompletely protected (read buggy/dangerous) ad-hoc implementations of these common collections. > > I favor having solid implementations that are complete and as correct as we can get them, leaving people to optimize as you suggest when it makes sense to do so. > > Bill > > Here is what you need. And you have a thread-safe collections for free.. and not only collections, you can use it for anything you want :) 'From Squeak3.10.2 of ''5 June 2008'' [latest update: #7179] on 23 October 2009 at 3:41:44 am'! Object subclass: #SharedResource instanceVariableNames: 'resource lock' classVariableNames: '' poolDictionaries: '' category: 'CorruptVM-tests'! !SharedResource methodsFor: 'as yet unclassified' stamp: 'sig 10/23/2009 03:36'! doesNotUnderstand: aMessage | result | lock critical: [ result := aMessage sentTo: resource ]. ^ result! ! !SharedResource methodsFor: 'as yet unclassified' stamp: 'sig 10/23/2009 03:36'! initialize lock := Semaphore forMutualExclusion ! ! !SharedResource methodsFor: 'as yet unclassified' stamp: 'sig 10/23/2009 03:35'! resource: anObject resource := anObject! ! "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! SharedResource class instanceVariableNames: ''! !SharedResource class methodsFor: 'as yet unclassified' stamp: 'sig 10/23/2009 03:37'! on: anObject ^ self new resource: anObject! ! ------------ i know its a brain-dead way.. but do you really think that you can get something better with implementing collections without ending up with fact, that almost every method will require a semaphore to protect from race condition? :) -- Best regards, Igor Stasenko AKA sig.

Oct. 23, 2009

Re: [Pharo-project] Thread-safe collections

by Igor Stasenko

2009/10/23 Michael van der Gulik <mikevdg(a)gmail.com>: > > > On Fri, Oct 23, 2009 at 11:23 AM, Igor Stasenko <siguctua(a)gmail.com> wrote: >> >> 2009/10/23 Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>: >> > Hello all, >> > >> > I just looked around for thread-safe collections, and found nothing that >> > looks like a shared dictionary. Â The squeak-dev archives contain the >> > obligatory newbie posts (wanting all collections to be thread safe) and the >> > expected replies, some of which are overly dismissive of the idea. >> > Â Something that protects things like #at:ifAbsentPut: and friends is >> > _really_ useful. Â Am I missing an existing implementation? >> > >> >> Imo, there's only one shared collection which is useful - shared queue. :) >> If you need more kinds of thread-safe collections, it probably >> indicating that the concurrent code (or model) which you designed is >> in pretty bad shape. >> >> Just compare two following snippets: >> >> 1 to: 100 do: [:i | Â collection threadSafeAt: i put: something]. >> >> and: >> >> semaphore critical: [ 1 to: 100 do: [:i | Â collection at: i put: >> something] ]. >> >> What you think, which one is better? >> > > A thread-safe collection package could have non-blocking implementations of > most methods. Writing an object reference into an array should be an atomic > operation and I assume most collections use Arrays for their implementation. > well, #basicAt:put: using a primitive, so its already atomic operation, given than VM doesn't running the code in multiple native threads. So, there is no point in changing something for Arrays, and therefore no point to invent a non-blocking.. its already meets our demands. Lets take a look at collections which doing something more sophisticated at #at:put:, like OrderedCollection or Dictionary. What kind of non-blocking we can introduce here? Dictionary may need to rehash the array due to growth.. OrderedCollection may also need to grow its storage (if you using #add: ) and such operations is far from being atomic. So, what kind of non-blocking you can invent here? The only choice is to use blocking, and hence we are getting back to example above.. what is more efficient: 1 to: 100 do: [:i | collection threadSafeAt: i put: something]. or: semaphore critical: [ 1 to: 100 do: [:i | collection at: i put: something] ]. > Gulik. > > -- > http://gulik.pbwiki.com/ > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

Oct. 23, 2009