March 2009 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Fwd: [vwnc] Would you start a new Smalltalk project today?

by Antony Blakey

On 23/03/2009, at 7:37 AM, Michael van der Gulik wrote: > Anybody who relies on the behaviour of the scheduler deserves to be > forced to program in BASIC. Semaphores work; use them. And they work with native threads, which was my point. A more pedantic answer to your response is that you take advantage of the scheduler all the time when considering priorities and atomicity, but I agree with the intent of your statement. > Smalltalk, the language rather than the implementation, already has > fantastic support for concurrency. Smalltalkers just don't know how > to use it. We don't need to rethink the model, we just need to learn > to use what we have. Mutexes and semaphores are hardly 'fantastic'. The state of the art in support for concurrency is way beyond what Smalltalk provides - locking is both primitive and generally not scalable. Antony Blakey ------------- CTO, Linkuistics Pty Ltd Ph: 0438 840 787 There is nothing more difficult to plan, more doubtful of success, nor more dangerous to manage than the creation of a new order of things... Whenever his enemies have the ability to attack the innovator, they do so with the passion of partisans, while the others defend him sluggishly, So that the innovator and his party alike are vulnerable. -- Niccolo Machiavelli, 1513, The Prince.

March 22, 2009

Re: [Pharo-project] Fwd: [vwnc] Would you start a new Smalltalk project today?

by Antony Blakey

On 23/03/2009, at 12:32 AM, StÃ©phane Ducasse wrote: >> Green threads make some underlying implementation details easier, but >> at the language level the difference they make is determined by the >> interleaving guarantees they provide, and the extent to which the >> user >> can take advantage of those guarantees e.g. the yield-point model. >> >> IMO the solution to concurrency is not to simply add native threads - >> it also requires a more functional type of programming, and that can >> be largely provided by immutable data structures and more powerful >> abstractions, such as those provided in Clojure. > > can you illustrate that point? Immutable data makes concurrency easier because you avoid concurrent modification and eliminate synchronisation issues. Clojure adds data types with specific concurrency semantics. Here's some exposition: http://clojure.org/concurrent_programming > Another question I would love to have immutability bit for pharo. > Now if you have immutable collection then I imagine that > - you should copy them when you want to modify them (functional) That happens automatically. Modification operations are functional e.g. f(data) -> data' data' is a different object than data, which remains unchanged. Efficiently implementing this is the key, but it's been done. > - I was wondering if the object that you put such immutable data > structure should > also be immutable Not necessarily - in the case of Clojure you might put a Ref/Var/Atom/ Agent into an immutable structure, and those objects have their own concurrency semantics as described in the link above. Antony Blakey ------------- CTO, Linkuistics Pty Ltd Ph: 0438 840 787 Borrow money from pessimists - they don't expect it back. -- Steven Wright

March 22, 2009

Re: [Pharo-project] Concurrent programming (was: Would you start a new Smalltalk project today?)

by Michael van der Gulik

Argh. Stupid GMail sent it prematurely. On Mon, Mar 23, 2009 at 10:45 AM, Michael van der Gulik <mikevdg(a)gmail.com>wrote: > > If, for example, you were trying to allocate an array of arrays > concurrently, then I would write (using my namespaces :-) ): > > array := Concurrent.Collections.Array new: 1000000000. " We ignore the cost > of making this array to begin with :-) " > array withIndexdo: [ :each :i | > " each will always be nil. " > array at: i put: (Array new: 10). > ]. > > Concurrent.Collections.Array>>withIndexDo: aBlock > | nThreads sema | > nThreads := 1000. " Pull this from a system setting or something. " > sema := Semaphore withSignals: 0-nThreads. " Needs implementing: signal > it -1000 times (or -999 or -1001? Unsure) " > 1 to: nThreads do: [ :i | > [ (nThreads*i) to: (nThreads*i - 1) do: [ :j > aBlock value: (self at: j) value: j ] > sema signal. > ] fork ]. > sema wait. > ^ self. > > Again, it probably won't just run without some bug fixing. And it won't work on non-closure Squeak anyway. Gulik. -- http://gulik.pbwiki.com/

March 22, 2009

[Pharo-project] Concurrent programming (was: Would you start a new Smalltalk project today?)

by Michael van der Gulik

On Mon, Mar 23, 2009 at 12:18 AM, Igor Stasenko <siguctua(a)gmail.com> wrote: > 2009/3/22 Igor Stasenko <siguctua(a)gmail.com>: > > 2009/3/22 Michael van der Gulik <mikevdg(a)gmail.com>: > >> > >> > >> On Sat, Mar 21, 2009 at 8:38 AM, Janko MivÅ¡ek <janko.mivsek(a)eranova.si> > >> wrote: > >>> > >>> Philippe Marschall pravi: > >>> >> Michael van der Gulik wrote: > >>> > >>> >> So now it seems that Gemstone is the only multi-core capable > Smalltalk > >>> >> VM :-(. > >>> > >>> > AFAIK Gemstone isn't multi-core capable as well. You can just run > >>> > multiple gems and they share the same persistent memory. Which is > >>> > similar but different. > >>> > >>> Well, Gemstone can for sure be considered as multi-core capable. Every > >>> gem runs on its own process and therefore can run on its own CPU core. > >>> All gems then share a Shared Memory Cache. So, a typical multi-core > >>> scenario. > >>> > >> By multi-core, I mean that the following code would spread CPU usage > over at > >> least two cores of a CPU or computer for a while: > >> > >> | sum1 sum2 | > >> > >> sum1 := 0. sum2 := 0. > >> > >> [ 1 to: 10000000 do: [ :i | sum1 := sum1 + 1 ] ] fork. > >> > >> [ 1 to: 10000000 do: [ :i | sum2 := sum2 + 1 ] ] fork. > >> > >> (I didn't try the above so there might be obvious bugs) > >> > >> If a VM can't distribute the load for the above over two or more CPU > cores, > >> I consider its multi-core capabilities a hack. No offense intended to > the > >> Hydra VM. > >> > > > > Michael, that's would be too ideal to be true, especially for smalltalk. > > > > Consider the following: > > > > | array sum1 sum2 | > > > > sum1 := 0. sum2 := 0. > > array := Array new: 10. > > > > [ 1 to: 10000000 do: [ :i | array at: (10 random) put: (Array new: 10) ] > ] fork. > > [ 1 to: 10000000 do: [ :i | array at: (10 random) put: (Array new: 10) ] > ] fork. > > 1 to: 10000000 do: [ :i | array at: (10 random) put: (Array new: 10) ]. > > > > This code reveals the following problems: > > - concurrent access to same object > The problem here is that the code above wasn't written with concurrency in mind. I'd be looking at what I'm trying to acheive, and then try to keep each thread's data structures as disentangled as possible. If, for example, you were trying to allocate an array of arrays concurrently, then I would write (using my namespaces :-) ): array := Concurrent.Collections.Array new: 1000000000. " We ignore the cost of making this array to begin with :-) " array withIndexdo: [ :each :i | " each will always be nil. " array at: i put: (Array new: 10). ]. Concurrent.Collections.Array>>withIndexDo: aBlock | nThreads sema | nThreads := 1000. " Pull this from a system setting or something. " sema := Semaphore withSignals: 0-nThreads. " Needs implementing: signal it -1000 times. " 1 to: nThreads do: [ :i | [ (nThreads*i) to: (nThreads*i - 1) do: [ :j aBlock value: (self at: j) value: j ] ] fork ]. ^ self. There are probably bugs, I haven't tried running this. No synchronisation should be needed in this example, making it fast. Currently, this wouldn't work on the Squeak VM because block contexts aren't er... "reentrant". > > - a heavy memory allocation during running 3 processes, which at some > > point should cause GC. > > While first is more or less on the hands of developer (write a proper > > code to avoid such things), but second is a problem that you need to > > solve to be able to collect garbage in real time , when there are > > multiple threads producing it. > Then... invent a better garbage collector. This might be a difficult problem to solve, but it isn't impossible. http://gulik.pbwiki.com/Block-based+virtual+machine (ideas only) > > But what strikes me, that there are a lot of code, which never cares > about it, for instance see > Symbol class>> intern: > and in some magical fashion it works w/o problems in green threading.. > i'm not sure it will continue running when you enable multiple native > threads. I plan to find and fix these at some stage. Firstly, I would change the behaviour of the scheduler to get rid of the predictable yielding behaviour to expose these bugs. Doing so is required for the implementation of "Dominions" in my SecureSqueak project anyway, and I'll try to feed my changes back into the community. I also plan to make the Squeak equivalent of java.util.concurrent at some stage. > > > There is another problem, that squeak processes is cheap (few bytes in > object memory), while allocating new native thread consumes > considerable amount of memory & address space. So, if you map > Processes to native threads, you will lose the ability in having > millions of them, instead you will be limited to thousands. > Answered by Anthony which I agree with. Gulik. -- http://gulik.pbwiki.com/

March 22, 2009

Re: [Pharo-project] Fwd: Issue 668: should clean the option-World click menu

by StÃ©phane Ducasse

On Mar 22, 2009, at 10:17 PM, Rob Rothwell wrote: > Ok...I'll try to figure out what those are. You CAN still create > new Project AND navigate to it, though... yes this is why we should remove the menus because Project will soon explode :) > > > On Sun, Mar 22, 2009 at 5:01 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > > wrote: > we should remove first the obsolete items (related to etoy and > project). > > stef > > On Mar 22, 2009, at 9:24 PM, Rob Rothwell wrote: > > > Ok...it seems to be build in: > > > > Morph>>buildMetaMenu: > > and > > PasteUpMorph>>addCustomMenuItems:hand: > > > > So...any thoughts on what should be in this menu? > > > > Take care, > > > > Rob > > > > 2009/3/21 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> > > probably > > I see > > > > > > > > > > On Mar 21, 2009, at 8:23 PM, Rob Rothwell wrote: > > > > Is this what you are talking about? What would you like instead? > > What was the intent of the option-World click menu? > > > > Rob > > > > 2009/3/20 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> > > > > > > Begin forwarded message: > > > > From: pharo-project-owner(a)lists.gforge.inria.fr > > Date: March 19, 2009 3:13:47 PM CEST > > To: ducasse(a)iam.unibe.ch > > Subject: Issue 668: should clean the option-World click menu > > > > You are not allowed to post to this mailing list, and your message > has > > been automatically rejected. If you think that your messages are > > being rejected in error, contact the mailing list owner at > > pharo-project-owner(a)lists.gforge.inria.fr. > > > > > > > > From: stÃ©phane ducasse <ducasse(a)iam.unibe.ch> > > Date: March 19, 2009 3:13:40 PM CEST > > To: Pharo Development <Pharo-project(a)lists.gforge.inria.fr> > > Subject: Issue 668: should clean the option-World click menu > > > > > > <star_on.gif> Issue 668: > > should clean the option-World click menu > > http://code.google.com/p/pharo/issues/detail?id=668 > > Apparently option-click on the world brings a new menu (which points > > to crappy stuff) and to the new pharo menu. > > > > > > > > _______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > <OldWorldMenu.jpg>_______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > > _______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > _______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

March 22, 2009

[Pharo-project] [Ann] System-PasswordManager

by Keith Hodges

I have published MCPasswordManager as a separate package http://www.squeaksource.com/mc/System-PasswordManager Keith

March 22, 2009

Re: [Pharo-project] Fwd: Issue 668: should clean the option-World click menu

by Rob Rothwell

Ok...I'll try to figure out what those are. You CAN still create new Project AND navigate to it, though... On Sun, Mar 22, 2009 at 5:01 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > wrote: > we should remove first the obsolete items (related to etoy and project). > > stef > > On Mar 22, 2009, at 9:24 PM, Rob Rothwell wrote: > > > Ok...it seems to be build in: > > > > Morph>>buildMetaMenu: > > and > > PasteUpMorph>>addCustomMenuItems:hand: > > > > So...any thoughts on what should be in this menu? > > > > Take care, > > > > Rob > > > > 2009/3/21 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> > > probably > > I see > > > > > > > > > > On Mar 21, 2009, at 8:23 PM, Rob Rothwell wrote: > > > > Is this what you are talking about? What would you like instead? > > What was the intent of the option-World click menu? > > > > Rob > > > > 2009/3/20 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> > > > > > > Begin forwarded message: > > > > From: pharo-project-owner(a)lists.gforge.inria.fr > > Date: March 19, 2009 3:13:47 PM CEST > > To: ducasse(a)iam.unibe.ch > > Subject: Issue 668: should clean the option-World click menu > > > > You are not allowed to post to this mailing list, and your message has > > been automatically rejected. If you think that your messages are > > being rejected in error, contact the mailing list owner at > > pharo-project-owner(a)lists.gforge.inria.fr. > > > > > > > > From: stÃ©phane ducasse <ducasse(a)iam.unibe.ch> > > Date: March 19, 2009 3:13:40 PM CEST > > To: Pharo Development <Pharo-project(a)lists.gforge.inria.fr> > > Subject: Issue 668: should clean the option-World click menu > > > > > > <star_on.gif> Issue 668: > > should clean the option-World click menu > > http://code.google.com/p/pharo/issues/detail?id=668 > > Apparently option-click on the world brings a new menu (which points > > to crappy stuff) and to the new pharo menu. > > > > > > > > _______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > <OldWorldMenu.jpg>_______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > > _______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > _______________________________________________ > > Pharo-project mailing list > > Pharo-project(a)lists.gforge.inria.fr > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

March 22, 2009

Re: [Pharo-project] [ANN] Closures for beta testers

by StÃ©phane Ducasse

run the tests - with the four fixes so far isValid, asMiniRepresentation methodSelector, variablesAndOffsetsDo: 2941 run, 2875 passes, 2 expected failures, 15 failures, 49 errors, 0 unexpected passes vs. in 10259 2929 run, 2908 passes, 2 expected failures, 12 failures, 7 errors, 0 unexpected passes Investigating the tests show that BlockContextTest PCCByCompilationTest ClosureCompilerTest are producing a lot of errors. I can post the two images somewhere if this helps. Stef

March 22, 2009

Re: [Pharo-project] Fwd: [vwnc] Would you start a new Smalltalk project today?

by Michael van der Gulik

On Mon, Mar 23, 2009 at 12:49 AM, Antony Blakey <antony.blakey(a)gmail.com>wrote: > > On 22/03/2009, at 9:48 PM, Igor Stasenko wrote: > > > Sorry, my fault. You need to sync in both cases (even with green > > threads). > > Green threads make some underlying implementation details easier, but > at the language level the difference they make is determined by the > interleaving guarantees they provide, and the extent to which the user > can take advantage of those guarantees e.g. the yield-point model. Anybody who relies on the behaviour of the scheduler deserves to be forced to program in BASIC. Semaphores work; use them. > > > IMO the solution to concurrency is not to simply add native threads - > it also requires a more functional type of programming, and that can > be largely provided by immutable data structures and more powerful > abstractions, such as those provided in Clojure. > > I'm not suggesting adding native threads to Squeak. Squeak is quite a > minefield as it is. I think it's time for a new model of Smalltalk > that includes native threads from the get-go, and is a rethinking of > the Smalltalk tradition, in the same way that Clojure is a rethinking > of the Lisp tradition. The parallels between Lisp and Smalltalk, in > terms of their paradigmatic roles, market issues, and communities, is > quite striking. Why should these two powerhouses of beautiful > programming not *both* be reborn. Smalltalk, the language rather than the implementation, already has fantastic support for concurrency. Smalltalkers just don't know how to use it. We don't need to rethink the model, we just need to learn to use what we have. > > > There is another problem, that squeak processes is cheap (few bytes in > > object memory), while allocating new native thread consumes > > considerable amount of memory & address space. So, if you map > > Processes to native threads, you will lose the ability in having > > millions of them, instead you will be limited to thousands. > > You don't need a 1:1 mapping. You can use a native thread pool to > execute lightweight threads. Erlang manages with huge numbers of > language level processes. > I agree here. Gulik. -- http://gulik.pbwiki.com/

March 22, 2009

Re: [Pharo-project] [ANN] Closures for beta testers

by StÃ©phane Ducasse

Yes I fixed that and asMinimalRepresentation already. Stef On Mar 22, 2009, at 9:57 PM, <bryce(a)kampjes.demon.co.uk> wrote: > Rob Rothwell writes: >> Well, I don't have any "tough, real world examples," but the >> example Bert >> gave me: >> >> multiply := Array new: 4. >> 1 to: 4 do: [:i | >> multiply at: i put: [:x | x * i]. >> ]. >> >> (multiply at: 3) value: 5. >> >> Resulted in the appropriate answer 15!!! > > I tried this on Linux, and it works. > > Poking around a bit more brought up the emergency evaluator because > BlockClosure doesn't understand #isValid:. > > Bryce > > P.S I'm not going to have much time in April as we have > family staying and our second son is due. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

March 22, 2009