- Pharo-dev - lists.pharo.org

Re: [Pharo-project] [squeak-dev] SharedQueue does scale (was: Re: SharedQueue doesn't scale)
by StÃ©phane Ducasse Oct. 19, 2010

Oct. 19, 2010

On Oct 18, 2010, at 10:03 PM, Levente Uzonyi wrote: > On Mon, 18 Oct 2010, StÃ©phane Ducasse wrote: > >>> >>> This benchmark is the real concurrent stress test. 100 processes are adding 10,000 elements to the queue while another 100 are reading from it. It clearly shows that Igor's queues are an order of magnitude faster. Also 200 concurrent processes cause much less slowdown compared to the sequential tests for them. >>> >>> So, even though SharedQueue is now faster than SharedQueue2, both will have to go IMHO. :) >> >> Hi Levente >> >> Naively why? > > Did you see the benchmarks? > >> and replace by what? > > By Igor's new collections. Though the name should be kept for compatibility reasons. Ok I was confused... too many emails and threads. So yes! Stef

1 0

Re: [Pharo-project] Speeding up Pharo 1.1
by StÃ©phane Ducasse Oct. 19, 2010

Oct. 19, 2010

Thanks I added that to our agenda! December and january will be busy. We will have fun! Stef On Oct 19, 2010, at 9:24 AM, Henrik Sperre Johansen wrote: > On 19.10.2010 09:17, StÃ©phane Ducasse wrote: >> How can we make progress on this issue? >> >> Stef > As far as I can tell (again, I haven't actually experienced this slowdown), any of: > - Port Levente's WeakKeyDictionary, which is much faster. > - Wait for new VM's supporting Igors finalization scheme, which should be much faster (haven't tested it personally) > - Rewrite the MC package caching to not use weak collections/ be smarter about what it keeps. > > Cheers, > Henry > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

1 0

Re: [Pharo-project] Stack should be reimplemented with Array
by StÃ©phane Ducasse Oct. 19, 2010

Oct. 19, 2010

On Oct 19, 2010, at 1:23 AM, Igor Stasenko wrote: > On 19 October 2010 00:25, Levente Uzonyi <leves(a)elte.hu> wrote: >> On Mon, 18 Oct 2010, Igor Stasenko wrote: >> >>> Stephane, i could say more: >>> - i don't like how LinkedList implemented. >>> >>> I don't see why it should mimic things like #at: #at:put: at all.. >> >> It's not mimicing those methods. A list usually support things like this, >> but the user should know the consequences. >> > > The consequences of such approach is much more far-reaching: leads to > bad design practices, crappy application performance, > and then tons of bugs and workarounds. > >>> IMO this protocol should be pruned from it, to not provoke uses which >>> completely do not fit for given data structure. >> >> I'm not sure if it's okay to remove features, because users lacking really >> basic CS knowledge may use them the wrong way. >> > > Imo, we should prohibit this from the beginning. Standard, core > classes should not contain an API, which could lead to > careless, abusive programming techniques. Yes and we should have comments comments comments and comments. > IMO, a kernel APIs should serve not only as an implementation of basic > system funcionality, it also must serve as guide, > how to best use these facilities, so people will learn what is the > right way to use it. > > We should teach users to use right tools for things they need. > > >>> >>> Removing/inserting into the middle of list is quite ineffective >>> operation (O(n)), >> >> As long as you don't give away the link objects, it's O(n), otherwise it can >> be O(1). > > giving away link objects... oh, that's the worst thing, which could > possibly happen :) > >> >>> while inserting at the begginning/end of list is O(1). >>> >>> Lists are sequenceable.. but sequenceable ~~ indexable. Period. >> >> Sequenceable is indexable, but good performance is not guaranteed. >> > Unless you representing an infinite collection(s). > Streams are good example of sequenceable, however non-indexable containers. > > >> >> Levente >> >>> >>> -- >>> Best regards, >>> Igor Stasenko AKA sig. >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

1 0

Re: [Pharo-project] Stack should be reimplemented with Array
by StÃ©phane Ducasse Oct. 19, 2010

Oct. 19, 2010

one point at a time :) I'm talking about stack :) On Oct 18, 2010, at 10:49 PM, Igor Stasenko wrote: > Stephane, i could say more: > - i don't like how LinkedList implemented. > > I don't see why it should mimic things like #at: #at:put: at all.. > IMO this protocol should be pruned from it, to not provoke uses which > completely do not fit for given data structure. > > Removing/inserting into the middle of list is quite ineffective > operation (O(n)), > while inserting at the begginning/end of list is O(1). > > Lists are sequenceable.. but sequenceable ~~ indexable. Period. > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

1 0

[Pharo-project] Little change for Objects As Methods
by Mariano Martinez Peck Oct. 19, 2010

Oct. 19, 2010

When you are using Objects as methods, MethodWrappers, or whatever with run:with:in, you have to implement not only that method, but also flushCache doing nothing because it is used in: MethodDictionary >> at: put: at: key put: value "Set the value at key to be value." | index | index := self findElementOrNil: key. (self basicAt: index) == nil ifTrue: [tally := tally + 1. self basicAt: index put: key] ifFalse: [(array at: index) flushCache]. array at: index put: value. self fullCheck. ^ value Ok, I can implement that in my classes, and in MethodWrappers, but I think it would be nicer and cleanly if you only have to implement run:with:in: since flushChache doesn't make sense for normal objects. So, I would like to change that code to at: key put: value "Set the value at key to be value." | index | index := self findElementOrNil: key. (self basicAt: index) == nil ifTrue: [tally := tally + 1. self basicAt: index put: key] ifFalse: [(array at: index) isCompiledMethod ifTrue: [flushCache]]. array at: index put: value. self fullCheck. ^ value Yes, yes, I know the if is ugly, but it gives us a cleaner way to define objects as methods. The other option is to implement flushCache in Object doing nothing but I like it less than this. opinions? thanks mariano

1 0

Re: [Pharo-project] Speeding up Pharo 1.1
by StÃ©phane Ducasse Oct. 19, 2010

Oct. 19, 2010

How can we make progress on this issue? Stef

2 1

Re: [Pharo-project] Speeding up Pharo 1.1
by StÃ©phane Ducasse Oct. 19, 2010

Oct. 19, 2010

On Oct 18, 2010, at 10:19 PM, Nicolas Cellier wrote: > 2010/10/17 Levente Uzonyi <leves(a)elte.hu>: >> On Sat, 16 Oct 2010, Bart Veenstra wrote: >> >>> Hi list, >>> >>> I have been working with Pharo for almost a month now, and I suspect >>> that the performance is degrading fast. UI tasks takes several seconds >>> to react to my keyboard. >> >> That kind of sluggishness is probably related to finalization/gc. Please >> send us the result of the following expression: >> >> (WeakArray classPool at: #FinalizationDependents) >> select: [ :each | each notNil ] >> thenCollect: [ :each | each class -> each size ] >> > > While updating pharo 1.2, after an EndOfCentralDirectory error, I got > a very unresponsive image... can you reproduce it? Which image ? Which vm ? Never heard about this error before > ((WeakArray classPool at: #FinalizationDependents) as: Array) > select: [ :each | each notNil ] > thenCollect: [ :each | each class -> each size] > -> > {(WeakIdentityKeyDictionary->22370). (WeakRegistry->0). > (WeakRegistry->0). (WeakRegistry->0). (WeakRegistry->2)} > > I tried to inspect it 5 minutes ago, but hey, answer in another email... > Ah, OK, cmd+shift+. just worked. I had to interrupt > #finalizationProcess itself finally... > Oh inspect is sorting the keys... unusable. > So what's in the array ? Things like > > (ScriptLoader>>#update12125 "a CompiledMethod(0)")->a > MCMethodDefinition(ScriptLoader>>update12125) > (PragmaCollector>>#release "a CompiledMethod(964689920)")->a > MCMethodDefinition(PragmaCollector>>release) > (Float>>#arcSin "a CompiledMethod(358088704)")->a > MCMethodDefinition(Float>>arcSin) > > I don't understand yet, but I begin to gather clues why updating pharo > goes so badly... Thanks. Any progress in that direction is welcome. May be this is related to change proposed by igor on weak finalization explicit call in MC that may not be fully integrated. Stef

2 1

Oct. 19, 2010

This is neat. Thanks for the information. Stef On Oct 18, 2010, at 10:41 PM, Alexander LazareviÄ wrote: > 2010/10/18 Adrien BARREAU <abarreau.dev(a)live.fr>: >> - If I run a headless Pharo, is it possible to display the graphical >> interface after? (for example, if something raises an exception, I want to >> open the GUI and use the debugger) > > It depends. If your headless Pharo image is running on a unix VM you > could utilize the genius OSProcess/XDisplayControl plugin available > for this VM. Load the appropriate packages from [1] into your Pharo > image and then you can close and reopen the window even on another > machine if it's running X11. For example this > > OSProcess thisOSProcess closeXDisplay. > (Delay forSeconds: 5) wait. > OSProcess thisOSProcess displayOnXServer: ':0.0'. > > will close the Window (headless) and reopen it (headful) again on the > local X server after five seconds. > If you're using Seaside in that image you could trigger window > open/close via Seaside or you could change the default handler for > errors to make the window open on the local display as a default, ... > > Alex > > [1] http://www.squeaksource.com/OSProcess > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

1 0

Oct. 19, 2010

Eliot > There was a post on this list back in August complaining about Stack inheriting from LinkedList. There was some discussion, but apparently no resolution, as Stack still inherits from LinkedList in the Pharo 1.2 Core Image I just downloaded. Rather than reimplementing it to forward to a contained LinkedList, I think it should use an Array internally like OrderedCollection does. An Array-based implementation of Stack would be faster than one based on LinkedList due to the basic stack operations (push, pop and top) being able to rely more on primitives. This assumes that you know roughly how a big a stack you need; if you don't, reallocating the array and copying its contents over and over again will cost you, but this is an exceptional case;-- most stacks are small and when they aren't, the programmer usually has enough information to select a reasonable stack size. > > *No*. Stack and OrderedCollection differ *usefully*. Adding an item to a Stack is always O(1). Adding an item to an OrderedCollection is O(N) because when the array overflows a new one must be allocated and the old objects copied to the new. Why not apply your speedups to OrderedCollection, or is that not possible? If not possible, then come up with a new name and add that. Please /don't/ change Stack. I would really like not having Stack inheriting from LinkedList (I hate subclassing). Now from a VM point of view is there a constraint that Stack as to be a subclass of LList? > > > Enclosed is a fileout of an Array-based Stack implementation. It trounces the LinkedList-based implementation easily, but more interestingly, it performs better than an OrderedCollection when used like a stack: > > Pushing is roughly ~30% faster: > r1 :=[100000 timesRepeat: [ > s := Stack new. > 10 timesRepeat: [s push: #foo]]] timeToRun. > r2 := [100000 timesRepeat: [ > oc := OrderedCollection new. > 10 timesRepeat: [oc addLast: #foo]]] timeToRun. > 100 - ((r1 / r2) asFloat * 100). > > as is pushing + popping: > r3 := [100000 timesRepeat: [ > s := Stack new. > 10 timesRepeat: [s push: #foo]. > 10 timesRepeat: [s pop]]] timeToRun. > r4 := [100000 timesRepeat: [ > oc := OrderedCollection new. > 10 timesRepeat: [oc addLast: #foo]. > 10 timesRepeat: [oc removeLast]]] timeToRun. > 100 - ((r3 / r4) asFloat * 100). > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

4 7

[Pharo-project] SharedQueue doesn't scale
by Igor Stasenko Oct. 19, 2010

Oct. 19, 2010

Hello, just out of interest, i tried to compare the speed of my FIFOQueue implementation and SharedQueue, using Levente's benchmarks for stack: "The linked Stack implementation" (1 to: 5) collect: [ :run | | stack | Smalltalk garbageCollect. stack := FIFOQueue new. { [ 1 to: 1000000 do: [ :each | stack nextPut: each ] ] timeToRun. [ 1 to: 1000000 do: [ :each | stack next ] ] timeToRun } ] #(#(291 69) #(170 65) #(168 66) #(168 65) #(168 65)) Then i changed FIFOQueue to SharedQueue and run it again.. waiting 1 minute.. wait a bit more.. then i came to smoke.. and after returning, it was still running.. i interrupted it, and inspected the queue size.. it was slightly above 300000 items. Of course, SharedQueue usually not used in scenarios, where you need to push such large number of items. So, its just a warning. Btw, here is another comparison (Stack vs thread-safe LIFO queue): (1 to: 5) collect: [ :run | | stack | Smalltalk garbageCollect. stack := Stack new. { [ 1 to: 1000000 do: [ :each | stack push: each ] ] timeToRun. [ 1 to: 1000000 do: [ :each | stack pop ] ] timeToRun } ] Stack: #(#(166 94) #(160 90) #(162 91) #(162 92) #(160 92)) LIFOQueue: #(#(172 250) #(174 248) #(172 250) #(174 252) #(172 250)) Yes, it is slower (mainly for reading). But it is price for being thread safe :) -- Best regards, Igor Stasenko AKA sig.

3 14