July 2009 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] optimizing select:thenCollect:

by Andres Valloud

I'd favor removing such methods. The allocation optimization can be implemented better whenever it's needed, and IME it doesn't happen that often. StÃ©phane Ducasse wrote: > so in that case I would be in favor to remove them because this > does not help and just bloat the image. > > > > On Jul 16, 2009, at 8:21 AM, Lukas Renggli wrote: > > >>> select: selectBlock thenCollect: collectBlock >>> >>> | newCollection | >>> newCollection := self species new. >>> self do: [:each | (selectBlock value: each) ifTrue: >>> [newCollection add: >>> (collectBlock value: each)]]. >>> ^newCollection >>> >> That code would also break Dictionary and Array. >> >> I think the methods #select:thenCollect: and #collect:thenSelect: >> should be removed: >> >> - This is a low level optimization that the compiler (or JIT) should >> optimize, not the developer. >> - 'a select: b thenCollect: c' is longer to write than '(a select: b) >> collect: c'. Furthermore it is not that obvious what it does. I have >> seen several people being confused about it. >> - This is just #select:thenCollect:, what about >> #reject:thenCollect:select:thenInjectInto:? What about all subclasses >> that require a new implementation (Array, Dictionary, ...) of such >> methods. >> >> If people really want to be able to combine enumerator methods for >> higher efficiently the Collection hierarchy should be fixed. Having >> external iterator objects like in C++ and Java is not that bad after >> all: >> >> result := aCollection iterator >> select: [ :e | ... ]; >> collect: [ :e | ... ]; >> contents >> >> The accessors #select:, #reject:, #collect: ... on the iterator object >> would just iteratively create a new iteration block that is lazily >> executed when the result is accessed with #contents, #size or #do:. >> >> Cheers, >> Lukas >> >> -- >> Lukas Renggli >> http://www.lukas-renggli.ch >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > . > >

July 16, 2009

Re: [Pharo-project] optimizing select:thenCollect:

by StÃ©phane Ducasse

Paolo why Iterable as a superclass of stream and collection? Why not a trait? Then I'm wondering why this would be faster. Hi all, I would like to ask your opinion about adding the "Iterable" common superclass of Collection and Stream to Pharo. This is already in GNU Smalltalk and is in my opinion a better alternative to the "lazy collections" idea. Here is how I did it: 1) Well, create the class and make Stream/Collection subclasses of it. 2) Push there from Collection #do: (abstract), #inject:into:, #fold: (if you have it), #do:separatedBy:, #detect:, #detect:ifNone:, #count: (if you have it), #allSatisfy:, #noneSatisfy:, #conform:, #contains: (if you have the last two). 3) Add #nextPutAllOn: nextPutAllOn: aStream "Write all the objects in the receiver to aStream" self do: [ :each | aStream nextPut: each ] 4) Change Stream>>#nextPutAllOn: to use "source nextPutAllOn: self". 5) On top of this I added "filtering" streams to implement #select:, #reject: and #collect:. Here is a clean-room MIT implementation: Stream >> select: block ^FilterStream new stream: source block: block result: true Stream >> reject: block ^FilterStream new stream: source block: block result: false Stream >> collect: block ^FilterStream new stream: source block: block FilterStream >> atEnd atEnd isNil ifTrue: [ [stream atEnd ifTrue: [^atEnd := true]. next := stream next. (block value: next) == result] whileFalse. atEnd := false]. ^atEnd FilterStream >> next self atEnd ifTrue: [^pastEnd]. atEnd := nil. ^next CollectStream >> atEnd ^stream atEnd CollectStream >> next ^block value: stream next On Jul 16, 2009, at 9:57 AM, Paolo Bonzini wrote: > >> If people really want to be able to combine enumerator methods for >> higher efficiently the Collection hierarchy should be fixed. Having >> external iterator objects like in C++ and Java is not that bad after >> all: >> >> result := aCollection iterator >> select: [ :e | ... ]; >> collect: [ :e | ... ]; >> contents > > It's already there and it's called #readStream. :-) It only lacks > #size. > > See http://code.google.com/p/pharo/issues/detail?id=958 for an outline > of the implementation. > > Paolo > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 16, 2009

Re: [Pharo-project] optimizing select:thenCollect:

by StÃ©phane Ducasse

>>> If people really want to be able to combine enumerator methods for >>> higher efficiently the Collection hierarchy should be fixed. Paolo why this would be faster? >>> Having >>> external iterator objects like in C++ and Java is not that bad after >>> all: >>> >>> result := aCollection iterator >>> select: [ :e | ... ]; >>> collect: [ :e | ... ]; >>> contents >> >> It's already there and it's called #readStream. :-) It only lacks >> #size. > > Not in Pharo, but it lacks all iteration methods. > >> See http://code.google.com/p/pharo/issues/detail?id=958 for an >> outline >> of the implementation. > > That would be a welcome addition to Pharo. > > Having an iterator class also avoids having to implement all these > iteration methods over and over again, e.g. in visitors. lukas do you have an example because I do not see why Also with traits you just have to implement do: and you get all the other one for free. > Is this going > to be ANSI? DNU? why ANSI has to be in the story? > > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 16, 2009

Re: [Pharo-project] optimizing select:thenCollect:

by StÃ©phane Ducasse

so in that case I would be in favor to remove them because this does not help and just bloat the image. On Jul 16, 2009, at 8:21 AM, Lukas Renggli wrote: >> select: selectBlock thenCollect: collectBlock >> >> | newCollection | >> newCollection := self species new. >> self do: [:each | (selectBlock value: each) ifTrue: >> [newCollection add: >> (collectBlock value: each)]]. >> ^newCollection > > That code would also break Dictionary and Array. > > I think the methods #select:thenCollect: and #collect:thenSelect: > should be removed: > > - This is a low level optimization that the compiler (or JIT) should > optimize, not the developer. > - 'a select: b thenCollect: c' is longer to write than '(a select: b) > collect: c'. Furthermore it is not that obvious what it does. I have > seen several people being confused about it. > - This is just #select:thenCollect:, what about > #reject:thenCollect:select:thenInjectInto:? What about all subclasses > that require a new implementation (Array, Dictionary, ...) of such > methods. > > If people really want to be able to combine enumerator methods for > higher efficiently the Collection hierarchy should be fixed. Having > external iterator objects like in C++ and Java is not that bad after > all: > > result := aCollection iterator > select: [ :e | ... ]; > collect: [ :e | ... ]; > contents > > The accessors #select:, #reject:, #collect: ... on the iterator object > would just iteratively create a new iteration block that is lazily > executed when the result is accessed with #contents, #size or #do:. > > Cheers, > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 16, 2009

Re: [Pharo-project] What is the story for fasterSets

by StÃ©phane Ducasse

Hi ralph > I am not quite sure what the conclusion was but based on the > comments I suggest that I release a new version of FasterSets > with the following changes: > 1) change method name noCompareAdd: to noCompareOrGrowAdd: > This helps emphasize that it can't be used publicly. > 2) Add methods addNew: and nastyAddNew: > I tried to generate a discussion on these methods but none was > forthcoming. > I am inclined to add them to FasterSets so that anybody who > wants to can > use them. I assume they will NOT be added to Pharo. why? we all want fast set :) > The method noCompareAdd: currently updates the tally value and > it has been > proposed to move this step outside of noCompareAdd: or create a > method > noCompareAddAll: or both. I can do this or someone else can. > If someone else does this then I strongly encourage that that they > release the change to FasterSets first. I will add whoever does > this to the list > of those able to modify FasterSets. > > Note that I want to incorporate FasterSets to any version of > Smalltalk that > would benefit from it so I want it to contain the final code that > Pharo uses. excellent! > Also note that I was recently told that GNU Smalltalk already does > what > FasterSets does. So let me know when you have a new version that we can incorporate. I do not believe in big changes but in avalanches of small incremental ones > > Regards, > > Ralph Boland

July 16, 2009

Re: [Pharo-project] Making some progress, and a few observations

by StÃ©phane Ducasse

Sounds like your machine is 4 times slower than ours. which is good we should all have machine like that to make pharo fast On Jul 16, 2009, at 3:07 AM, Schwab,Wilhelm K wrote: > On my crusty old Linux box: > > 155339805 bytecodes/sec; 3931964 sends/sec > > However, I always like to test on less than my customers will have, > so by some arguments, this machine is just about right. > > Bill > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr > ] On Behalf Of Alain Plantec [alain.plantec(a)free.fr] > Sent: Wednesday, July 15, 2009 6:19 PM > To: Pharo-project(a)lists.gforge.inria.fr > Subject: Re: [Pharo-project] Making some progress, and a few > observations > > Adrian Lienhard a Ã©crit : >> Bill, >> >> Could you post the results of the following benchmark? >> >> 0 tinyBenchmarks >> > >> On my MacBook Pro with 2.4GHz and the Mac 4.1.1beta2U VM I get: >> >> '543236074 bytecodes/sec; 12101592 sends/sec' >> > On Intel Core2 Duo T7700 at 2.4 GHz, Linux Ubuntu 9.04, 64 bit > pharo-core : #10379 '585812356 bytecodes/sec; 14531440 sends/sec' > pharo : #10373 '574312955 bytecodes/sec; 14261923 sends/sec' > > not so bad with linux :) > > Cheers > Alain > >> Cheers, >> Adrian >> >> On Jul 15, 2009, at 22:34 , Schwab,Wilhelm K wrote: >> >> >>> Stef, >>> >>> No browser should be that slow. I suspect the Linux vm is not a >>> speed demon, the xp machines I use are heavily burdened by anit- >>> virus software and other performance drains, etc. My linux box at >>> home might be further hindered by the fact that I bought a wide >>> screen monitor for it, and use LOTS of screen real estate. Still, >>> when I compare Pharo with other software (including Dolphin) under >>> similarly abusive conditions, Pharo is not quick. >>> >>> This particular image is small; it is based on the most recent web >>> image, and is 22 MB. It has some of my Seaside code in it, but has >>> not yet executed same. I have a couple of Seaside cleanup >>> expressions that help when images bloat up, but looking at the >>> current size makes me think I'm missing an opportunity. All that >>> aside, on my linux box, I find that after the saving cursor >>> disappears, it can take a few seconds for the image to again become >>> responsive. On xp in my office, there is at least a second of down >>> time after the cursor reverts to normal. >>> >>> In doing some SIF imports, I noted that it was hard to tell when the >>> image was still busy. I have no idea how long it should take to do >>> the work it was doing, but it took long enough that some gentle >>> prodding becomes irresistable: "is it done yet?" I remember lots of >>> debate about Cursor class>>showWhile: on Dolphin; it might be one of >>> those things that is tricky to get right. >>> >>> A little bit nicer news: I have it running on my office machine >>> (xp), with FreeType in use, and it's ok with the standard tools and >>> the w2k theme. >>> >>> Bill >>> >>> >>> >>> -----Original Message----- >>> From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr >>> ] On Behalf Of StÃ©phane Ducasse >>> Sent: Wednesday, July 15, 2009 2:31 PM >>> To: Pharo-project(a)lists.gforge.inria.fr >>> Subject: Re: [Pharo-project] Making some progress, and a few >>> observations >>> >>> >>>> Stef, >>>> >>>> In terms of response times, even with a lot of things turned off, I >>>> was still surprised at how long it takes to change from even one >>>> method to another. >>>> >>> which browser are you using. >>> Because the old browser should not be that slow. >>> >>> >>> >>>> Anything past 0.2 sec (or whatever that threshold is) starts to add >>>> up; Pharo sometimes takes up to a second, which is _really_ shows >>>> up. >>>> >>>> Saving an image is not necessarily quick either, I _think_ >>>> particuarly >>>> on Linux?? >>>> >>> What is the size of your image: here jannik generated an image of >>> 800 mb. >>> >>> >>>> Also, I get the sense the wait cursors are displayed, on Linux, >>>> for a >>>> smaller fraction of the down time than on Windows. >>>> >>>> Bill >>>> >>>> >>>> -----Original Message----- >>>> From: pharo-project-bounces(a)lists.gforge.inria.fr >>>> [mailto:pharo-project-bounces@lists.gforge.inria.fr >>>> ] On Behalf Of StÃ©phane Ducasse >>>> Sent: Wednesday, July 15, 2009 12:58 PM >>>> To: Pharo-project(a)lists.gforge.inria.fr >>>> Subject: Re: [Pharo-project] Making some progress, and a few >>>> observations >>>> >>>> >>>> On Jul 15, 2009, at 6:04 PM, Schwab,Wilhelm K wrote: >>>> >>>> >>>>> Hello all, >>>>> >>>>> I managed to create an install script; as I suspected would be >>>>> case, >>>>> it was very anticlimactic. I tried creating a password-protected >>>>> directory under my personal web site, but that did not go well. >>>>> The >>>>> authentication is weird, so it was probably asking a lot. A >>>>> directory repository worked. >>>>> >>>> Good to know >>>> >>>>> Does Monticello/PackageInfo see a change in method category (aka >>>>> package) as a change to the package? >>>>> >>>> normally it should. >>>> >>>> >>>>> It appears not, and it worries me a little in that it seems to >>>>> make >>>>> it easy to lose work by forgetting to save it. Do any of you >>>>> script >>>>> saving your packages? >>>>> >>>> you can check in ScriptLoader to see how we compute the packages >>>> that >>>> changed (not only dirty but also new packages) >>>> >>>> >>>>> The results are untested at present, but I used SIF to transfer a >>>>> fair amount of code into Pharo. To cope with the naming of the >>>>> ODBC >>>>> classes, I ended up doing things like Smalltalk at:#DBConnection >>>>> put:ODBCConnection, and that worked out nicely, at least AFAICT at >>>>> this stage. SIF finally ended up complaining about running out of >>>>> items when processing the last file. I reserve the right to later >>>>> report that it was a miserable failure, but it looks like I most >>>>> of >>>>> the code imported. >>>>> >>>>> Using the standard tools, the w2k theme, disabling faded >>>>> backgrounds >>>>> and enabling fast drag, performance on an older Linux system is ok >>>>> (more or less). There are still things that take too long, >>>>> >>>> like what? >>>> >>>> >>>>> and Pharo's responses are no better (sometimes worse) than even >>>>> software running over a remote desktop connection. We need a >>>>> speed >>>>> boost, but judicious settings help a little. >>>>> >>>>> Is there a way to disable the anti-aliased fonts? In fairness, I >>>>> should try turning that off to see if any speed boost is worth the >>>>> price. >>>>> >>>>> Bill >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pharo-project mailing list >>>>> Pharo-project(a)lists.gforge.inria.fr >>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo- >>>>> project >>>>> >>>> _______________________________________________ >>>> Pharo-project mailing list >>>> Pharo-project(a)lists.gforge.inria.fr >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>> >>>> _______________________________________________ >>>> Pharo-project mailing list >>>> Pharo-project(a)lists.gforge.inria.fr >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> >> > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 16, 2009

Re: [Pharo-project] optimizing select:thenCollect:

by Lukas Renggli

>> If people really want to be able to combine enumerator methods for >> higher efficiently the Collection hierarchy should be fixed. Having >> external iterator objects like in C++ and Java is not that bad after >> all: >> >> Â Â result := aCollection iterator >> Â Â Â Â select: [ :e | ... ]; >> Â Â Â Â collect: [ :e | ... ]; >> Â Â Â Â contents > > It's already there and it's called #readStream. :-) Â It only lacks #size. Not in Pharo, but it lacks all iteration methods. > See http://code.google.com/p/pharo/issues/detail?id=958 for an outline > of the implementation. That would be a welcome addition to Pharo. Having an iterator class also avoids having to implement all these iteration methods over and over again, e.g. in visitors. Is this going to be ANSI? Lukas -- Lukas Renggli http://www.lukas-renggli.ch

July 16, 2009

[Pharo-project] Could not we have a group of motivated guys really checking MC1.6

by StÃ©phane Ducasse

Hi guys I spent 3 hours looking for improvement in MC1.5. (of course I got problem when loading the code - but this can be fixed) When I could load it, it is indeed much faster. Finally And matthew nicely pointed me to his improvement. I will have a look at the specific speedup made by matthew and see if I can make that work. Now instead of extracting a speedup from that code (and looking like a thief) I would prefer that we do a task force and check MC1.6 improve it clean it if necessary. So does anybody with knowledge want to help on that topic? Stef Here is my conversation with matthew > Hi matthew > > could you give me some hints abotu what I should look for? > Because I loaded these two files I found on > > http://installer.pbworks.com/LevelPlayingField-Monticello15 > > But this is not that simple to spot changes. I'd rather you just used MC1.5 rather than port it, but ok There are two speedups in Monticello 1.6: Scanning the class heiarchy looking for package elements is 10x faster, due to a rewrite of the scanning code. Loading is 3-6x faster, especially for large packages, due to the atomic loader. I believe you can't make much use of the second, as the atomic loader doesn't support traits yet. The rewritten scanning code is in class PackageInfo, category single-pass. To merge it into Monticello, simply call PackageInfo >> classesDo:methodsDo:displayingProgress: from MCPackage >> snapshot. Attached is a changeset of the relevant code, extracted from mc1.6. You may have to slightly modify MCPackage >> snapshot, as it contains instructions for interacting with the Orphanage, which I'm guessing your branch of Monticello does not include. -- Matthew Fulmer -- http://mtfulmer.wordpress.com/

July 16, 2009

[Pharo-project] Fwd: How to integrate Kernel-Extensions

by StÃ©phane Ducasse

> > > what is the status of the integration of these changes? > > Stef > > On Oct 1, 2008, at 10:53 PM, Damien Pollet wrote: > >> Hi list, we (with Stef) are browsing Kernel-Extensions, here are some >> notes for discussion, as we think there is stuff that should be >> merged >> in Pharo. >> >> Null and Split-Join could be packaged on their own since they are >> well-defined features. >> >> Most of the cleanup methods make sense, but what's the difference >> between cleanUp and freeSomeSpace ? >> >> AppRegistry class >> removeObsolete >> "AppRegistry removeObsoleteClasses" >> self registeredClasses copy do:[:cls| >> (cls class isObsolete or:[cls isBehavior and:[cls isObsolete]]) >> ifTrue:[self unregister: cls]]. >> self subclasses do:[:cls| cls removeObsolete]. >> >> Behavior class >> flushObsoleteSubclasses >> "Behavior flushObsoleteSubclasses" >> ObsoleteSubclasses finalizeValues. >> >> Behavior >> sourceMatchesBytecodeAt: >> >> Class class >> freeSomeSpace >> "Fix some problems with classes. Use Smalltalk allClassesDo: since it >> does not use the subclass list, which we may be modifying during this >> iteration" >> Smalltalk allClassesDo: [:ea | >> ea fixObsoleteSuperclass. >> ea removeDuplicateSubclassEntries] >> >> Collection >> contains: should probably be standardized as part of >> the STEPS project >> Collection >> reject:thenDo: and select:thenDo: should be >> included and similar enumerations reimplemented in this way too >> >> SystemDictionary >> forgetClass: aClass logged: aBool >> badly >> named but interesting >> "Delete the class, aClass, from the system. >> Note that this doesn't do everything required to dispose of a >> class - >> to do that use Class>>removeFromSystem." >> >> aBool ifTrue: [SystemChangeNotifier uniqueInstance classRemoved: >> aClass fromCategory: aClass category]. >> SystemOrganization removeElement: aClass name. >> self removeFromStartUpList: aClass. >> self removeFromShutDownList: aClass. >> self removeKey: aClass name ifAbsent: []. >> DataStream classRemoved: aClass. >> self flushClassNameCache >> >> SystemDictionary >> globals needs to pay attention that compiled >> methods share the associations >> >> Number >> to: stop do: aBlock displayingProgress: aString would >> be >> probably nicer with the block argument last >> >> >> === To discuss: >> >> Collection >> removeAll: comment should explain in which cases >> it is useful >> "the slow way" >> ^ self removeAllSuchThat: [ :each | true ] >> >> SequenceableCollection >> <= smells funnyâ¦ what is it useful for ? >> SequenceableCollection >> replacing:with: replaces in place, not >> obvious from the name >> the stream methods like putOn: â¦ convenience ? are they coherent ? >> >> String >> withoutTrailing: probably makes sense but then this is >> mostly useful for line breaksâ¦ why not a #trimLinebreak >> >> Set >> likeOrAdd: ok but then that should be available for any >> collection (and renamed addIfAbsent:) >> "Include newObject as one of the receiver's elements, but only if >> not already present. Answer the oldObject." >> >> >> === Useless: >> >> ProtoObject >> iconOrThumbnailOfSize: >> Object >> askFor: and askFor:ifDefault: that's a workaround >> >> Object >> deprecated is already here >> Object >> log should be packaged in a logging package since it's a >> convenience method >> >> SequenceableCollection >> copyReplacing:with: just to type one >> space less ? >> >> >> >> -- >> Damien Pollet >> type less, do more [ | ] http://people.untyped.org/damien.pollet >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > >

July 16, 2009

Re: [Pharo-project] optimizing select:thenCollect:

by Paolo Bonzini

> If people really want to be able to combine enumerator methods for > higher efficiently the Collection hierarchy should be fixed. Having > external iterator objects like in C++ and Java is not that bad after > all: > > result := aCollection iterator > select: [ :e | ... ]; > collect: [ :e | ... ]; > contents It's already there and it's called #readStream. :-) It only lacks #size. See http://code.google.com/p/pharo/issues/detail?id=958 for an outline of the implementation. Paolo

July 16, 2009