July 2009 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] FasterSets

by Andres Valloud

Keep in mind that growing on addAll: before the duplicates are filtered out may require significantly more memory than necessary. For example, Set new addAll: "all instances of string" I am sure there is an example than illustrates the issue better than the above one. While Set new addAll: "consecutive integers" may run faster, I think doing (Set new: desiredCapacity) addAll: "consecutive integers" may be a better compromise. Andres. StÃ©phane Ducasse wrote: > Thanks this is really good. > We will probably integrate these changes really fast > > On Jul 1, 2009, at 1:03 PM, Henrik Johansen wrote: > > >> I took some time to check this yesterday, my comments on the code >> itself are basically the same as what Lucas posted in the issue >> tracker. >> Works as advertised, but should probably clean out the "gunk" which is >> unused after it's integrated. >> >> Some feedback on possible improvements: >> - What it does, is basically speed up the adds to the new array in a >> grow that might happen Why not take this all to the logical >> conclusion, and instead optimize an addAll: method to be used in grow, >> that assumes elements are not in set (and themselves are not >> duplicates)? F.ex. it could also add to tally just once, instead of >> one increase per element as the noCompareAdd: does. (an add method >> with no tally add, no = object check, and no growCheck). >> >> - The usual addAll: method can be sped up in a similar way, by >> performing one grow operation before adding elements if necessary >> (pessimistically assuming all elements are unique), then use an add >> method which does not check for grow need at all. (an add method with >> tally add and = object check, but no growCheck) >> If desirable, you could also shrink the capacity after adds are >> done, to match the real amount of new elements. >> >> Adding (1 to: 1200000) to a new Set with such an addAll: method took >> 1/2 to 1/3rd the time of the regular addAll: method, even with the >> lower grow cost of FasterSets. (Lower gains if Set is presized, and if >> hash-calculation is bigger part of cost) >> >> Note: If you want to stay ANSI-compatible, one tricky point in such an >> optimization is adding all objects in collection up to a nil element, >> then raising an error. (ie: must produce equivalent results to add: >> performed of each of collections elements in sequence ) >> >> Cheers, >> Henry >> >> >> On Jun 25, 2009, at 8:55 03PM, StÃ©phane Ducasse wrote: >> >> >>> I would really like to assess FasterSets for Pharo1.1 >>> >>> Begin forwarded message: >>> >>> >>>> From: Ralph Boland <rpboland(a)gmail.com> >>>> Date: June 25, 2009 8:30:13 PM CEDT >>>> To: StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> >>>> Subject: Re: about fasterTests >>>> >>>> 2009/6/25 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: >>>> >>>>> we are interest to get better libraries in pharo. >>>>> If you want to join and help making sure that your code in >>>>> integrated in >>>>> pharo1.1 >>>>> please say it and may be join the pharo mailing-list. >>>>> >>>>> Stef >>>>> >>>>> >>>> Yes, I would like my code 'FasterSets' added to Pharo. >>>> I joined the pharo mailing list. >>>> Today I will mail you my MIT license release. >>>> Probably will take a couple of weeks (From Calgary, Canada) to get >>>> there. >>>> Once I am added as a contributer I will release my code to Pharo for >>>> verification. >>>> In the meantime if any Squeakers try out my code and report bugs I >>>> will fix them >>>> before releasing to Pharo. >>>> >>>> Some comments: >>>> >>>> 1) Since FasterSets modifies low level methods in class Set and its >>>> subclasses it >>>> may be incompatible with packages that create subclasses of Set >>>> and then >>>> override low level methods. If someone could send me a list of >>>> package add-ons >>>> to Pharo that do this then I can download them and make any >>>> modifications >>>> necessary for them to run with FasterSets and make these changes >>>> part of my >>>> release. Note that it is no more difficult to subclass from Set >>>> with FasterSets >>>> than without; it is just different and only different if you do >>>> low level things. >>>> >>>> 2) FasterSets uses a method 'noCompareOrGrowAdd:' which adds >>>> an element to a set without doing compares or growing the set. >>>> Needless to say it is a private method. >>>> I also have a public method 'nastyAddNew:' which, like >>>> 'noCompareOrGrowAdd:' adds an element to a set without checking >>>> if the >>>> element is already there but does do a grow if needed. >>>> This is useful (i.e. faster) in situations where you KNOW the >>>> object >>>> you are adding to a set is not there already. >>>> However if the object IS already there then you now have two >>>> of them in your set; i.e. you now have a subtle bug. >>>> I use 'nastyAddNew:' in my code but never made it a part of >>>> FasterSets >>>> because it is questionable whether or not such a method should be >>>> available generally. However, if I can decide I like the >>>> 'nastyAddNew:' >>>> method and am willing to use it despite its risks so can others >>>> so there >>>> is an argument for it being added. >>>> While my expectation is that you don't want 'nastyAddNew:' in >>>> Pharo >>>> I thought you should at least be aware of it. >>>> >>>> Regards, >>>> >>>> Ralph Boland >>>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >>> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > . > >

July 2, 2009

Re: [Pharo-project] why FileDirectory sucks

by John M McIntosh

I think there is a whole day of reading on the squeak list now about backwards forward compatibility if you change the meaning of FileSystem default defaultDirectory and have it return a URI, that would break things, or assumptions etc. Personally I'd think it should be URI defaultFileSystemDirectory and others, temp, document, vm, etc.. Er so FileSystem default defaultDirectory would really do ^URI defaultFileSystemDirectory absolutePath On 1-Jul-09, at 4:01 PM, Stephen Pair wrote: > > > On Wed, Jul 1, 2009 at 5:32 PM, Michael Rueger <m.rueger(a)acm.org> > wrote: > Igor Stasenko wrote: > > > how then i could write a 'FileDirectory default' in terms of URIs? > > 'file://.' asURI ? > > No, you would still write FileDirectory default :-) > > But then you would do something like > > (FileDirectory default uri resolveRelativePath: 'myDir/images') > directory > > I think I would obliterate FileDirectory and write something more > like "FileSystem default defaultDirectory", which would answer a URI. > > - Stephen -- = = = ======================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ========================================================================

July 2, 2009

Re: [Pharo-project] why FileDirectory sucks

by John M McIntosh

On 1-Jul-09, at 4:02 PM, Michael Rueger wrote: > It actually is. New BSD, which is basically MIT with the advertising > clause. > So if we add code from Sophie (other than fixes) we would need to list > the new BSD license as well. > > Michael I must turn the A/C up another notch before talking about licensing issues without rechecking... Grab some ice water, goodness knows we don't need another license thread... -- = = = ======================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ========================================================================

July 2, 2009

Re: [Pharo-project] why FileDirectory sucks

by Michael Rueger

StÃ©phane Ducasse wrote: >> Well actually in Sophie we used URI http://en.wikipedia.org/wiki/URI >> *hint* all that code is MIT > > I know I just did not find time.... > Good I do not know why a moment I thought that the sophie license was > different. It actually is. New BSD, which is basically MIT with the advertising clause. So if we add code from Sophie (other than fixes) we would need to list the new BSD license as well. Michael

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by Stephen Pair

On Wed, Jul 1, 2009 at 5:32 PM, Michael Rueger <m.rueger(a)acm.org> wrote: > Igor Stasenko wrote: > > > how then i could write a 'FileDirectory default' in terms of URIs? > > 'file://.' asURI ? > > No, you would still write FileDirectory default :-) > > But then you would do something like > > (FileDirectory default uri resolveRelativePath: 'myDir/images') directory > I think I would obliterate FileDirectory and write something more like "FileSystem default defaultDirectory", which would answer a URI. - Stephen

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by StÃ©phane Ducasse

Ok I also like some parts of the rio api. Arg too much to do. Stef On Jul 1, 2009, at 11:32 PM, Michael Rueger wrote: > Igor Stasenko wrote: > >> how then i could write a 'FileDirectory default' in terms of URIs? >> 'file://.' asURI ? > > No, you would still write FileDirectory default :-) > > But then you would do something like > > (FileDirectory default uri resolveRelativePath: 'myDir/images') > directory > > I know, doesn't look to elegant, but it's of context you rarely really > work with directories, but rather paths. > > Also take into account the FileLocations package which gives you calls > for userDataURI, tempURI etc, allowing to transparently integrate with > you platform specific paths. > > Michael > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by StÃ©phane Ducasse

> Well actually in Sophie we used URI http://en.wikipedia.org/wiki/URI > *hint* all that code is MIT I know I just did not find time.... Good I do not know why a moment I thought that the sophie license was different. > > By using relative URI then asking for the read or readwrite stream > later we could build URI constructs that referred to disk, to a server > in the cloud, to memory. > > Oh and let's not forget the Squeak file primitive logic is *so* > crippled that asking if a file exists either means opening the file, > or iterating over N files in your > directory to confirm it exists. Again for Sophie we added a FFI call > to see if the operating system would tell us if the file existed. > > So I'd suggest one thinks about reusing the URI logic > > PS we also went to great pains to ensure the URI got encoded correctly > so unicode aware file systems actually would end up with the proper > characters. this is on my radar but it is full. > > > On 1-Jul-09, at 10:28 AM, Stephen Pair wrote: > >> On Wed, Jul 1, 2009 at 9:42 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr >>> wrote: >> Hi guys >> >> we should really consider to have an alternate (may be building on / >> rewriting part of RIO) >> but FileDirectory sucks. >> >> I have a file >> >> FileDirectory default >> >> and now I want to get two levels up. >> >> Well after 20 min looking around I could not find a way. >> I'm probably too stupid for squeak arcane logic. >> >> Smalltalk deserves a better library for file. What a crap this stuff. >> >> Stef >> >> LoL...how many years did it take you to come to this conclusion? >> Anyway, if anyone does take up the job of rewriting the file system >> interface, I have a suggestion based on some recent stuff we've done >> here at work. We wanted a way of running test cases for code that >> uses the file system, but we didn't want to actually use the file >> system (for speed...when you run many thousands of test cases, >> hitting the disk can really slow things down). We wanted to >> substitute an in memory simulation of a file system. We had to go >> to great lengths to accomplish this (in VisualWorks). It would be >> really cool if things like filenames and directories didn't make >> assumptions about the file system with which they are used (so that >> you could have filenames for in memory file systems, or filenames >> for other file systems that one might connect with in ways other >> than the built in local file system primitives). >> >> - Stephen >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > -- > = > = > = > = > = > ====================================================================== > John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: > squeaker68882 > Corporate Smalltalk Consulting Ltd. http:// > www.smalltalkconsulting.com > = > = > = > = > = > ====================================================================== > > > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by StÃ©phane Ducasse

On Jul 1, 2009, at 7:28 PM, Stephen Pair wrote: > On Wed, Jul 1, 2009 at 9:42 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > > wrote: > Hi guys > > we should really consider to have an alternate (may be building on / > rewriting part of RIO) > but FileDirectory sucks. > > I have a file > > FileDirectory default > > and now I want to get two levels up. > > Well after 20 min looking around I could not find a way. > I'm probably too stupid for squeak arcane logic. > > Smalltalk deserves a better library for file. What a crap this stuff. > > Stef > > LoL...how many years did it take you to come to this conclusion? No I knew it. I just got burned regularly. > Anyway, if anyone does take up the job of rewriting the file system > interface, I have a suggestion based on some recent stuff we've done > here at work. We wanted a way of running test cases for code that > uses the file system, but we didn't want to actually use the file > system (for speed...when you run many thousands of test cases, > hitting the disk can really slow things down). We wanted to > substitute an in memory simulation of a file system. We had to go > to great lengths to accomplish this (in VisualWorks). It would be > really cool if things like filenames and directories didn't make > assumptions about the file system with which they are used (so that > you could have filenames for in memory file systems, or filenames > for other file systems that one might connect with in ways other > than the built in local file system primitives). nice idea. > > - Stephen > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by StÃ©phane Ducasse

:) Yes! On Jul 1, 2009, at 7:18 PM, Schwab,Wilhelm K wrote: > Yikes!!!! :) Anybody else for #parent or #up as a helper? > > > > > > -----Original Message----- > From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr > ] On Behalf Of Lukas Renggli > Sent: Wednesday, July 01, 2009 12:08 PM > To: Pharo-project(a)lists.gforge.inria.fr > Subject: Re: [Pharo-project] why FileDirectory sucks > >> Well after 20 min looking around I could not find a way. >> I'm probably too stupid for squeak arcane logic. > > FileDirectory default containingDirectory containingDirectory > > Yes, it took me a long time to find out the first time. > > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by StÃ©phane Ducasse

On Jul 1, 2009, at 7:10 PM, Lukas Renggli wrote: >> Smalltalk deserves a better library for file. What a crap this stuff. > > The problem is that a good platform independent library is not a > trivial thing. Also from the architectural point of view this is a > rather difficult problem. The biggest problem is though that the > primitives where everything is built upon are arcane too. Yes. I know. Thanks for the containerDirectory. I ended up cutting string .... grgr > > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 1, 2009