July 2009 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] FasterSets

by StÃ©phane Ducasse

Thanks this is really good. We will probably integrate these changes really fast On Jul 1, 2009, at 1:03 PM, Henrik Johansen wrote: > I took some time to check this yesterday, my comments on the code > itself are basically the same as what Lucas posted in the issue > tracker. > Works as advertised, but should probably clean out the "gunk" which is > unused after it's integrated. > > Some feedback on possible improvements: > - What it does, is basically speed up the adds to the new array in a > grow that might happen Why not take this all to the logical > conclusion, and instead optimize an addAll: method to be used in grow, > that assumes elements are not in set (and themselves are not > duplicates)? F.ex. it could also add to tally just once, instead of > one increase per element as the noCompareAdd: does. (an add method > with no tally add, no = object check, and no growCheck). > > - The usual addAll: method can be sped up in a similar way, by > performing one grow operation before adding elements if necessary > (pessimistically assuming all elements are unique), then use an add > method which does not check for grow need at all. (an add method with > tally add and = object check, but no growCheck) > If desirable, you could also shrink the capacity after adds are > done, to match the real amount of new elements. > > Adding (1 to: 1200000) to a new Set with such an addAll: method took > 1/2 to 1/3rd the time of the regular addAll: method, even with the > lower grow cost of FasterSets. (Lower gains if Set is presized, and if > hash-calculation is bigger part of cost) > > Note: If you want to stay ANSI-compatible, one tricky point in such an > optimization is adding all objects in collection up to a nil element, > then raising an error. (ie: must produce equivalent results to add: > performed of each of collections elements in sequence ) > > Cheers, > Henry > > > On Jun 25, 2009, at 8:55 03PM, StÃ©phane Ducasse wrote: > >> I would really like to assess FasterSets for Pharo1.1 >> >> Begin forwarded message: >> >>> From: Ralph Boland <rpboland(a)gmail.com> >>> Date: June 25, 2009 8:30:13 PM CEDT >>> To: StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> >>> Subject: Re: about fasterTests >>> >>> 2009/6/25 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: >>>> we are interest to get better libraries in pharo. >>>> If you want to join and help making sure that your code in >>>> integrated in >>>> pharo1.1 >>>> please say it and may be join the pharo mailing-list. >>>> >>>> Stef >>>> >>> >>> Yes, I would like my code 'FasterSets' added to Pharo. >>> I joined the pharo mailing list. >>> Today I will mail you my MIT license release. >>> Probably will take a couple of weeks (From Calgary, Canada) to get >>> there. >>> Once I am added as a contributer I will release my code to Pharo for >>> verification. >>> In the meantime if any Squeakers try out my code and report bugs I >>> will fix them >>> before releasing to Pharo. >>> >>> Some comments: >>> >>> 1) Since FasterSets modifies low level methods in class Set and its >>> subclasses it >>> may be incompatible with packages that create subclasses of Set >>> and then >>> override low level methods. If someone could send me a list of >>> package add-ons >>> to Pharo that do this then I can download them and make any >>> modifications >>> necessary for them to run with FasterSets and make these changes >>> part of my >>> release. Note that it is no more difficult to subclass from Set >>> with FasterSets >>> than without; it is just different and only different if you do >>> low level things. >>> >>> 2) FasterSets uses a method 'noCompareOrGrowAdd:' which adds >>> an element to a set without doing compares or growing the set. >>> Needless to say it is a private method. >>> I also have a public method 'nastyAddNew:' which, like >>> 'noCompareOrGrowAdd:' adds an element to a set without checking >>> if the >>> element is already there but does do a grow if needed. >>> This is useful (i.e. faster) in situations where you KNOW the >>> object >>> you are adding to a set is not there already. >>> However if the object IS already there then you now have two >>> of them in your set; i.e. you now have a subtle bug. >>> I use 'nastyAddNew:' in my code but never made it a part of >>> FasterSets >>> because it is questionable whether or not such a method should be >>> available generally. However, if I can decide I like the >>> 'nastyAddNew:' >>> method and am willing to use it despite its risks so can others >>> so there >>> is an argument for it being added. >>> While my expectation is that you don't want 'nastyAddNew:' in >>> Pharo >>> I thought you should at least be aware of it. >>> >>> Regards, >>> >>> Ralph Boland >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 1, 2009

[Pharo-project] could be worth looking at MCM update mechanism

by StÃ©phane Ducasse

From: Andreas Raab <andreas.raab <at> gmx.de> Subject: MCM update script Newsgroups: gmane.comp.lang.smalltalk.squeak.general Date: 2009-07-01 07:10:22 GMT (15 hours and 14 minutes ago) Folks - I've set up a little test for the suggested MCM update script via Monticello. In order to try it out, do the following: * Get a copy of http://ftp.squeak.org/3.10/Squeak3.10-7159-basic.zip * Fire up Monticello and point it at http://squeaksource.com/MCUpdateTest * Load the latest MonticelloConfigurations package from there and ONLY that package (the fixes included are a strict prerequisite for this to work) Now run the following code: MCMcmUpdater updateFromRepositories: #( 'http://squeaksource.com/MCUpdateTest' ). This will go to the repository and discover that there is update-ar. 1.mcm and update-ar.2.mcm. It will load the first, find that there is nothing to do, move on to the second. The second has actually re- ordered the Exceptions and the Morphic package to allow for loading of a modification enabling SystemProgressMorph to change its label. It then uses the configuration to update all the remaining listed packages (and you can see the effect of the updated label right away). If we'd find that at some point the order needs to be changed to be able to load a particular modification, we would just issue an update.mcm with the new order. Hope this illustrates what I am imagining for an update mechanism using Monticello. Cheers, - Andreas

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by Michael Rueger

Igor Stasenko wrote: > how then i could write a 'FileDirectory default' in terms of URIs? > 'file://.' asURI ? No, you would still write FileDirectory default :-) But then you would do something like (FileDirectory default uri resolveRelativePath: 'myDir/images') directory I know, doesn't look to elegant, but it's of context you rarely really work with directories, but rather paths. Also take into account the FileLocations package which gives you calls for userDataURI, tempURI etc, allowing to transparently integrate with you platform specific paths. Michael

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by Igor Stasenko

2009/7/1 Michael Rueger <m.rueger(a)acm.org>: > Stephen Pair wrote: > >> accomplish this (in VisualWorks). Â It would be really cool if things >> like filenames and directories didn't make assumptions about the file >> system with which they are used (so that you could have filenames for in >> memory file systems, or filenames for other file systems that one might >> connect with in ways other than the built in local file system primitives). > > broken record... ;-) > > ...use URIs :-) > how then i could write a 'FileDirectory default' in terms of URIs? 'file://.' asURI ? > Then send things like stream, writeStream etc to the URI and have the > scheme dispatch resolve the rest. > > Michael > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by Michael Rueger

Stephen Pair wrote: > accomplish this (in VisualWorks). It would be really cool if things > like filenames and directories didn't make assumptions about the file > system with which they are used (so that you could have filenames for in > memory file systems, or filenames for other file systems that one might > connect with in ways other than the built in local file system primitives). broken record... ;-) ...use URIs :-) Then send things like stream, writeStream etc to the URI and have the scheme dispatch resolve the rest. Michael

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by Michael Rueger

Schwab,Wilhelm K wrote: > One thing that seems to be missing is a good abstraction of relative > paths, and some type of #, message to allow relative ones to be > tacked onto full ones (FileDirectory etc.)?? In Sophie and to an even larger extent in an experimental fork I based everything on URIs using resolveRelativePath: where needed. The resolving rules for URIs are well spec'ed and deal with both relative and absolute paths. Platform specific path strings are created only when needed. Michael

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by Stephen Pair

On Wed, Jul 1, 2009 at 2:18 PM, John M McIntosh < johnmci(a)smalltalkconsulting.com> wrote: > > So I'd suggest one thinks about reusing the URI logic > 2009/7/1 Cameron Sanders <camsanders01(a)roadrunner.com> > > And back to the main topic here... aren't URL's fairly well generalized? > Are they not the right way to fully generalize locations? > Yes, I think they are, our code just has a lot of legacy use of Filename (in VW). But I think there are still some pitfalls with URIs...one that I can think if is having a good means of efficiently reusing connections as you interact with a URI and still keep your client code blissfully unaware of where the actual data associated with the URI is coming from. That can get tricky especially as you consider that some connections aren't stateless (like FTP connections). - Stephen

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by John M McIntosh

Well actually in Sophie we used URI http://en.wikipedia.org/wiki/URI *hint* all that code is MIT By using relative URI then asking for the read or readwrite stream later we could build URI constructs that referred to disk, to a server in the cloud, to memory. Oh and let's not forget the Squeak file primitive logic is *so* crippled that asking if a file exists either means opening the file, or iterating over N files in your directory to confirm it exists. Again for Sophie we added a FFI call to see if the operating system would tell us if the file existed. So I'd suggest one thinks about reusing the URI logic PS we also went to great pains to ensure the URI got encoded correctly so unicode aware file systems actually would end up with the proper characters. On 1-Jul-09, at 10:28 AM, Stephen Pair wrote: > On Wed, Jul 1, 2009 at 9:42 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > > wrote: > Hi guys > > we should really consider to have an alternate (may be building on / > rewriting part of RIO) > but FileDirectory sucks. > > I have a file > > FileDirectory default > > and now I want to get two levels up. > > Well after 20 min looking around I could not find a way. > I'm probably too stupid for squeak arcane logic. > > Smalltalk deserves a better library for file. What a crap this stuff. > > Stef > > LoL...how many years did it take you to come to this conclusion? > Anyway, if anyone does take up the job of rewriting the file system > interface, I have a suggestion based on some recent stuff we've done > here at work. We wanted a way of running test cases for code that > uses the file system, but we didn't want to actually use the file > system (for speed...when you run many thousands of test cases, > hitting the disk can really slow things down). We wanted to > substitute an in memory simulation of a file system. We had to go > to great lengths to accomplish this (in VisualWorks). It would be > really cool if things like filenames and directories didn't make > assumptions about the file system with which they are used (so that > you could have filenames for in memory file systems, or filenames > for other file systems that one might connect with in ways other > than the built in local file system primitives). > > - Stephen > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- = = = ======================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ========================================================================

July 1, 2009

Re: [Pharo-project] Problem with fork

by Henrik Sperre Johansen

Also, neither is cascading sends in a thread (like the ts show: xxx; cr in your example)... For things like that, you can use ThreadSafeTranscript with: [ :aStream | ], or if you prefer Transcript protocols, implement a method doing the same but with self as parameter. - This will have some overhead though, seeing as how you'll usually be reentering the critical section once for each call. Something like: TreadSafeTranscript >> doUninterruptably: aBlock accessSemaphore critical: [aBlock value: self] and you'll write: 1 to: 10 do: [:i | [ tr doUninterruptably: [:transcript | transcript show: i printString, ' fork ' ; cr.]. (semaphores at: i) signal. ] fork. Cheers, Henry On 01.07.2009 19:45, Henrik Sperre Johansen wrote: > Look at forMutualExclusion implementation, and you'll see it's set up > with one excess signal. > So the first thing to wait for it, doesn't have to wait, ie. your > semaphores do:[]... falls straight through and 'all forks processed' > gets displayed before any of the forks are run. > Substitute with new, and you SHOULD get the behaviour you expect. > However, show: is NOT threadsafe, it releases access lock between > nextPutAll and endEntry, thus you often end up with several threads > writing to stream, then endEntry displaying them multiple times > If you change nextPutAll: back to show: in the code below, this is > apparent (at least it was on my machine...) > > |semaphores tr| > semaphores := Array new: 10. > tr := ThreadSafeTranscript new. > tr open. > 1 to: 10 do: [ :index | semaphores at: index put: Semaphore new ]. > > 1 to: 10 do: [:i | > [ > tr nextPutAll: i printString, ' fork '; cr. > (semaphores at: i) signal. > ] fork > ]. > > semaphores do: [:each | each wait ]. > tr show: 'all forks proccesed'; cr. > > Cheers, > Henry > > > On 01.07.2009 16:41, Mariano Martinez Peck wrote: >> Hi folks! I am doing some benchmarks and I having problems with fork >> (I think). >> >> I have this method collect: >> >> collect >> |semaphores tr| >> semaphores := Array new: 10. >> tr := ThreadSafeTranscript new. >> tr open. >> 1 to: 10 do: [ :index | semaphores at: index put: Semaphore >> forMutualExclusion ]. >> >> 1 to: 10 do: [:i | >> [ >> tr show: 'one fork'; cr. >> (semaphores at: i) signal. >> ] fork >> ]. >> >> semaphores do: [:each | each wait ]. >> tr show: 'all forks proccesed'; cr. >> >> >> What I want is that the method collect returns when ALL of the forks >> created inside, are finished. Obviously in my real code, I don't do a >> transcript show: but another thing. >> >> I would expect something like this: >> >> one fork >> one fork >> one fork >> one fork >> one fork >> one fork >> one fork >> one fork >> one fork >> one fork >> all forks proccesed >> >> But the output is: >> >> >> all forks proccesed >> one fork >> one fork >> one fork >> one forkone fork >> one fork >> one fork >> one forkone fork >> one fork >> one fork >> one forkone fork >> >> So, I guess it isn't working :( I don't know too much about forks >> so any help is welcome! >> >> can be a problem with ThreadSafeTranscript ? If so, how could I >> test if my code is working as expected ? >> >> Thanks, >> >> Mariano >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > ------------------------------------------------------------------------ > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

July 1, 2009

Re: [Pharo-project] why FileDirectory sucks

by Cameron Sanders

On Jul 1, 2009, at 1:28 PM, Stephen Pair wrote: > On Wed, Jul 1, 2009 at 9:42 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > > wrote: > Hi guys > > we should really consider to have an alternate (may be building on / > rewriting part of RIO) > but FileDirectory sucks. > > I have a file > > FileDirectory default > > and now I want to get two levels up. > > Well after 20 min looking around I could not find a way. > I'm probably too stupid for squeak arcane logic. > > Smalltalk deserves a better library for file. What a crap this stuff. > > Stef > > LoL...how many years did it take you to come to this conclusion? > Anyway, if anyone does take up the job of rewriting the file system > interface, I have a suggestion based on some recent stuff we've done > here at work. We wanted a way of running test cases for code that > uses the file system, but we didn't want to actually use the file > system (for speed...when you run many thousands of test cases, > hitting the disk can really slow things down). We wanted to > substitute an in memory simulation of a file system. We had to go > to great lengths to accomplish this (in VisualWorks). It would be > really cool if things like filenames and directories didn't make > assumptions about the file system with which they are used (so that > you could have filenames for in memory file systems, or filenames > for other file systems that one might connect with in ways other > than the built in local file system primitives). > > - Stephen A related notion, but a tangent (and then returning to the above thread)... One thing I did in Java (in 2001) and Python (later) --and have been considering doing in my current smalltalk code base-- was to create "Accessors" to access relational DB's, directory based files, and memory-based 2d-tables in a generic fashion. An Accessor, as I defined it, looked a lot like a combo of a DB connection and results set. The file-based Accessors were opened on a directory as the DB, where the files were tables (and those could be read or written in several formats: CSV, tab-delimited, etc.), and subdirectories were treated like tables of Blobs; another variant allowed an accessor to be opened on a directory as if it were bound to a specific table, and so the directory looked like a table of Blobs. Obviously, any level of complexity, and "auto" handling of file types is possible. (And I am not describing what I did well, there was symmetry and a recursive nature... and I'm too lazy to look and detail it right now; instead my goal is just to kick some ideas out.) Obviously the Accessors allowed me to ignore the data source. I keep puzzling that I have not encountered this type of thing elsewhere. And back to the main topic here... aren't URL's fairly well generalized? Are they not the right way to fully generalize locations? -cam

July 1, 2009