- Pharo-dev - lists.pharo.org

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

Levente, one more question: is there a case (small sets?) where LargeIdentitySet is not recommended? my question is, should I ONLY use it for places where I know I can have large sets or should it also work for small sets as well? from my tests it seems to work well also with small ones...but just wondering. Thanks On Fri, Dec 16, 2011 at 8:43 PM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > Wow...... > > Henrik: thank you so much for teaching me :) Indeed, I didn't notice that > in that case I use using a LargePositiveInteger, and indeed I had removed > the basicSize since I also noticed it wouldn't make much sense. Second, > thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our > small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks > for fixing it and commiting a new version. > > All I can say is that I am impressed by the numbers it is really much > faster. > I still don't understand why I send this email with a subject say > IdentitySet because what I really need is a fast/large IdentityDictionary > :( Anyway, there's a place where we can use this LargeIdentitySet in Fuel > I think). > > So Levente, you say this is not possible to adapt this for dictionary? > can we contact Eliot to provide such a primitive? > > thanks > > > On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: > >> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >> >> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>>> >>>> How about my numbers? :) >>>> >>>> "Preallocate objects, so we won't count gc time." >>>> n := 1000000. >>>> objects := Array new: n streamContents: [ :stream | >>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>> >>>> set := IdentitySet new: n. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>> >>>> set := LargeIdentitySet new. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>> >>>> set := (PluggableSet new: n) >>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>> equalBlock: [ :a :b | a == b ]; >>>> yourself. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>> >>>> >>>> I also have a LargeIdentityDictionary, which is relatively fast, but >>>> not as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>> have a primitive that could support it. If we had a primitive like >>>> primitive 132 which would return the index of the element if found or 0 if >>>> not, then we could have a really fast LargeIdentityDictionary. >>>> >>>> >>>> Levente >>>> >>> Hehe yes, if writing a version fully exploiting the limited range, >>> that's probably the approach I would go for as well. >>> (IAssuming it's the version at http://leves.web.elte.hu/** >>> squeak/LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>> ) >>> >>> Mariano commented in the version at http://www.squeaksource.com/** >>> FuelExperiments <http://www.squeaksource.com/FuelExperiments> that it's >>> slow for them, which I guess is due to not adopting #identityHash calls to >>> #basicIdentityHash calls for Pharo: >>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>> >>> Uploaded a new version to the MC repository which is adapted for Pharo, >>> on the same machine my numbers were taken from, it does the same test as I >>> used above in 871 ms. (181 with preallocation). >>> >> >> Cool. One more thing: in Squeak the method using primitive 132 directly >> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >> this was also added to Pharo, then the #pointsTo: sends should be changed >> to #instVarsInclude:, otherwise Array can be reported as included even if >> it wasn't added. >> I'll upload my LargeIdentityDictionary implementation to the same place >> this evening, since it's still 2-3 factor faster than other solutionts and >> there seem to be demand for it. >> >> >> Levente >> >> >>> Cheers, >>> Henry >>> >>> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

Dec. 16, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

Wow...... Henrik: thank you so much for teaching me :) Indeed, I didn't notice that in that case I use using a LargePositiveInteger, and indeed I had removed the basicSize since I also noticed it wouldn't make much sense. Second, thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks for fixing it and commiting a new version. All I can say is that I am impressed by the numbers it is really much faster. I still don't understand why I send this email with a subject say IdentitySet because what I really need is a fast/large IdentityDictionary :( Anyway, there's a place where we can use this LargeIdentitySet in Fuel I think). So Levente, you say this is not possible to adapt this for dictionary? can we contact Eliot to provide such a primitive? thanks On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: > On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: > > On 16.12.2011 03:26, Levente Uzonyi wrote: >> >>> >>> How about my numbers? :) >>> >>> "Preallocate objects, so we won't count gc time." >>> n := 1000000. >>> objects := Array new: n streamContents: [ :stream | >>> n timesRepeat: [ stream nextPut: Object new ] ]. >>> >>> set := IdentitySet new: n. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>> >>> set := LargeIdentitySet new. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>> >>> set := (PluggableSet new: n) >>> hashBlock: [ :object | object identityHash * 4096 + object class >>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>> equalBlock: [ :a :b | a == b ]; >>> yourself. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>> >>> >>> I also have a LargeIdentityDictionary, which is relatively fast, but not >>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>> have a primitive that could support it. If we had a primitive like >>> primitive 132 which would return the index of the element if found or 0 if >>> not, then we could have a really fast LargeIdentityDictionary. >>> >>> >>> Levente >>> >> Hehe yes, if writing a version fully exploiting the limited range, that's >> probably the approach I would go for as well. >> (IAssuming it's the version at http://leves.web.elte.hu/** >> squeak/LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >> ) >> >> Mariano commented in the version at http://www.squeaksource.com/** >> FuelExperiments <http://www.squeaksource.com/FuelExperiments> that it's >> slow for them, which I guess is due to not adopting #identityHash calls to >> #basicIdentityHash calls for Pharo: >> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >> So it basically uses 1 bucket instead of 4096... Whoops. :) >> >> Uploaded a new version to the MC repository which is adapted for Pharo, >> on the same machine my numbers were taken from, it does the same test as I >> used above in 871 ms. (181 with preallocation). >> > > Cool. One more thing: in Squeak the method using primitive 132 directly > was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If > this was also added to Pharo, then the #pointsTo: sends should be changed > to #instVarsInclude:, otherwise Array can be reported as included even if > it wasn't added. > I'll upload my LargeIdentityDictionary implementation to the same place > this evening, since it's still 2-3 factor faster than other solutionts and > there seem to be demand for it. > > > Levente > > >> Cheers, >> Henry >> >> >> > -- Mariano http://marianopeck.wordpress.com

Dec. 16, 2011

Re: [Pharo-project] [COTDC] 97 - CRCError

by Sean P. DeNigris

Hello? Hello? Hello? Is there anybody in there? Just nod if you can hear me. Is there anyone at home? http://www.youtube.com/watch?v=ZB1cNkC71vE -- View this message in context: http://forum.world.st/COTDC-97-CRCError-tp4197055p4205923.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Dec. 16, 2011

[Pharo-project] Help with #cleanUpForRelease is broken: MessageNotUnderstood: ByteSymbol>>run:with:in:

by Mariano Martinez Peck

Hi guys. Somethine is very strange. I cannot do ScriptLoader new cleanUpForRelease because I get an error saying MessageNotUnderstood: ByteSymbol>>run:with:in: So....all the info is here: http://code.google.com/p/pharo/issues/detail?id=5005 I checked and it is not related to methodDictionary rehash..... so, do you have in mind what can we be doing in cleanUpForRelease which causes tu put symbols in the value of the association of a MethodDictionary rather than a CompiledMethod ??? Thanks! -- Mariano http://marianopeck.wordpress.com

Dec. 16, 2011

Re: [Pharo-project] Startup Preferences

by Sean P. DeNigris

Benjamin Van Ryseghem-2 wrote > > You are right. It doesn't add a lot, except maybe that you can set an > action do be performed only once per image and not at every launch. > > But may be you can do it also with Metacello :) > If I understand correctly, Metacello would. My primary motivation here is to make sure we squeeze all the juice out of our current tools before adding more. I've been working a lot with Morphic, and I see how hard it is to remove bloat once it's been incorporated into the image. I always go back to two items of the manifesto: * Beauty to learn from * Clean, lean and fast It is imperative that we add as little as possible, especially where a context shift is required. Sean -- View this message in context: http://forum.world.st/Startup-Preferences-tp4129591p4205896.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Dec. 16, 2011

Re: [Pharo-project] Startup Preferences

by Benjamin

On Dec 16, 2011, at 5:41 PM, Sean P. DeNigris wrote: > > Benjamin Van Ryseghem-2 wrote >> >> But sometimes, we want different script for different version. >> > If you mean different Pharo versions, this is part of what Metacello > provides e.g. load only in on #'Pharo1.3.x'. > > > Benjamin Van Ryseghem-2 wrote >> >> When Stef is integrating, paths for scripts are different, and so are >> repos. >> > So each person would have their own configuration (just like they would > presumably have their own startup preference file > > > Benjamin Van Ryseghem-2 wrote >> >> The Metacello approach is network dependent. >> > Metacello is not any more network dependent than a preference file. Each > person could keep the config on their local machine. > > It still seems that the preference file approach adds complexity without > providing any benefit. Of course I may be missing something ;-) So, now that > the Metacello approach is on the table, which is already used successfully > (at least by myself and Mariano), what does the preference file add - what > can you do with the file that can't be done with Metacello? You are right. It doesn't add a lot, except maybe that you can set an action do be performed only once per image and not at every launch. But may be you can do it also with Metacello :) But the thing is, we could use metacello to laod something from your hard drive instead of serializing. The thing is to have an automatic mechanism which can load sth ( a st file, a mc configuration, a fuel objet, idk ) which is shared among images. So yes, why not replaced StartupAtomics by metacello script ? I think the only argument is the "unique execution" thing Ben > > -- > View this message in context: http://forum.world.st/Startup-Preferences-tp4129591p4205366.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >

Dec. 16, 2011

Re: [Pharo-project] NB Bug?

by Schwab,Wilhelm K

I won't go away over functionVoid() vs. functionName(), in part, because I already ran into the problem and fixed it above FFI, so I assume/expect/hope that NB would get (more or less) inserted as an alternative output syntax. That said, I think NB should try to be helpful. From the replies so far, variable args is the real zinger. My hunch is that the default should be to take an empty arg list as just that, with some additional cue to NB that a variable list is in order. The suggestion of noting that a Smalltalk collection has been passed is one way to help that along. Is the ... in int printf(const char *format, ...); valid C syntax? If so, should we support functionName(...)? However, I suspect that one would sooner want to send a message (#hasVariableArguments???) to flag the handful of functions that might be ambiguous. Bill ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Norbert Hartl [norbert(a)hartl.name] Sent: Friday, December 16, 2011 6:39 AM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] NB Bug? Am 16.12.2011 um 11:47 schrieb Igor Stasenko: > On 16 December 2011 11:02, Stefan Marr <smalltalk(a)stefan-marr.de> wrote: >> >> On 16 Dec 2011, at 10:49, Igor Stasenko wrote: >>>> >>> oh please. that's even less obvious usage :) >>> there's a lot of different stuff in C syntax which you can play with. >>> But it doesn't means that you need to routinely support >>> everything in FFI callout specs. >>> >>> as i said, the function signature syntax are close to C syntax, but >>> not strictly C. >> >> Igor, what is your goal with that? >> Have something working, or have something usable? >> >> If your goal is to have something usable, then your direct competitor is LuaJIT2's FFI >> http://luajit.org/ext_ffi.html >> and you should show similar ambitions. (Note specifically the 'cut&past part') >> Otherwise, it will always remain just a nice try, an experiment that never got finished/productized. >> >> My understanding is that your boss got other plans for Pharo... >> > well, there are things which i would like to have, like integrating > with Cog (so a call to generated native code > will be performed directly , instead via primitive), non-blocking and callbacks. > This is much more important to have than subtleties, how to define a > function which takes no arguments by using '()' or '(void)'.. > Kind of 'triangle has three corners' vs 'triangle has three edges' discussion. > Igor, importance lies in the eye of the beholder. You are doing great stuff and you don't want to waste your time with things you consider unnecessary. But you need to decide what is the goal of NativeBoost. If you like to keep the pace and realize cool demos then you shouldn't listen to other people. If you think you like to make it an accepted piece of software that has a lot of users then you need listen to them. People posting in this thread are your potential users. They tell you things the fail over. If this happens more often you'll loose them because they don't want to waste their time with non-working stuff (the term non-working also lies in the eye of the beholder). Basically it is that easy. If you don't care about the (probably stupid) wishes of your users it stays your private pet project. Norbert

Dec. 16, 2011

Re: [Pharo-project] Startup Preferences

by Sean P. DeNigris

Benjamin Van Ryseghem-2 wrote > > But sometimes, we want different script for different version. > If you mean different Pharo versions, this is part of what Metacello provides e.g. load only in on #'Pharo1.3.x'. Benjamin Van Ryseghem-2 wrote > > When Stef is integrating, paths for scripts are different, and so are > repos. > So each person would have their own configuration (just like they would presumably have their own startup preference file Benjamin Van Ryseghem-2 wrote > > The Metacello approach is network dependent. > Metacello is not any more network dependent than a preference file. Each person could keep the config on their local machine. It still seems that the preference file approach adds complexity without providing any benefit. Of course I may be missing something ;-) So, now that the Metacello approach is on the table, which is already used successfully (at least by myself and Mariano), what does the preference file add - what can you do with the file that can't be done with Metacello? -- View this message in context: http://forum.world.st/Startup-Preferences-tp4129591p4205366.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Dec. 16, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: > On 16.12.2011 03:26, Levente Uzonyi wrote: >> >> How about my numbers? :) >> >> "Preallocate objects, so we won't count gc time." >> n := 1000000. >> objects := Array new: n streamContents: [ :stream | >> n timesRepeat: [ stream nextPut: Object new ] ]. >> >> set := IdentitySet new: n. >> Smalltalk garbageCollect. >> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >> >> set := LargeIdentitySet new. >> Smalltalk garbageCollect. >> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >> >> set := (PluggableSet new: n) >> hashBlock: [ :object | object identityHash * 4096 + object class >> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >> equalBlock: [ :a :b | a == b ]; >> yourself. >> Smalltalk garbageCollect. >> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >> >> >> I also have a LargeIdentityDictionary, which is relatively fast, but not as >> fast as LargeIdentitySet, because (for some unknown reason) we don't have a >> primitive that could support it. If we had a primitive like primitive 132 >> which would return the index of the element if found or 0 if not, then we >> could have a really fast LargeIdentityDictionary. >> >> >> Levente > Hehe yes, if writing a version fully exploiting the limited range, that's > probably the approach I would go for as well. > (IAssuming it's the version at > http://leves.web.elte.hu/squeak/LargeIdentitySet.st) > > Mariano commented in the version at > http://www.squeaksource.com/FuelExperiments that it's slow for them, which I > guess is due to not adopting #identityHash calls to #basicIdentityHash calls > for Pharo: > ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 > So it basically uses 1 bucket instead of 4096... Whoops. :) > > Uploaded a new version to the MC repository which is adapted for Pharo, on > the same machine my numbers were taken from, it does the same test as I used > above in 871 ms. (181 with preallocation). Cool. One more thing: in Squeak the method using primitive 132 directly was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If this was also added to Pharo, then the #pointsTo: sends should be changed to #instVarsInclude:, otherwise Array can be reported as included even if it wasn't added. I'll upload my LargeIdentityDictionary implementation to the same place this evening, since it's still 2-3 factor faster than other solutionts and there seem to be demand for it. Levente > > Cheers, > Henry > >

Dec. 16, 2011

Re: [Pharo-project] NB Bug?

by Ben Coman

Dec. 16, 2011