December 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Old smalltalker is slightly lost

by StÃ©phane Ducasse

pay attention morph position is doomed (global instead of local). Stef On Dec 16, 2011, at 9:16 AM, Kasper Ãsterbye wrote: > Thanks all, > > It was in essence the drawOn: method I had not been able to figure out as the key, and some misunderstandings on the corners specifications of nested morphs. All your hints summarized to me getting going (and of cause the browsing of all the stuff in the image). > > Best, > > Kasper

Dec. 17, 2011

Re: [Pharo-project] Have you played with Ameba?

by HwaJong Oh

Markus, Forget Ameba. Ani is here. Zini das Wuslon is implemented. It moves like a real one. Gofer new squeaksource: 'DaliotsPlayground'; package: 'ConfigurationOfDaliotsPlayground'; load. (Smalltalk at: #ConfigurationOfDaliotsPlayground) project lastVersion load: 'Ani' http://appdal.com/groups/36442/wiki/9eee9/Ani.html Best Regards HwaJong Oh -- View this message in context: http://forum.world.st/Have-you-played-with-Ameba-tp3585624p4207593.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Dec. 17, 2011

Re: [Pharo-project] Class rename on materialization

by Yanni Chiu

On 16/12/11 3:09 PM, Mariano Martinez Peck wrote: > so guys??? Should we only renames Classes/Traits/ClassPools/Methods or > we directly rename all ocurrences of a symbol ? I haven't any use cases to decide. If you want a guess, then I'd say leave the symbols unchanged, because it's simpler to implement. Since there's no data points, at this point, why do work that may prove to be unnecessary.

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Benoit St-Jean

Hi Levente, Have you tried a different value for your array and tallies size instead of 4096 in the code you provided for LargeIdentitySet? Such as prime numbers, as it is usually the case for hash tables? I've tried 3079, and 4093 (prime numbers) and 3079 performs a little bit better .Â Other meaningful suggestions can be found here : http://planetmath.org/encyclopedia/GoodHashTablePrimes.html P.S.Â So far, 3079 is the best candidate saving between 26 and 35% on very large sets !! P.P.S.Â Nice work! ----------------- Benoit St-Jean Yahoo! Messenger: bstjean A standpoint is an intellectual horizon of radius zero. (Albert Einstein) >________________________________ > From: Levente Uzonyi <leves(a)elte.hu> >To: Pharo-project(a)lists.gforge.inria.fr >Sent: Friday, December 16, 2011 6:52:29 PM >Subject: Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ? > >On Fri, 16 Dec 2011, Mariano Martinez Peck wrote: > >> Wow...... >> >> Henrik: thank you so much for teaching me :)Â Indeed, I didn't notice that >> in that case I use using a LargePositiveInteger, and indeed I had removed >> the basicSize since I also noticed it wouldn't make much sense.Â Second, >> thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our >> small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks >> for fixing it and commiting a new version. >> >> All I can say is that I am impressed by the numbers it is really much >> faster. >> I still don't understand why I send this email with a subject say >> IdentitySet because what I really need is a fast/large IdentityDictionary >> :(Â Anyway, there's a place where we can use this LargeIdentitySet in Fuel >> I think). >> >> So Levente, you say this is not possible to adapt this for dictionary?Â can >> we contact Eliot to provide such a primitive? > >As promised, I uploaded my LargeIdentityDictionary implementation to >http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st . >The numbers will be a bit worse compared to LargeIdentitySet, because of >the lack of the primitive, but it's still 2-3x faster than other >solutions (IdentityDictionary, PluggableIdentityDictionary, subclassing, >etc). I'm about to propose this primitive with other improvements on the >vm-dev list. > > >Levente > >> >> thanks >> >> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >>> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>>Â On 16.12.2011 03:26, Levente Uzonyi wrote: >>>> >>>>> >>>>> How about my numbers? :) >>>>> >>>>> "Preallocate objects, so we won't count gc time." >>>>> n := 1000000. >>>>> objects := Array new: n streamContents: [ :stream | >>>>>Â Â n timesRepeat: [ stream nextPut: Object new ] ]. >>>>> >>>>> set := IdentitySet new: n. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>> >>>>> set := LargeIdentitySet new. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>> >>>>> set := (PluggableSet new: n) >>>>>Â Â hashBlock: [ :object | object identityHash * 4096 + object class >>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>>Â Â equalBlock: [ :a :b | a == b ]; >>>>>Â Â yourself. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>> >>>>> >>>>> I also have a LargeIdentityDictionary, which is relatively fast, but not >>>>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>>> have a primitive that could support it. If we had a primitive like >>>>> primitive 132 which would return the index of the element if found or 0 if >>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>> >>>>> >>>>> Levente >>>>> >>>> Hehe yes, if writing a version fully exploiting the limited range, that's >>>> probably the approach I would go for as well. >>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>> squeak/LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>> ) >>>> >>>> Mariano commented in the version at http://www.squeaksource.com/** >>>> FuelExperiments <http://www.squeaksource.com/FuelExperiments> that it's >>>> slow for them, which I guess is due to not adopting #identityHash calls to >>>> #basicIdentityHash calls for Pharo: >>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>> >>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>> on the same machine my numbers were taken from, it does the same test as I >>>> used above in 871 ms. (181 with preallocation). >>>> >>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>>> Cheers, >>>> Henry >>>> >>>> >>>> >>> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> > > > >

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

On Sat, Dec 17, 2011 at 1:25 AM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > > > On Sat, Dec 17, 2011 at 12:52 AM, Levente Uzonyi <leves(a)elte.hu> wrote: > >> On Fri, 16 Dec 2011, Mariano Martinez Peck wrote: >> >> Wow...... >>> >>> Henrik: thank you so much for teaching me :) Indeed, I didn't notice >>> that >>> in that case I use using a LargePositiveInteger, and indeed I had removed >>> the basicSize since I also noticed it wouldn't make much sense. Second, >>> thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our >>> small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks >>> for fixing it and commiting a new version. >>> >>> All I can say is that I am impressed by the numbers it is really much >>> faster. >>> I still don't understand why I send this email with a subject say >>> IdentitySet because what I really need is a fast/large IdentityDictionary >>> :( Anyway, there's a place where we can use this LargeIdentitySet in >>> Fuel >>> I think). >>> >>> So Levente, you say this is not possible to adapt this for dictionary? >>> can >>> we contact Eliot to provide such a primitive? >>> >> >> As promised, I uploaded my LargeIdentityDictionary implementation to >> http://leves.web.elte.hu/**squeak/**LargeIdentityDictionary.st<http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st>. >> The numbers will be a bit worse compared to LargeIdentitySet, because of >> the lack of the primitive, but it's still 2-3x faster than other solutions >> (IdentityDictionary, PluggableIdentityDictionary, subclassing, etc). I'm >> about to propose this primitive with other improvements on the vm-dev list. >> >> > Thanks Levente. Is there something else I should change apart from > #identityHash to #basicIdentityHash for Pharo? > Because I tried to running Fuel tests using an instance of > LargeIdentityDictionary in the place were we usually use a > IdentityDictionary and some tests are failing. It is difficult to reproduce > them or to isolate from Fuel. I will try to figure it out, but in the > meanwhile, maybe you already know something. > I commited the first changes (#basicIdentityHash) in > http://www.squeaksource.com/FuelExperiments and explained in the commit > what should be changed in Fuel. I commented that because magically Herny > fixed for us ;) > > Just for fun I took a Pharo image and in IdentityDictionaryTest I changed the method classToBeTested ^ LargeIdentityDictionary and there are 7 failures and 48 errors while with IdentityDictionary they are all green. So I guess there are some differences. The red ones are easy because I guess it is just that LargeIdentityDictionary does not have the whole protocol but a subset. The problem are the failing one I think because the suggest the usage or API is different? Thanks in advance, > Time to sleep now...thanks! > > > >> >> Levente >> >> >>> thanks >>> >>> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >>> >>> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>>> >>>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>>> >>>>> >>>>> >>>>>> How about my numbers? :) >>>>>> >>>>>> "Preallocate objects, so we won't count gc time." >>>>>> n := 1000000. >>>>>> objects := Array new: n streamContents: [ :stream | >>>>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>>>> >>>>>> set := IdentitySet new: n. >>>>>> Smalltalk garbageCollect. >>>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>>> >>>>>> set := LargeIdentitySet new. >>>>>> Smalltalk garbageCollect. >>>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>>> >>>>>> set := (PluggableSet new: n) >>>>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>>> equalBlock: [ :a :b | a == b ]; >>>>>> yourself. >>>>>> Smalltalk garbageCollect. >>>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>>> >>>>>> >>>>>> I also have a LargeIdentityDictionary, which is relatively fast, but >>>>>> not >>>>>> as fast as LargeIdentitySet, because (for some unknown reason) we >>>>>> don't >>>>>> have a primitive that could support it. If we had a primitive like >>>>>> primitive 132 which would return the index of the element if found or >>>>>> 0 if >>>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>>> >>>>>> >>>>>> Levente >>>>>> >>>>>> Hehe yes, if writing a version fully exploiting the limited range, >>>>> that's >>>>> probably the approach I would go for as well. >>>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>>> squeak/LargeIdentitySet.st<htt**p://leves.web.elte.hu/squeak/** >>>>> LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>>> > >>>>> ) >>>>> >>>>> Mariano commented in the version at http://www.squeaksource.com/** >>>>> FuelExperiments <http://www.squeaksource.com/**FuelExperiments<http://www.squeaksource.com/FuelExperiments>> >>>>> that it's >>>>> >>>>> slow for them, which I guess is due to not adopting #identityHash >>>>> calls to >>>>> #basicIdentityHash calls for Pharo: >>>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>>> >>>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>>> on the same machine my numbers were taken from, it does the same test >>>>> as I >>>>> used above in 871 ms. (181 with preallocation). >>>>> >>>>> >>>> Cool. One more thing: in Squeak the method using primitive 132 directly >>>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. >>>> If >>>> this was also added to Pharo, then the #pointsTo: sends should be >>>> changed >>>> to #instVarsInclude:, otherwise Array can be reported as included even >>>> if >>>> it wasn't added. >>>> I'll upload my LargeIdentityDictionary implementation to the same place >>>> this evening, since it's still 2-3 factor faster than other solutionts >>>> and >>>> there seem to be demand for it. >>>> >>>> >>>> Levente >>>> >>>> >>>> Cheers, >>>>> Henry >>>>> >>>>> >>>>> >>>>> >>>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

BTW...I notice that I cannod do: LargeIdentityDictionary new: 5454 So...it doesn't help if I know before hand the exact number of objects I will put? thanks On Sat, Dec 17, 2011 at 1:25 AM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > > > On Sat, Dec 17, 2011 at 12:52 AM, Levente Uzonyi <leves(a)elte.hu> wrote: > >> On Fri, 16 Dec 2011, Mariano Martinez Peck wrote: >> >> Wow...... >>> >>> Henrik: thank you so much for teaching me :) Indeed, I didn't notice >>> that >>> in that case I use using a LargePositiveInteger, and indeed I had removed >>> the basicSize since I also noticed it wouldn't make much sense. Second, >>> thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our >>> small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks >>> for fixing it and commiting a new version. >>> >>> All I can say is that I am impressed by the numbers it is really much >>> faster. >>> I still don't understand why I send this email with a subject say >>> IdentitySet because what I really need is a fast/large IdentityDictionary >>> :( Anyway, there's a place where we can use this LargeIdentitySet in >>> Fuel >>> I think). >>> >>> So Levente, you say this is not possible to adapt this for dictionary? >>> can >>> we contact Eliot to provide such a primitive? >>> >> >> As promised, I uploaded my LargeIdentityDictionary implementation to >> http://leves.web.elte.hu/**squeak/**LargeIdentityDictionary.st<http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st>. >> The numbers will be a bit worse compared to LargeIdentitySet, because of >> the lack of the primitive, but it's still 2-3x faster than other solutions >> (IdentityDictionary, PluggableIdentityDictionary, subclassing, etc). I'm >> about to propose this primitive with other improvements on the vm-dev list. >> >> > Thanks Levente. Is there something else I should change apart from > #identityHash to #basicIdentityHash for Pharo? > Because I tried to running Fuel tests using an instance of > LargeIdentityDictionary in the place were we usually use a > IdentityDictionary and some tests are failing. It is difficult to reproduce > them or to isolate from Fuel. I will try to figure it out, but in the > meanwhile, maybe you already know something. > I commited the first changes (#basicIdentityHash) in > http://www.squeaksource.com/FuelExperiments and explained in the commit > what should be changed in Fuel. I commented that because magically Herny > fixed for us ;) > > Time to sleep now...thanks! > > > >> >> Levente >> >> >>> thanks >>> >>> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >>> >>> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>>> >>>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>>> >>>>> >>>>> >>>>>> How about my numbers? :) >>>>>> >>>>>> "Preallocate objects, so we won't count gc time." >>>>>> n := 1000000. >>>>>> objects := Array new: n streamContents: [ :stream | >>>>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>>>> >>>>>> set := IdentitySet new: n. >>>>>> Smalltalk garbageCollect. >>>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>>> >>>>>> set := LargeIdentitySet new. >>>>>> Smalltalk garbageCollect. >>>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>>> >>>>>> set := (PluggableSet new: n) >>>>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>>> equalBlock: [ :a :b | a == b ]; >>>>>> yourself. >>>>>> Smalltalk garbageCollect. >>>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>>> >>>>>> >>>>>> I also have a LargeIdentityDictionary, which is relatively fast, but >>>>>> not >>>>>> as fast as LargeIdentitySet, because (for some unknown reason) we >>>>>> don't >>>>>> have a primitive that could support it. If we had a primitive like >>>>>> primitive 132 which would return the index of the element if found or >>>>>> 0 if >>>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>>> >>>>>> >>>>>> Levente >>>>>> >>>>>> Hehe yes, if writing a version fully exploiting the limited range, >>>>> that's >>>>> probably the approach I would go for as well. >>>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>>> squeak/LargeIdentitySet.st<htt**p://leves.web.elte.hu/squeak/** >>>>> LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>>> > >>>>> ) >>>>> >>>>> Mariano commented in the version at http://www.squeaksource.com/** >>>>> FuelExperiments <http://www.squeaksource.com/**FuelExperiments<http://www.squeaksource.com/FuelExperiments>> >>>>> that it's >>>>> >>>>> slow for them, which I guess is due to not adopting #identityHash >>>>> calls to >>>>> #basicIdentityHash calls for Pharo: >>>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>>> >>>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>>> on the same machine my numbers were taken from, it does the same test >>>>> as I >>>>> used above in 871 ms. (181 with preallocation). >>>>> >>>>> >>>> Cool. One more thing: in Squeak the method using primitive 132 directly >>>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. >>>> If >>>> this was also added to Pharo, then the #pointsTo: sends should be >>>> changed >>>> to #instVarsInclude:, otherwise Array can be reported as included even >>>> if >>>> it wasn't added. >>>> I'll upload my LargeIdentityDictionary implementation to the same place >>>> this evening, since it's still 2-3 factor faster than other solutionts >>>> and >>>> there seem to be demand for it. >>>> >>>> >>>> Levente >>>> >>>> >>>> Cheers, >>>>> Henry >>>>> >>>>> >>>>> >>>>> >>>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

On Sat, Dec 17, 2011 at 12:52 AM, Levente Uzonyi <leves(a)elte.hu> wrote: > On Fri, 16 Dec 2011, Mariano Martinez Peck wrote: > > Wow...... >> >> Henrik: thank you so much for teaching me :) Indeed, I didn't notice that >> in that case I use using a LargePositiveInteger, and indeed I had removed >> the basicSize since I also noticed it wouldn't make much sense. Second, >> thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our >> small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks >> for fixing it and commiting a new version. >> >> All I can say is that I am impressed by the numbers it is really much >> faster. >> I still don't understand why I send this email with a subject say >> IdentitySet because what I really need is a fast/large IdentityDictionary >> :( Anyway, there's a place where we can use this LargeIdentitySet in Fuel >> I think). >> >> So Levente, you say this is not possible to adapt this for dictionary? >> can >> we contact Eliot to provide such a primitive? >> > > As promised, I uploaded my LargeIdentityDictionary implementation to > http://leves.web.elte.hu/**squeak/**LargeIdentityDictionary.st<http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st>. > The numbers will be a bit worse compared to LargeIdentitySet, because of > the lack of the primitive, but it's still 2-3x faster than other solutions > (IdentityDictionary, PluggableIdentityDictionary, subclassing, etc). I'm > about to propose this primitive with other improvements on the vm-dev list. > > Thanks Levente. Is there something else I should change apart from #identityHash to #basicIdentityHash for Pharo? Because I tried to running Fuel tests using an instance of LargeIdentityDictionary in the place were we usually use a IdentityDictionary and some tests are failing. It is difficult to reproduce them or to isolate from Fuel. I will try to figure it out, but in the meanwhile, maybe you already know something. I commited the first changes (#basicIdentityHash) in http://www.squeaksource.com/FuelExperiments and explained in the commit what should be changed in Fuel. I commented that because magically Herny fixed for us ;) Time to sleep now...thanks! > > Levente > > >> thanks >> >> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>>> >>>> >>>>> How about my numbers? :) >>>>> >>>>> "Preallocate objects, so we won't count gc time." >>>>> n := 1000000. >>>>> objects := Array new: n streamContents: [ :stream | >>>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>>> >>>>> set := IdentitySet new: n. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>> >>>>> set := LargeIdentitySet new. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>> >>>>> set := (PluggableSet new: n) >>>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>> equalBlock: [ :a :b | a == b ]; >>>>> yourself. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>> >>>>> >>>>> I also have a LargeIdentityDictionary, which is relatively fast, but >>>>> not >>>>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>>> have a primitive that could support it. If we had a primitive like >>>>> primitive 132 which would return the index of the element if found or >>>>> 0 if >>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>> >>>>> >>>>> Levente >>>>> >>>>> Hehe yes, if writing a version fully exploiting the limited range, >>>> that's >>>> probably the approach I would go for as well. >>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>> squeak/LargeIdentitySet.st<htt**p://leves.web.elte.hu/squeak/** >>>> LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>> > >>>> ) >>>> >>>> Mariano commented in the version at http://www.squeaksource.com/** >>>> FuelExperiments <http://www.squeaksource.com/**FuelExperiments<http://www.squeaksource.com/FuelExperiments>> >>>> that it's >>>> >>>> slow for them, which I guess is due to not adopting #identityHash calls >>>> to >>>> #basicIdentityHash calls for Pharo: >>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>> >>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>> on the same machine my numbers were taken from, it does the same test >>>> as I >>>> used above in 871 ms. (181 with preallocation). >>>> >>>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts >>> and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>> Cheers, >>>> Henry >>>> >>>> >>>> >>>> >>> >> >> -- >> Mariano >> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >> >> > -- Mariano http://marianopeck.wordpress.com

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Fri, 16 Dec 2011, Mariano Martinez Peck wrote: > On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: > >> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >> >> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>>> >>>> How about my numbers? :) >>>> >>>> "Preallocate objects, so we won't count gc time." >>>> n := 1000000. >>>> objects := Array new: n streamContents: [ :stream | >>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>> >>>> set := IdentitySet new: n. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>> >>>> set := LargeIdentitySet new. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>> >>>> set := (PluggableSet new: n) >>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>> equalBlock: [ :a :b | a == b ]; >>>> yourself. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>> >>>> >>>> I also have a LargeIdentityDictionary, which is relatively fast, but not >>>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>> have a primitive that could support it. If we had a primitive like >>>> primitive 132 which would return the index of the element if found or 0 if >>>> not, then we could have a really fast LargeIdentityDictionary. >>>> >>>> >>>> Levente >>>> >>> Hehe yes, if writing a version fully exploiting the limited range, that's >>> probably the approach I would go for as well. >>> (IAssuming it's the version at http://leves.web.elte.hu/** >>> squeak/LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>> ) >>> >>> Mariano commented in the version at http://www.squeaksource.com/** >>> FuelExperiments <http://www.squeaksource.com/FuelExperiments> that it's >>> slow for them, which I guess is due to not adopting #identityHash calls to >>> #basicIdentityHash calls for Pharo: >>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>> >>> Uploaded a new version to the MC repository which is adapted for Pharo, >>> on the same machine my numbers were taken from, it does the same test as I >>> used above in 871 ms. (181 with preallocation). >>> >> >> Cool. One more thing: in Squeak the method using primitive 132 directly >> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >> this was also added to Pharo, then the #pointsTo: sends should be changed >> to #instVarsInclude:, otherwise Array can be reported as included even if >> it wasn't added. >> > > In Pharo we have: > > pointsTo: anObject > "This method returns true if self contains a pointer to anObject, > and returns false otherwise" > <primitive: 132> > > So I guess it is correct to let it like this. Right, until you apply the patch. :) > > > >> I'll upload my LargeIdentityDictionary implementation to the same place >> this evening, since it's still 2-3 factor faster than other solutionts and >> there seem to be demand for it. >> >> > I am lost. thought I read something saying you couldn't do that for > Dictionaries because you needed a primitive? Sorry...long day, maybe I am > just crazy ;) > Anyway, I would appreaciate and take a look to LargeIdentityDictionary if > you do it. > BTW, I guess both are MIT ? Yes, there's a license.txt file at the download page. Levente > > Thanks Levente > > > >> >> Levente >> >> >>> Cheers, >>> Henry >>> >>> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 16, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Fri, 16 Dec 2011, Mariano Martinez Peck wrote: > Levente, one more question: is there a case (small sets?) where > LargeIdentitySet is not recommended? my question is, should I ONLY use it > for places where I know I can have large sets or should it also work for > small sets as well? from my tests it seems to work well also with small > ones...but just wondering. One drawback of this implementation is that it allocates an array with 4096 slots even if it's empty. So it has an overhead about 16kB compared to IdentitySet. This number doubles for LargeIdentityDictionary. So if you're using only a few of these collections, then they won't cause any problem. Levente > > Thanks > > On Fri, Dec 16, 2011 at 8:43 PM, Mariano Martinez Peck < > marianopeck(a)gmail.com> wrote: > >> Wow...... >> >> Henrik: thank you so much for teaching me :) Indeed, I didn't notice that >> in that case I use using a LargePositiveInteger, and indeed I had removed >> the basicSize since I also noticed it wouldn't make much sense. Second, >> thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our >> small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks >> for fixing it and commiting a new version. >> >> All I can say is that I am impressed by the numbers it is really much >> faster. >> I still don't understand why I send this email with a subject say >> IdentitySet because what I really need is a fast/large IdentityDictionary >> :( Anyway, there's a place where we can use this LargeIdentitySet in Fuel >> I think). >> >> So Levente, you say this is not possible to adapt this for dictionary? >> can we contact Eliot to provide such a primitive? >> >> thanks >> >> >> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >>> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>>> >>>>> >>>>> How about my numbers? :) >>>>> >>>>> "Preallocate objects, so we won't count gc time." >>>>> n := 1000000. >>>>> objects := Array new: n streamContents: [ :stream | >>>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>>> >>>>> set := IdentitySet new: n. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>> >>>>> set := LargeIdentitySet new. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>> >>>>> set := (PluggableSet new: n) >>>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>> equalBlock: [ :a :b | a == b ]; >>>>> yourself. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>> >>>>> >>>>> I also have a LargeIdentityDictionary, which is relatively fast, but >>>>> not as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>>> have a primitive that could support it. If we had a primitive like >>>>> primitive 132 which would return the index of the element if found or 0 if >>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>> >>>>> >>>>> Levente >>>>> >>>> Hehe yes, if writing a version fully exploiting the limited range, >>>> that's probably the approach I would go for as well. >>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>> squeak/LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>> ) >>>> >>>> Mariano commented in the version at http://www.squeaksource.com/** >>>> FuelExperiments <http://www.squeaksource.com/FuelExperiments> that it's >>>> slow for them, which I guess is due to not adopting #identityHash calls to >>>> #basicIdentityHash calls for Pharo: >>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>> >>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>> on the same machine my numbers were taken from, it does the same test as I >>>> used above in 871 ms. (181 with preallocation). >>>> >>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>>> Cheers, >>>> Henry >>>> >>>> >>>> >>> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 16, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Fri, 16 Dec 2011, Mariano Martinez Peck wrote: > Wow...... > > Henrik: thank you so much for teaching me :) Indeed, I didn't notice that > in that case I use using a LargePositiveInteger, and indeed I had removed > the basicSize since I also noticed it wouldn't make much sense. Second, > thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our > small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks > for fixing it and commiting a new version. > > All I can say is that I am impressed by the numbers it is really much > faster. > I still don't understand why I send this email with a subject say > IdentitySet because what I really need is a fast/large IdentityDictionary > :( Anyway, there's a place where we can use this LargeIdentitySet in Fuel > I think). > > So Levente, you say this is not possible to adapt this for dictionary? can > we contact Eliot to provide such a primitive? As promised, I uploaded my LargeIdentityDictionary implementation to http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st . The numbers will be a bit worse compared to LargeIdentitySet, because of the lack of the primitive, but it's still 2-3x faster than other solutions (IdentityDictionary, PluggableIdentityDictionary, subclassing, etc). I'm about to propose this primitive with other improvements on the vm-dev list. Levente > > thanks > > On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: > >> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >> >> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>>> >>>> How about my numbers? :) >>>> >>>> "Preallocate objects, so we won't count gc time." >>>> n := 1000000. >>>> objects := Array new: n streamContents: [ :stream | >>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>> >>>> set := IdentitySet new: n. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>> >>>> set := LargeIdentitySet new. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>> >>>> set := (PluggableSet new: n) >>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>> equalBlock: [ :a :b | a == b ]; >>>> yourself. >>>> Smalltalk garbageCollect. >>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>> >>>> >>>> I also have a LargeIdentityDictionary, which is relatively fast, but not >>>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>> have a primitive that could support it. If we had a primitive like >>>> primitive 132 which would return the index of the element if found or 0 if >>>> not, then we could have a really fast LargeIdentityDictionary. >>>> >>>> >>>> Levente >>>> >>> Hehe yes, if writing a version fully exploiting the limited range, that's >>> probably the approach I would go for as well. >>> (IAssuming it's the version at http://leves.web.elte.hu/** >>> squeak/LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>> ) >>> >>> Mariano commented in the version at http://www.squeaksource.com/** >>> FuelExperiments <http://www.squeaksource.com/FuelExperiments> that it's >>> slow for them, which I guess is due to not adopting #identityHash calls to >>> #basicIdentityHash calls for Pharo: >>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>> >>> Uploaded a new version to the MC repository which is adapted for Pharo, >>> on the same machine my numbers were taken from, it does the same test as I >>> used above in 871 ms. (181 with preallocation). >>> >> >> Cool. One more thing: in Squeak the method using primitive 132 directly >> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >> this was also added to Pharo, then the #pointsTo: sends should be changed >> to #instVarsInclude:, otherwise Array can be reported as included even if >> it wasn't added. >> I'll upload my LargeIdentityDictionary implementation to the same place >> this evening, since it's still 2-3 factor faster than other solutionts and >> there seem to be demand for it. >> >> >> Levente >> >> >>> Cheers, >>> Henry >>> >>> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 16, 2011