December 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Class rename on materialization

by Martin Dias

On Sat, Dec 17, 2011 at 2:44 AM, Yanni Chiu <yanni(a)rogers.com> wrote: > On 16/12/11 3:09 PM, Mariano Martinez Peck wrote: > >> so guys??? Should we only renames Classes/Traits/ClassPools/**Methods >> or >> we directly rename all ocurrences of a symbol ? >> > > I haven't any use cases to decide. If you want a guess, then I'd say leave > the symbols unchanged, because it's simpler to implement. Since there's no > data points, at this point, why do work that may prove to be unnecessary. > > Well, so I guess we will change just the classes and traits, and not every symbol. I was in favor of renaming every symbol because the "class rename" refactoring changes any method who mention the original name. Thanks! Martin

Dec. 18, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Sat, 17 Dec 2011, Mariano Martinez Peck wrote: >>>> >>> >>> Can't you debug it ? >>> >> >> > Of course I could do it when I have some time. It already took me 1 hour to > isolate the bug from Fuel and make a reproducible test. I hope that at > least you find that useful. I found the bug and I thought I could send my > progress and see if someone could continue it. Otherwise, I was going to > debug it as soon as I have some free time (which I don't have much this > year). > > >> It took 2 minutes to find this sentence at line 19 of >> LargeDictionary>>at:put: >> >> > But you are so smart and a Collection hacker. Me, I am newbie, and just the > method at:put: scares me, but I understand if it has to be that way in > order to be fast. And so far it seems to be fast, so I am happy with it. > > > >> (values at: hash) at: newIndex put: key >> > > Can you replace key with value and retry the tests ? >> >> > Yes, that works. Thanks! > > Now I noticed that SmallIntegers storing is really slow This is not entirely true. Every hash table can be made slow if you know how it's hash function works. Since in your example all numbers are congruent to 1 modulo 4096 and the size of the array is also 4096, therefore they will be mapped to the same slot (1). Try using random (or at least more realistic) numbers in your test. If this is a real problem, then it may be necessary to implement a new hash method, which uses the primitive for general objects and does something else with SmallIntegers and that method shoul be used from these large collections. Levente > > numbers := OrderedCollection new. > numbers add: 1. > numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. > numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | > each negated] ). > > dict := LargeIdentityDictionary new. > [numbers do: [:each | > dict at: each put: dict size + 1. > ]. > ] timeToRun > -> 12657 > > > whereas with IdentityDictionary it is 37. I guess it could be related to > #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it is also > also there. > I will try to take a look into it during the next week. > > Thanks in advance > > > >> Nicolas >> >>>> >>>> Thanks for any idea. >>>> >>>> >>>> >>>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>>> <stephane.ducasse(a)inria.fr> wrote: >>>>> >>>>> >>>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>>> >>>>>> Cool. One more thing: in Squeak the method using primitive 132 >> directly >>>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as >> expected. If >>>>>> this was also added to Pharo, then the #pointsTo: sends should be >> changed to >>>>>> #instVarsInclude:, otherwise Array can be reported as included even >> if it >>>>>> wasn't added. >>>>>> I'll upload my LargeIdentityDictionary implementation to the same >> place >>>>>> this evening, since it's still 2-3 factor faster than other >> solutionts and >>>>>> there seem to be demand for it. >>>>> >>>>> Levente >>>>> >>>>>> "in Squeak the method using primitive 132 directly was renamed to >>>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>>> I do not get the following. Indeed pointTo: looks like instVarsInclude: >>>>> now I do not understand the rest of your paragraph. >>>>> >>>>> Stef >>>> >>>> >>>> >>>> >>>> -- >>>> Mariano >>>> http://marianopeck.wordpress.com >>>> >> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Sat, 17 Dec 2011, Nicolas Cellier wrote: > 2011/12/17 Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com>: >> 2011/12/17 Mariano Martinez Peck <marianopeck(a)gmail.com>: >>> Hi Levente. I take a deeper look to see why LargeIdentityDictionary is not >>> working as expected for us. >>> >>> The way we are putting objects in the Dict is always this way: >>> >>> registerIndexesOn: aDictionary >>> >>> Â Â Â self objects do: [ :instance | aDictionary at: instance put: aDictionary >>> size + 1 ]. >>> >>> >>> So?keys are the objects of the graphs and the values are the internal >>> position in the serialization.Â However, when I run a serialization it looks >>> like if the follow happens that when we are doing: >>> >>>>> encodeReferenceTo: anObject >>> >>> Â Â Â indexCluster >>> Â Â Â Â Â Â serialize: (objectsIndexes at: anObject) >>> Â Â Â Â Â Â on: stream >>> >>> >>> (objectsIndexes at: anObject) == anObjectÂ Â ->Â true >>> >>> soÂ (objectsIndexes at: anObject)Â is answering "self" rather than the >>> number (which was the dict size + 1 at the serialization moment). >>> >>> Something interesting is that most tests seem to pass and the problem is >>> with benchmarks (where graphs are much bigger), so maybe it could be >>> something related to that.Â For example, the following benchmark for bitmaps >>> is failing: >>> >>> Â Â Â Â Â Â Â Â numbers := OrderedCollection new. >>> Â Â Â bitmaps := OrderedCollection new. >>> Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. >>> Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. >>> Â Â Â ^ bitmaps >>> >>> it happens the mentioned problem when trying to serialize the collection >>> 'bitmaps'. >>> >>> >>> I tried to reproduce it outside Fuel and I think I succeeded (not sure). The >>> following does work with IdentityDictionary but not with >>> LargeIdentityDictionary. Basically I am simulating the way we use this dict >>> in Fuel: >>> >>> testLargeIdentityDictionary >>> >>> Â Â Â | numbers bitmaps dict | >>> >>> "Create some bitmaps" >>> >>> Â Â Â numbers := OrderedCollection new. >>> Â Â Â bitmaps := OrderedCollection new. >>> Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. >>> Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. >>> >>> "Put them in a dictionary using dict as as key" >>> Â Â Â dict := LargeIdentityDictionary new. >>> Â Â Â bitmaps do: [:each | >>> Â Â Â Â Â Â Â dict at: each put: dict size + 1. >>> Â Â Â Â Â Â ]. >>> >>> "Check they are correct" >>> Â Â Â bitmaps doWithIndex: [:each :index | self assert: index == (dict at: >>> each)] >>> >>> Of course that test works with IdentityDictionary. >>> >> >> Can't you debug it ? >> > > It took 2 minutes to find this sentence at line 19 of LargeDictionary>>at:put: > > (values at: hash) at: newIndex put: key > > Can you replace key with value and retry the tests ? Nice find, thanks. :) Levente > > Nicolas > >>> >>> Thanks for any idea. >>> >>> >>> >>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>> <stephane.ducasse(a)inria.fr> wrote: >>>> >>>> >>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>> >>>>> Cool. One more thing: in Squeak the method using primitive 132 directly >>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>>>> this was also added to Pharo, then the #pointsTo: sends should be changed to >>>>> #instVarsInclude:, otherwise Array can be reported as included even if it >>>>> wasn't added. >>>>> I'll upload my LargeIdentityDictionary implementation to the same place >>>>> this evening, since it's still 2-3 factor faster than other solutionts and >>>>> there seem to be demand for it. >>>> >>>> Levente >>>> >>>>> "in Squeak the method using primitive 132 directly was renamed to >>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>> I do not get the following. Indeed pointTo: looks like instVarsInclude: >>>> now I do not understand the rest of your paragraph. >>>> >>>> Stef >>> >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> > >

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Sat, 17 Dec 2011, StÃ©phane Ducasse wrote: > > On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: > >> Cool. One more thing: in Squeak the method using primitive 132 directly was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If this was also added to Pharo, then the #pointsTo: sends should be changed to #instVarsInclude:, otherwise Array can be reported as included even if it wasn't added. >> I'll upload my LargeIdentityDictionary implementation to the same place this evening, since it's still 2-3 factor faster than other solutionts and there seem to be demand for it. > > Levente > >> "in Squeak the method using primitive 132 directly was renamed to #instVarsInclude:, so now #pointsTo: works as expected." > I do not get the following. Indeed pointTo: looks like instVarsInclude: now I do not understand the rest of your paragraph. #pointsTo: should answer true if the receiver has a pointer to the argument. Every object (except for SmallIntegers and instances of compact classes) holds a pointer to its class in the header. Since the primitive only checks the slots, it will not always return true if the argument is the object's class. This breaks pointer finding/tracing tools, that's why it was changed in Squeak. See how #instVarsInclude: and #pointsTo: is implemented there. Levente > > Stef >

Dec. 17, 2011

Re: [Pharo-project] Time to rename Object>>#name ?

by Levente Uzonyi

On Sat, 17 Dec 2011, Janko MivÅ¡ek wrote: > Hi guys, > > Isn't a time come to finally rename this unfortunate method #name in > Object to something more internal? From its comment it seems it is using > just for debugging: > > Object>>name > "Answer a name for the receiver. This is used generically in the > title of certain inspectors, such as the referred-to inspector, and > specificially by various subsystems. By default, we let the object > just print itself out.. " > > ^self printString > > I namely just crashed my image and spend quite some time to find why, by > implementing: > > MyClass>>printString > > ^'aMyClass', self name You should almost never override #printString. If you want to change the printString of an object, then you should override #printOn:. Levente > > Later I renamed #name to say #myName but forgot to rename a call in this > #printString method too. Boom, infinite recursion! > > Please allow us therefore to use #name in our classes for things a word > "name" is meant! > > Best regards > Janko > > -- > Janko MivÅ¡ek > Aida/Web > Smalltalk Web Application Server > http://www.aidaweb.si > >

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Fri, 16 Dec 2011, Benoit St-Jean wrote: > Hi Levente, Have you tried a different value for your array and tallies size instead of 4096 in the code you provided for LargeIdentitySet? Such as prime numbers, as it is usually the case for hash tables? I've tried 3079, and 4093 (prime numbers) and 3079 performs a little bit better .Â Other meaningful suggestions can be found here : http://planetmath.org/encyclopedia/GoodHashTablePrimes.html P.S.Â So far, 3079 is the best candidate saving between 26 and 35% on very large sets !! P.P.S.Â Nice work! Using primes is suboptimal in this case, because there are exactly 4096 possible identityHash values (for non-SmallInteger objects), so the mapping from hash value to slot is as simple (and fast) as possible. Also note that not all hash tables use prime sizes, using a power of two for size and multiplying in the hash function with a special prime is also very common. Levente ----------------- Benoit St-Jean Yahoo! Messenger: bstjean A standpoint is an intellectual horizon of radius zero. (Albert Einstein) >________________________________ > From: Levente Uzonyi <leves(a)elte.hu> >To: Pharo-project(a)lists.gforge.inria.fr >Sent: Friday, December 16, 2011 6:52:29 PM >Subject: Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ? > >On Fri, 16 Dec 2011, Mariano Martinez Peck wrote: > >> Wow...... >> >> Henrik: thank you so much for teaching me :)Â Indeed, I didn't notice that >> in that case I use using a LargePositiveInteger, and indeed I had removed >> the basicSize since I also noticed it wouldn't make much sense.Â Second, >> thanks a LOT for finding Levente's LargeIdentitySet in Fuel repo with our >> small adaptation (#addIfNotPresent: anObject ifPresentDo: aBlock). Thanks >> for fixing it and commiting a new version. >> >> All I can say is that I am impressed by the numbers it is really much >> faster. >> I still don't understand why I send this email with a subject say >> IdentitySet because what I really need is a fast/large IdentityDictionary >> :(Â Anyway, there's a place where we can use this LargeIdentitySet in Fuel >> I think). >> >> So Levente, you say this is not possible to adapt this for dictionary?Â can >> we contact Eliot to provide such a primitive? > >As promised, I uploaded my LargeIdentityDictionary implementation to >http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st . >The numbers will be a bit worse compared to LargeIdentitySet, because of >the lack of the primitive, but it's still 2-3x faster than other >solutions (IdentityDictionary, PluggableIdentityDictionary, subclassing, >etc). I'm about to propose this primitive with other improvements on the >vm-dev list. > > >Levente > >> >> thanks >> >> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >>> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>>Â On 16.12.2011 03:26, Levente Uzonyi wrote: >>>> >>>>> >>>>> How about my numbers? :) >>>>> >>>>> "Preallocate objects, so we won't count gc time." >>>>> n := 1000000. >>>>> objects := Array new: n streamContents: [ :stream | >>>>>Â Â n timesRepeat: [ stream nextPut: Object new ] ]. >>>>> >>>>> set := IdentitySet new: n. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>> >>>>> set := LargeIdentitySet new. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>> >>>>> set := (PluggableSet new: n) >>>>>Â Â hashBlock: [ :object | object identityHash * 4096 + object class >>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>>Â Â equalBlock: [ :a :b | a == b ]; >>>>>Â Â yourself. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>> >>>>> >>>>> I also have a LargeIdentityDictionary, which is relatively fast, but not >>>>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>>> have a primitive that could support it. If we had a primitive like >>>>> primitive 132 which would return the index of the element if found or 0 if >>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>> >>>>> >>>>> Levente >>>>> >>>> Hehe yes, if writing a version fully exploiting the limited range, that's >>>> probably the approach I would go for as well. >>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>> squeak/LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>> ) >>>> >>>> Mariano commented in the version at http://www.squeaksource.com/** >>>> FuelExperiments <http://www.squeaksource.com/FuelExperiments> that it's >>>> slow for them, which I guess is due to not adopting #identityHash calls to >>>> #basicIdentityHash calls for Pharo: >>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>> >>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>> on the same machine my numbers were taken from, it does the same test as I >>>> used above in 871 ms. (181 with preallocation). >>>> >>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>>> Cheers, >>>> Henry >>>> >>>> >>>> >>> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> > > > >

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

> >> > > > > Can't you debug it ? > > > > Of course I could do it when I have some time. It already took me 1 hour to isolate the bug from Fuel and make a reproducible test. I hope that at least you find that useful. I found the bug and I thought I could send my progress and see if someone could continue it. Otherwise, I was going to debug it as soon as I have some free time (which I don't have much this year). > It took 2 minutes to find this sentence at line 19 of > LargeDictionary>>at:put: > > But you are so smart and a Collection hacker. Me, I am newbie, and just the method at:put: scares me, but I understand if it has to be that way in order to be fast. And so far it seems to be fast, so I am happy with it. > (values at: hash) at: newIndex put: key > Can you replace key with value and retry the tests ? > > Yes, that works. Thanks! Now I noticed that SmallIntegers storing is really slow numbers := OrderedCollection new. numbers add: 1. numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | each negated] ). dict := LargeIdentityDictionary new. [numbers do: [:each | dict at: each put: dict size + 1. ]. ] timeToRun -> 12657 whereas with IdentityDictionary it is 37. I guess it could be related to #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it is also also there. I will try to take a look into it during the next week. Thanks in advance > Nicolas > > >> > >> Thanks for any idea. > >> > >> > >> > >> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse > >> <stephane.ducasse(a)inria.fr> wrote: > >>> > >>> > >>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: > >>> > >>> > Cool. One more thing: in Squeak the method using primitive 132 > directly > >>> > was renamed to #instVarsInclude:, so now #pointsTo: works as > expected. If > >>> > this was also added to Pharo, then the #pointsTo: sends should be > changed to > >>> > #instVarsInclude:, otherwise Array can be reported as included even > if it > >>> > wasn't added. > >>> > I'll upload my LargeIdentityDictionary implementation to the same > place > >>> > this evening, since it's still 2-3 factor faster than other > solutionts and > >>> > there seem to be demand for it. > >>> > >>> Levente > >>> > >>> > "in Squeak the method using primitive 132 directly was renamed to > >>> > #instVarsInclude:, so now #pointsTo: works as expected." > >>> I do not get the following. Indeed pointTo: looks like instVarsInclude: > >>> now I do not understand the rest of your paragraph. > >>> > >>> Stef > >> > >> > >> > >> > >> -- > >> Mariano > >> http://marianopeck.wordpress.com > >> > > -- Mariano http://marianopeck.wordpress.com

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Nicolas Cellier

2011/12/17 Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com>: > 2011/12/17 Mariano Martinez Peck <marianopeck(a)gmail.com>: >> Hi Levente. I take a deeper look to see why LargeIdentityDictionary is not >> working as expected for us. >> >> The way we are putting objects in the Dict is always this way: >> >> registerIndexesOn: aDictionary >> >> Â Â Â self objects do: [ :instance | aDictionary at: instance put: aDictionary >> size + 1 ]. >> >> >> Soâ¦keys are the objects of the graphs and the values are the internal >> position in the serialization.Â However, when I run a serialization it looks >> like if the follow happens that when we are doing: >> >>>> encodeReferenceTo: anObject >> >> Â Â Â indexCluster >> Â Â Â Â Â Â serialize: (objectsIndexes at: anObject) >> Â Â Â Â Â Â on: stream >> >> >> (objectsIndexes at: anObject) == anObjectÂ Â ->Â true >> >> soÂ (objectsIndexes at: anObject)Â is answering "self" rather than the >> number (which was the dict size + 1 at the serialization moment). >> >> Something interesting is that most tests seem to pass and the problem is >> with benchmarks (where graphs are much bigger), so maybe it could be >> something related to that.Â For example, the following benchmark for bitmaps >> is failing: >> >> Â Â Â Â Â Â Â Â numbers := OrderedCollection new. >> Â Â Â bitmaps := OrderedCollection new. >> Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. >> Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. >> Â Â Â ^ bitmaps >> >> it happens the mentioned problem when trying to serialize the collection >> 'bitmaps'. >> >> >> I tried to reproduce it outside Fuel and I think I succeeded (not sure). The >> following does work with IdentityDictionary but not with >> LargeIdentityDictionary. Basically I am simulating the way we use this dict >> in Fuel: >> >> testLargeIdentityDictionary >> >> Â Â Â | numbers bitmaps dict | >> >> "Create some bitmaps" >> >> Â Â Â numbers := OrderedCollection new. >> Â Â Â bitmaps := OrderedCollection new. >> Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. >> Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. >> >> "Put them in a dictionary using dict as as key" >> Â Â Â dict := LargeIdentityDictionary new. >> Â Â Â bitmaps do: [:each | >> Â Â Â Â Â Â Â dict at: each put: dict size + 1. >> Â Â Â Â Â Â ]. >> >> "Check they are correct" >> Â Â Â bitmaps doWithIndex: [:each :index | self assert: index == (dict at: >> each)] >> >> Of course that test works with IdentityDictionary. >> > > Can't you debug it ? > It took 2 minutes to find this sentence at line 19 of LargeDictionary>>at:put: (values at: hash) at: newIndex put: key Can you replace key with value and retry the tests ? Nicolas >> >> Thanks for any idea. >> >> >> >> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >> <stephane.ducasse(a)inria.fr> wrote: >>> >>> >>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>> >>> > Cool. One more thing: in Squeak the method using primitive 132 directly >>> > was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> > this was also added to Pharo, then the #pointsTo: sends should be changed to >>> > #instVarsInclude:, otherwise Array can be reported as included even if it >>> > wasn't added. >>> > I'll upload my LargeIdentityDictionary implementation to the same place >>> > this evening, since it's still 2-3 factor faster than other solutionts and >>> > there seem to be demand for it. >>> >>> Levente >>> >>> > "in Squeak the method using primitive 132 directly was renamed to >>> > #instVarsInclude:, so now #pointsTo: works as expected." >>> I do not get the following. Indeed pointTo: looks like instVarsInclude: >>> now I do not understand the rest of your paragraph. >>> >>> Stef >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >>

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Nicolas Cellier

2011/12/17 Mariano Martinez Peck <marianopeck(a)gmail.com>: > Hi Levente. I take a deeper look to see why LargeIdentityDictionary is not > working as expected for us. > > The way we are putting objects in the Dict is always this way: > > registerIndexesOn: aDictionary > > Â Â Â self objects do: [ :instance | aDictionary at: instance put: aDictionary > size + 1 ]. > > > Soâ¦keys are the objects of the graphs and the values are the internal > position in the serialization.Â However, when I run a serialization it looks > like if the follow happens that when we are doing: > >>> encodeReferenceTo: anObject > > Â Â Â indexCluster > Â Â Â Â Â Â serialize: (objectsIndexes at: anObject) > Â Â Â Â Â Â on: stream > > > (objectsIndexes at: anObject) == anObjectÂ Â ->Â true > > soÂ (objectsIndexes at: anObject)Â is answering "self" rather than the > number (which was the dict size + 1 at the serialization moment). > > Something interesting is that most tests seem to pass and the problem is > with benchmarks (where graphs are much bigger), so maybe it could be > something related to that.Â For example, the following benchmark for bitmaps > is failing: > > Â Â Â Â Â Â Â Â numbers := OrderedCollection new. > Â Â Â bitmaps := OrderedCollection new. > Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. > Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. > Â Â Â ^ bitmaps > > it happens the mentioned problem when trying to serialize the collection > 'bitmaps'. > > > I tried to reproduce it outside Fuel and I think I succeeded (not sure). The > following does work with IdentityDictionary but not with > LargeIdentityDictionary. Basically I am simulating the way we use this dict > in Fuel: > > testLargeIdentityDictionary > > Â Â Â | numbers bitmaps dict | > > "Create some bitmaps" > > Â Â Â numbers := OrderedCollection new. > Â Â Â bitmaps := OrderedCollection new. > Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. > Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. > > "Put them in a dictionary using dict as as key" > Â Â Â dict := LargeIdentityDictionary new. > Â Â Â bitmaps do: [:each | > Â Â Â Â Â Â Â dict at: each put: dict size + 1. > Â Â Â Â Â Â ]. > > "Check they are correct" > Â Â Â bitmaps doWithIndex: [:each :index | self assert: index == (dict at: > each)] > > Of course that test works with IdentityDictionary. > Can't you debug it ? > > Thanks for any idea. > > > > On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse > <stephane.ducasse(a)inria.fr> wrote: >> >> >> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >> >> > Cool. One more thing: in Squeak the method using primitive 132 directly >> > was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >> > this was also added to Pharo, then the #pointsTo: sends should be changed to >> > #instVarsInclude:, otherwise Array can be reported as included even if it >> > wasn't added. >> > I'll upload my LargeIdentityDictionary implementation to the same place >> > this evening, since it's still 2-3 factor faster than other solutionts and >> > there seem to be demand for it. >> >> Levente >> >> > "in Squeak the method using primitive 132 directly was renamed to >> > #instVarsInclude:, so now #pointsTo: works as expected." >> I do not get the following. Indeed pointTo: looks like instVarsInclude: >> now I do not understand the rest of your paragraph. >> >> Stef > > > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 17, 2011

Re: [Pharo-project] NB Bug?

by Igor Stasenko

On 16 December 2011 13:39, Norbert Hartl <norbert(a)hartl.name> wrote: > > Am 16.12.2011 um 11:47 schrieb Igor Stasenko: > >> On 16 December 2011 11:02, Stefan Marr <smalltalk(a)stefan-marr.de> wrote: >>> >>> On 16 Dec 2011, at 10:49, Igor Stasenko wrote: >>>>> >>>> oh please. that's even less obvious usage :) >>>> there's a lot of different stuff in C syntax which you can play with. >>>> But it doesn't means that you need to routinely support >>>> everything in FFI callout specs. >>>> >>>> as i said, the function signature syntax are close to C syntax, but >>>> not strictly C. >>> >>> Igor, what is your goal with that? >>> Have something working, or have something usable? >>> >>> If your goal is to have something usable, then your direct competitor is LuaJIT2's FFI >>> http://luajit.org/ext_ffi.html >>> and you should show similar ambitions. (Note specifically the 'cut&past part') >>> Otherwise, it will always remain just a nice try, an experiment that never got finished/productized. >>> >>> My understanding is that your boss got other plans for Pharo... >>> >> well, there are things which i would like to have, like integrating >> with Cog (so a call to generated native code >> will be performed directly , instead via primitive), non-blocking and callbacks. >> This is much more important to have than subtleties, how to define a >> function which takes no arguments by using Â '()' or '(void)'.. >> Kind of 'triangle has three corners' vs 'triangle has three edges' discussion. >> > Igor, > > importance lies in the eye of the beholder. You are doing great stuff and you don't want to waste your time with things you consider unnecessary. But you need to decide what is the goal of NativeBoost. If you like to keep the pace and realize cool demos then you shouldn't listen to other people. If you think you like to make it an accepted piece of software that has a lot of users then you need listen to them. People posting in this thread are your potential users. They tell you things the fail over. If this happens more often you'll loose them because they don't want to waste their time with non-working stuff (the term non-working also lies in the eye of the beholder). Basically it is that easy. If you don't care about the (probably stupid) wishes of your users it stays your private pet project. > Yes, i understand what you saying about. :) > Norbert -- Best regards, Igor Stasenko.

Dec. 17, 2011