On Sun, 18 Dec 2011, Mariano Martinez Peck wrote:

2011/12/18 Levente Uzonyi <leves@elte.hu>

On Sun, 18 Dec 2011, Mariano Martinez Peck wrote:

�Actually, the long test is

"Create some bitmaps"

� �numbers := �OrderedCollection new.
�numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray.
�numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each |
each negated] ).

"Put them in a dictionary using dict as as key"
�dict := LargeIdentityDictionary new.
�numbers do: [:each |
� � � dict at: each put: dict size + 1.
� � �].

"Check they are correct"
�numbers doWithIndex: [:each :index | self assert: index == (dict at:
each)]

Fixed.

That was FAST :)
Thanks Levente. Where can I find the fix?

It should be in the FuelExperiments repository, though I had to upload it one more time for some reason.

Levente

Cheers

Levente

On Sun, Dec 18, 2011 at 7:17 PM, Mariano Martinez Peck <
marianopeck@gmail.com> wrote:

�Now I noticed that SmallIntegers storing is really slow

This is not entirely true. Every hash table can be made slow if you
know
how it's hash function works. Since in your example all numbers are
congruent to 1 modulo 4096 and the size of the array is also 4096,
therefore they will be mapped to the same slot (1). Try using random
(or at
least more realistic) numbers in your test. If this is a real problem,
then
it may be necessary to implement a new hash method, which uses the
primitive for general objects and does something else with
SmallIntegers
and that method shoul be used from these large collections.

I uploaded a new mcz to the FuelExperiments repository, which is
unaffected by the identity hash differences between Pharo and Squeak and
works better with SmallIntegers. It includes both the dictionary and the
set and (in theory) has slightly better performance for non-SmallInteger
objects.


�Hi Levente. First let me say I really appreaciate your effort in make

it
work. It makes definitvly Fuel faster and it would make sense to
integrate
them also in Squeak and Pharo.
Now, I found another bug. To reproduce it:

| dict |
dict := LargeIdentityDictionary new.
dict at: 16941057 put: 1035.

Now, it seems 16941057 largeIdentityHash answers 4096 and the comment of
#largeIdentityHash �says ""Return an integer between 0 and 4095 based on
all of my bits. Make sure that for nearby receivers the result will not
be
nearby with high probability.""

The thing is that LargeIdentityHashedCollection permuteHash: hash + 1
answers 4096, and hence, when in #at:put: you do:

at: key put: value

� | hash |
� (keys at: (hash := key largeIdentityHash + 1))

then you get the out of bounds.

Modyfing

initialize

� | rng |
� rng := Random seed: 664399324.
� PermutationMap := (1 to: 4096) asArray shuffleBy: rng

�to 4095 make some tests to pass, but there are some other failing.

I will try to continue to see if I find the error.

Thanks!


�Levente

Levente


�numbers := �OrderedCollection new.

�numbers add: 1.
�numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray.
�numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each
|
each negated] ).

�dict := LargeIdentityDictionary new.
�[numbers do: [:each |
� � �dict at: each put: dict size + 1.
� � ].
�] timeToRun
�-> 12657


whereas with IdentityDictionary it is 37. �I guess it could be related
to
#identityHash or #basicIdentityHash. �I tried in Squeak 4.3 and it is
also
also there.
I will try to take a look into it during the next week.

Thanks in advance



�Nicolas

�Thanks for any idea.

On Sat, Dec 17, 2011 at 1:50 PM, St�phane Ducasse
<stephane.ducasse@inria.fr> wrote:

On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote:

�Cool. One more thing: in Squeak the method using primitive 132

�directly

�was renamed to #instVarsInclude:, so now #pointsTo: works as

�expected. If

�this was also added to Pharo, then the #pointsTo: sends should be

�changed to

�#instVarsInclude:, otherwise Array can be reported as included even

�if it

�wasn't added.

I'll upload my LargeIdentityDictionary implementation to the same

�place

�this evening, since it's still 2-3 factor faster than other

�solutionts and

�there seem to be demand for it.

Levente

�"in Squeak the method using primitive 132 directly was renamed to

#instVarsInclude:, so now #pointsTo: works as expected."

�I do not get the following. Indeed pointTo: looks like

instVarsInclude:
now I do not understand the rest of your paragraph.

Stef

--
Mariano

http://marianopeck.wordpress.****com<http://marianopeck.**
wordpress.com <http://marianopeck.wordpress.com>>

--
Mariano
http://marianopeck.wordpress.****com <http://marianopeck.wordpress.**
com <http://marianopeck.wordpress.com>>

--
Mariano
http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com>

--
Mariano
http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com>

--
Mariano
http://marianopeck.wordpress.com