December 2011 - Pharo-dev - lists.pharo.org

[Pharo-project] Newbie observations on use of Metacello

by Ben Coman

Some observations from the perspective of as a newbie on Pharo Package Management. I'm likely to me missing in the philosophy of the design, but from the narrow view of my own usage.... 1. When I read about Metacello being the package management system, I expected there to be a "command"(*) Metacallo but instead I see Gopher being used. This is a mixed message that lowers my comfort level feeling I understand the system as I'm trying to pick up a bundle of new ideas. As I now understand it Metacello is more about a system of backend configuration than a front end user interface. I am now more comfortable using Gopher direct - but I reflect on how I felt the first time. (*) While technically a "command" is like a verb, and Gopher is like a noun rather than a verb - I can't yet completely disengage my prior learning that considers the first part of the statement to be a "command" - and I expect other newbies might find the same. 2. The strong convention of naming things ConfigurationOfXXXX is somewhat redundant from a user perspective. 3. If copy-paste into the Workspace from a web site a sample package load (for example b> below) and then highlight the whole thing and execute, it tells me "Unknown variable: ConfigurationOfSqueakDBX, please correct, or cancel" Duh! What? Oh, I need to execute it in two parts. No one wrote that down anywhere.... The alternative "(Smalltalk at: #ConfigurationOfSqueakDBX)" adds further complexity in the eyes of a newbie. Based on the above points, it would be nice to have... a> Metacello configuration: 'SqueakDBX' load rather than... b> Gofer new b> squeaksource: 'MetacelloRepository'; b> package: 'ConfigurationOfSqueakDBX'; b> load. b> ConfigurationOfSqueakDBX project latestVersion load. While only a small saving in typing, I think it significant for newbies in terms of polish and the marketing message that Metacello really is the chosen package management system. A very rough implementation (with much of the syntax likely wrong) might be just... c> Metacello class >> package: aPackage c> | packageConfig | c> packageConfig := 'ConfigurationOf', aPackage; c> Gopher new c> squeaksource: 'MetacelloRepository'; c> package: packageConfig c> load c> ^((Smalltalk at: (packageConfig asSymbol)) Now after I've written the above I contemplate that there must have been an early design decision to not implement a separate Metacello class to avoid duplication/overlap with Monticello. However perhaps now that the system is implemented as it is, a thin shim can be put over the top to assist getting the ConfigurationOfXXX files downloaded. Additional things this Metacello class might later provide: a. Improved reliability by downloading from a system of package repository mirrors. SqueakSource was having some issues and I found [1] a fortunate backup. b. A GUI for browsing package versions c. A GUI for browsing compatibility tables from a continuous integration system that tries different permutations of Montecello packages together. 4. It seems that every ConfigurationOfXXX has its own category in the SystemBrowser. Each of these categories has only one class. This seems to me like unneeded clutter of the System Browser. It would be neater if Metacello specified its configurations to be grouped under a single category "MetacelloConfigurations" or similar. 5. The sample Metacello PBE is very good in the detail for developers to provide configurations. However there is not much directed clearly at a 'mere' user. Even though Smalltalk makes us all developers, newbies all start as mere users while exploring the system. [1] Gofer new url: 'http://dsal.cl/squeaksource/MonticelloRedirect'; package: 'MontiRedirect'; load. MRManager redirectFrom: 'http://www.squeaksource.com/' to: 'http://dsal.cl/squeaksource/'. Cheers, Ben

Dec. 19, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by StÃ©phane Ducasse

Hi levente We will certainly do that. Right now I'm resting from the crazy weeks before christmas and too many articles writing at the last minute (this is part of the job but working even the saturday and sunday can simply lead to energy burnout. Anyway I'm happy we did it). @Mariano I opened a bug entry for it. If you have the time (have a look). I will try tomorrow but I would like to relax and I booked a real holiday day: (gym, game with kids and probably hacking with igor :)). Stef > The best would be to add the patch to Pharo, which fixes #pointsTo: and adds #instVarsIncludes:. Then change the code to use #instVarsInclude: in these collections. If that's all done, then this code should work in both Pharo and Squeak. > >

Dec. 18, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: > 2011/12/18 Levente Uzonyi <leves(a)elte.hu> > >> On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: >> >> 2011/12/18 Levente Uzonyi <leves(a)elte.hu> >>> >>> On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: >>>> >>>> Actually, the long test is >>>> >>>>> "Create some bitmaps" >>>>> >>>>> numbers := OrderedCollection new. >>>>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>>>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | >>>>> each negated] ). >>>>> >>>>> "Put them in a dictionary using dict as as key" >>>>> dict := LargeIdentityDictionary new. >>>>> numbers do: [:each | >>>>> dict at: each put: dict size + 1. >>>>> ]. >>>>> >>>>> "Check they are correct" >>>>> numbers doWithIndex: [:each :index | self assert: index == (dict at: >>>>> each)] >>>>> >>>>> >>>> Fixed. >>>> >>>> >>>> That was FAST :) >>> Thanks Levente. Where can I find the fix? >>> >> >> It should be in the FuelExperiments repository, though I had to upload it >> one more time for some reason. >> >> > Thanks Levente. Well, now all our tests and benchmarks pass like a charm > :) We will soon include it in the "trunk" of Fuel. Will them be available > in a repository for squeak? because if so, I can adapt ConfiguratioOfFuel > so that in Squeak it loads another collections. The best would be to add the patch to Pharo, which fixes #pointsTo: and adds #instVarsIncludes:. Then change the code to use #instVarsInclude: in these collections. If that's all done, then this code should work in both Pharo and Squeak. > > So...from the benchmarks I can say that in large graphs the improve in > performance was 30% in serialization, and only use in one place the set and > in another one the dictionary. Great, hopefully the new primitive will increase this even more. :) Levente > > Thanks!!! > > > >> >> Levente >> >> >>> Cheers >>> >>> >>> >>>> Levente >>>> >>>> >>>> On Sun, Dec 18, 2011 at 7:17 PM, Mariano Martinez Peck < >>>>> marianopeck(a)gmail.com> wrote: >>>>> >>>>> >>>>> >>>>>> Now I noticed that SmallIntegers storing is really slow >>>>>> >>>>>>> >>>>>>>>> >>>>>>>>> This is not entirely true. Every hash table can be made slow if you >>>>>>>> know >>>>>>>> how it's hash function works. Since in your example all numbers are >>>>>>>> congruent to 1 modulo 4096 and the size of the array is also 4096, >>>>>>>> therefore they will be mapped to the same slot (1). Try using random >>>>>>>> (or at >>>>>>>> least more realistic) numbers in your test. If this is a real >>>>>>>> problem, >>>>>>>> then >>>>>>>> it may be necessary to implement a new hash method, which uses the >>>>>>>> primitive for general objects and does something else with >>>>>>>> SmallIntegers >>>>>>>> and that method shoul be used from these large collections. >>>>>>>> >>>>>>>> >>>>>>>> I uploaded a new mcz to the FuelExperiments repository, which is >>>>>>> unaffected by the identity hash differences between Pharo and Squeak >>>>>>> and >>>>>>> works better with SmallIntegers. It includes both the dictionary and >>>>>>> the >>>>>>> set and (in theory) has slightly better performance for >>>>>>> non-SmallInteger >>>>>>> objects. >>>>>>> >>>>>>> >>>>>>> Hi Levente. First let me say I really appreaciate your effort in make >>>>>>> >>>>>> it >>>>>> work. It makes definitvly Fuel faster and it would make sense to >>>>>> integrate >>>>>> them also in Squeak and Pharo. >>>>>> Now, I found another bug. To reproduce it: >>>>>> >>>>>> | dict | >>>>>> dict := LargeIdentityDictionary new. >>>>>> dict at: 16941057 put: 1035. >>>>>> >>>>>> Now, it seems 16941057 largeIdentityHash answers 4096 and the comment >>>>>> of >>>>>> #largeIdentityHash says ""Return an integer between 0 and 4095 based >>>>>> on >>>>>> all of my bits. Make sure that for nearby receivers the result will not >>>>>> be >>>>>> nearby with high probability."" >>>>>> >>>>>> The thing is that LargeIdentityHashedCollection permuteHash: hash + 1 >>>>>> answers 4096, and hence, when in #at:put: you do: >>>>>> >>>>>> at: key put: value >>>>>> >>>>>> | hash | >>>>>> (keys at: (hash := key largeIdentityHash + 1)) >>>>>> >>>>>> then you get the out of bounds. >>>>>> >>>>>> Modyfing >>>>>> >>>>>> initialize >>>>>> >>>>>> | rng | >>>>>> rng := Random seed: 664399324. >>>>>> PermutationMap := (1 to: 4096) asArray shuffleBy: rng >>>>>> >>>>>> to 4095 make some tests to pass, but there are some other failing. >>>>>> >>>>>> I will try to continue to see if I find the error. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> >>>>>> Levente >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Levente >>>>>>>> >>>>>>>> >>>>>>>> numbers := OrderedCollection new. >>>>>>>> >>>>>>>>> numbers add: 1. >>>>>>>>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>>>>>>>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: >>>>>>>>> [:each >>>>>>>>> | >>>>>>>>> each negated] ). >>>>>>>>> >>>>>>>>> dict := LargeIdentityDictionary new. >>>>>>>>> [numbers do: [:each | >>>>>>>>> dict at: each put: dict size + 1. >>>>>>>>> ]. >>>>>>>>> ] timeToRun >>>>>>>>> -> 12657 >>>>>>>>> >>>>>>>>> >>>>>>>>> whereas with IdentityDictionary it is 37. I guess it could be >>>>>>>>> related >>>>>>>>> to >>>>>>>>> #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it >>>>>>>>> is >>>>>>>>> also >>>>>>>>> also there. >>>>>>>>> I will try to take a look into it during the next week. >>>>>>>>> >>>>>>>>> Thanks in advance >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Nicolas >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks for any idea. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>>>>>>>>>>> <stephane.ducasse(a)inria.fr> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Cool. One more thing: in Squeak the method using primitive 132 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> directly >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> expected. If >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> this was also added to Pharo, then the #pointsTo: sends should >>>>>>>>>> be >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> changed to >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> #instVarsInclude:, otherwise Array can be reported as included >>>>>>>>>> even >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> if it >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> wasn't added. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I'll upload my LargeIdentityDictionary implementation to the same >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> place >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> this evening, since it's still 2-3 factor faster than other >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> solutionts and >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> there seem to be demand for it. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>> Levente >>>>>>>>>>>>> >>>>>>>>>>>>> "in Squeak the method using primitive 132 directly was renamed >>>>>>>>>>>>> to >>>>>>>>>>>>> >>>>>>>>>>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>>>>>>>>>>>> >>>>>>>>>>>>>> I do not get the following. Indeed pointTo: looks like >>>>>>>>>>>>>> >>>>>>>>>>>>> instVarsInclude: >>>>>>>>>>>>> now I do not understand the rest of your paragraph. >>>>>>>>>>>>> >>>>>>>>>>>>> Stef >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Mariano >>>>>>>>>>>> http://marianopeck.wordpress.******com<http://marianopeck.** >>>>>>>>>>>> wordpress.com <http://marianopeck.wordpress.**com<http://marianopeck.wordpress.com> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> Mariano >>>>>>>>> http://marianopeck.wordpress.******com < >>>>>>>>> http://marianopeck.wordpress.**** >>>>>>>>> com <http://marianopeck.wordpress.**com<http://marianopeck.wordpress.com> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> -- >>>>>> Mariano >>>>>> http://marianopeck.wordpress.****com <http://marianopeck.wordpress.** >>>>>> com <http://marianopeck.wordpress.com>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> -- >>>>> Mariano >>>>> http://marianopeck.wordpress.****com <http://marianopeck.wordpress.** >>>>> com <http://marianopeck.wordpress.com>> >>>>> >>>>> >>>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 18, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

2011/12/18 Levente Uzonyi <leves(a)elte.hu> > On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: > > 2011/12/18 Levente Uzonyi <leves(a)elte.hu> >> >> On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: >>> >>> Actually, the long test is >>> >>>> "Create some bitmaps" >>>> >>>> numbers := OrderedCollection new. >>>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | >>>> each negated] ). >>>> >>>> "Put them in a dictionary using dict as as key" >>>> dict := LargeIdentityDictionary new. >>>> numbers do: [:each | >>>> dict at: each put: dict size + 1. >>>> ]. >>>> >>>> "Check they are correct" >>>> numbers doWithIndex: [:each :index | self assert: index == (dict at: >>>> each)] >>>> >>>> >>> Fixed. >>> >>> >>> That was FAST :) >> Thanks Levente. Where can I find the fix? >> > > It should be in the FuelExperiments repository, though I had to upload it > one more time for some reason. > > Thanks Levente. Well, now all our tests and benchmarks pass like a charm :) We will soon include it in the "trunk" of Fuel. Will them be available in a repository for squeak? because if so, I can adapt ConfiguratioOfFuel so that in Squeak it loads another collections. So...from the benchmarks I can say that in large graphs the improve in performance was 30% in serialization, and only use in one place the set and in another one the dictionary. Thanks!!! > > Levente > > >> Cheers >> >> >> >>> Levente >>> >>> >>> On Sun, Dec 18, 2011 at 7:17 PM, Mariano Martinez Peck < >>>> marianopeck(a)gmail.com> wrote: >>>> >>>> >>>> >>>>> Now I noticed that SmallIntegers storing is really slow >>>>> >>>>>> >>>>>>>> >>>>>>>> This is not entirely true. Every hash table can be made slow if you >>>>>>> know >>>>>>> how it's hash function works. Since in your example all numbers are >>>>>>> congruent to 1 modulo 4096 and the size of the array is also 4096, >>>>>>> therefore they will be mapped to the same slot (1). Try using random >>>>>>> (or at >>>>>>> least more realistic) numbers in your test. If this is a real >>>>>>> problem, >>>>>>> then >>>>>>> it may be necessary to implement a new hash method, which uses the >>>>>>> primitive for general objects and does something else with >>>>>>> SmallIntegers >>>>>>> and that method shoul be used from these large collections. >>>>>>> >>>>>>> >>>>>>> I uploaded a new mcz to the FuelExperiments repository, which is >>>>>> unaffected by the identity hash differences between Pharo and Squeak >>>>>> and >>>>>> works better with SmallIntegers. It includes both the dictionary and >>>>>> the >>>>>> set and (in theory) has slightly better performance for >>>>>> non-SmallInteger >>>>>> objects. >>>>>> >>>>>> >>>>>> Hi Levente. First let me say I really appreaciate your effort in make >>>>>> >>>>> it >>>>> work. It makes definitvly Fuel faster and it would make sense to >>>>> integrate >>>>> them also in Squeak and Pharo. >>>>> Now, I found another bug. To reproduce it: >>>>> >>>>> | dict | >>>>> dict := LargeIdentityDictionary new. >>>>> dict at: 16941057 put: 1035. >>>>> >>>>> Now, it seems 16941057 largeIdentityHash answers 4096 and the comment >>>>> of >>>>> #largeIdentityHash says ""Return an integer between 0 and 4095 based >>>>> on >>>>> all of my bits. Make sure that for nearby receivers the result will not >>>>> be >>>>> nearby with high probability."" >>>>> >>>>> The thing is that LargeIdentityHashedCollection permuteHash: hash + 1 >>>>> answers 4096, and hence, when in #at:put: you do: >>>>> >>>>> at: key put: value >>>>> >>>>> | hash | >>>>> (keys at: (hash := key largeIdentityHash + 1)) >>>>> >>>>> then you get the out of bounds. >>>>> >>>>> Modyfing >>>>> >>>>> initialize >>>>> >>>>> | rng | >>>>> rng := Random seed: 664399324. >>>>> PermutationMap := (1 to: 4096) asArray shuffleBy: rng >>>>> >>>>> to 4095 make some tests to pass, but there are some other failing. >>>>> >>>>> I will try to continue to see if I find the error. >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> Levente >>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Levente >>>>>>> >>>>>>> >>>>>>> numbers := OrderedCollection new. >>>>>>> >>>>>>>> numbers add: 1. >>>>>>>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>>>>>>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: >>>>>>>> [:each >>>>>>>> | >>>>>>>> each negated] ). >>>>>>>> >>>>>>>> dict := LargeIdentityDictionary new. >>>>>>>> [numbers do: [:each | >>>>>>>> dict at: each put: dict size + 1. >>>>>>>> ]. >>>>>>>> ] timeToRun >>>>>>>> -> 12657 >>>>>>>> >>>>>>>> >>>>>>>> whereas with IdentityDictionary it is 37. I guess it could be >>>>>>>> related >>>>>>>> to >>>>>>>> #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it >>>>>>>> is >>>>>>>> also >>>>>>>> also there. >>>>>>>> I will try to take a look into it during the next week. >>>>>>>> >>>>>>>> Thanks in advance >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Nicolas >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks for any idea. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>>>>>>>>>> <stephane.ducasse(a)inria.fr> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>>>>>>>>>> >>>>>>>>>>>> Cool. One more thing: in Squeak the method using primitive 132 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> directly >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> expected. If >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> this was also added to Pharo, then the #pointsTo: sends should >>>>>>>>> be >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> changed to >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> #instVarsInclude:, otherwise Array can be reported as included >>>>>>>>> even >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> if it >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> wasn't added. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I'll upload my LargeIdentityDictionary implementation to the same >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> place >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> this evening, since it's still 2-3 factor faster than other >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> solutionts and >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> there seem to be demand for it. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>> Levente >>>>>>>>>>>> >>>>>>>>>>>> "in Squeak the method using primitive 132 directly was renamed >>>>>>>>>>>> to >>>>>>>>>>>> >>>>>>>>>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>>>>>>>>>>> >>>>>>>>>>>>> I do not get the following. Indeed pointTo: looks like >>>>>>>>>>>>> >>>>>>>>>>>> instVarsInclude: >>>>>>>>>>>> now I do not understand the rest of your paragraph. >>>>>>>>>>>> >>>>>>>>>>>> Stef >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Mariano >>>>>>>>>>> http://marianopeck.wordpress.******com<http://marianopeck.** >>>>>>>>>>> wordpress.com <http://marianopeck.wordpress.**com<http://marianopeck.wordpress.com> >>>>>>>>>>> >> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>> Mariano >>>>>>>> http://marianopeck.wordpress.******com < >>>>>>>> http://marianopeck.wordpress.**** >>>>>>>> com <http://marianopeck.wordpress.**com<http://marianopeck.wordpress.com> >>>>>>>> >> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>> -- >>>>> Mariano >>>>> http://marianopeck.wordpress.****com <http://marianopeck.wordpress.** >>>>> com <http://marianopeck.wordpress.com>> >>>>> >>>>> >>>>> >>>>> >>>> -- >>>> Mariano >>>> http://marianopeck.wordpress.****com <http://marianopeck.wordpress.** >>>> com <http://marianopeck.wordpress.com>> >>>> >>>> >>> >> >> -- >> Mariano >> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >> > -- Mariano http://marianopeck.wordpress.com

Dec. 18, 2011

Re: [Pharo-project] Anyone want more timezones in the base image?

by Sven Van Caekenberghe

Paul, On 16 Dec 2011, at 22:53, Paul DeBruicker wrote: > Hi - > > Because of how Internet Explorer reports javascript dates back to Seaside (as detected by Brian Murphy-Dye when using the FullCalendar jQuery plugin) I added the ability to determine the timezone based on the abbreviation IE reports. I used the list here: https://en.wikipedia.org/wiki/Time_zone_abbreviations to create an expanded version of > > TimeZone class>>#timeZones > > which encompasses those shown in the list on wikipedia. And added some convenience methods. > > Its not as useful or sophisticated as David Lewis's TimeZone database or Chronos, but its more than what's in the image now. > > If someone wants it I can make an issue and package my changes up for review. Right now I'm just including them as extensions to the JQWidgetBox-JQFullCalendar-Core package. > > > Thanks for whatever guidance you care to provide. > > > Paul DeBruicker I had a quick look at your code and it seems like a logical addition (given what TimeZone currently is and what you want to do: interpret tz abbreviations). I have 3 remarks/comments: - the list is currently a dynamic array that is recreated on every access, wouldn't a lazy initialized class variable be better ? - one other goal of a list of timezones is to present them to the user to pick one; for that TimeZone as it stands now is insufficient (there is no country/city mapping, which is the most user friendly way to pick one, IMHO). it seems a bit wastefull to have such a list and not be able to support that goal as well. - on Mac OS X and Linux, there is enough data in /usr/share/zoneinfo to dynamically load/parse from, if something equivalent existed on Windows, we could use that (and benefit from OS level updates to this DB); I know it is more work, I'm just thing out loud. Sven PS: there are very few users of TimeZone in the image ;-)

Dec. 18, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: > 2011/12/18 Levente Uzonyi <leves(a)elte.hu> > >> On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: >> >> Actually, the long test is >>> "Create some bitmaps" >>> >>> numbers := OrderedCollection new. >>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | >>> each negated] ). >>> >>> "Put them in a dictionary using dict as as key" >>> dict := LargeIdentityDictionary new. >>> numbers do: [:each | >>> dict at: each put: dict size + 1. >>> ]. >>> >>> "Check they are correct" >>> numbers doWithIndex: [:each :index | self assert: index == (dict at: >>> each)] >>> >> >> Fixed. >> >> > That was FAST :) > Thanks Levente. Where can I find the fix? It should be in the FuelExperiments repository, though I had to upload it one more time for some reason. Levente > > Cheers > > >> >> Levente >> >> >>> On Sun, Dec 18, 2011 at 7:17 PM, Mariano Martinez Peck < >>> marianopeck(a)gmail.com> wrote: >>> >>> >>>> >>>> Now I noticed that SmallIntegers storing is really slow >>>>>>> >>>>>>> >>>>>> This is not entirely true. Every hash table can be made slow if you >>>>>> know >>>>>> how it's hash function works. Since in your example all numbers are >>>>>> congruent to 1 modulo 4096 and the size of the array is also 4096, >>>>>> therefore they will be mapped to the same slot (1). Try using random >>>>>> (or at >>>>>> least more realistic) numbers in your test. If this is a real problem, >>>>>> then >>>>>> it may be necessary to implement a new hash method, which uses the >>>>>> primitive for general objects and does something else with >>>>>> SmallIntegers >>>>>> and that method shoul be used from these large collections. >>>>>> >>>>>> >>>>> I uploaded a new mcz to the FuelExperiments repository, which is >>>>> unaffected by the identity hash differences between Pharo and Squeak and >>>>> works better with SmallIntegers. It includes both the dictionary and the >>>>> set and (in theory) has slightly better performance for non-SmallInteger >>>>> objects. >>>>> >>>>> >>>>> Hi Levente. First let me say I really appreaciate your effort in make >>>> it >>>> work. It makes definitvly Fuel faster and it would make sense to >>>> integrate >>>> them also in Squeak and Pharo. >>>> Now, I found another bug. To reproduce it: >>>> >>>> | dict | >>>> dict := LargeIdentityDictionary new. >>>> dict at: 16941057 put: 1035. >>>> >>>> Now, it seems 16941057 largeIdentityHash answers 4096 and the comment of >>>> #largeIdentityHash says ""Return an integer between 0 and 4095 based on >>>> all of my bits. Make sure that for nearby receivers the result will not >>>> be >>>> nearby with high probability."" >>>> >>>> The thing is that LargeIdentityHashedCollection permuteHash: hash + 1 >>>> answers 4096, and hence, when in #at:put: you do: >>>> >>>> at: key put: value >>>> >>>> | hash | >>>> (keys at: (hash := key largeIdentityHash + 1)) >>>> >>>> then you get the out of bounds. >>>> >>>> Modyfing >>>> >>>> initialize >>>> >>>> | rng | >>>> rng := Random seed: 664399324. >>>> PermutationMap := (1 to: 4096) asArray shuffleBy: rng >>>> >>>> to 4095 make some tests to pass, but there are some other failing. >>>> >>>> I will try to continue to see if I find the error. >>>> >>>> Thanks! >>>> >>>> >>>> Levente >>>>> >>>>> >>>>> >>>>> >>>>>> Levente >>>>>> >>>>>> >>>>>> numbers := OrderedCollection new. >>>>>>> numbers add: 1. >>>>>>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>>>>>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each >>>>>>> | >>>>>>> each negated] ). >>>>>>> >>>>>>> dict := LargeIdentityDictionary new. >>>>>>> [numbers do: [:each | >>>>>>> dict at: each put: dict size + 1. >>>>>>> ]. >>>>>>> ] timeToRun >>>>>>> -> 12657 >>>>>>> >>>>>>> >>>>>>> whereas with IdentityDictionary it is 37. I guess it could be related >>>>>>> to >>>>>>> #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it is >>>>>>> also >>>>>>> also there. >>>>>>> I will try to take a look into it during the next week. >>>>>>> >>>>>>> Thanks in advance >>>>>>> >>>>>>> >>>>>>> >>>>>>> Nicolas >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks for any idea. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>>>>>>>>> <stephane.ducasse(a)inria.fr> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>>>>>>>>> >>>>>>>>>>> Cool. One more thing: in Squeak the method using primitive 132 >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> directly >>>>>>>>>>> >>>>>>>>>> >>>>>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> expected. If >>>>>>>>>>> >>>>>>>>>> >>>>>>>> this was also added to Pharo, then the #pointsTo: sends should be >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> changed to >>>>>>>>>>> >>>>>>>>>> >>>>>>>> #instVarsInclude:, otherwise Array can be reported as included even >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> if it >>>>>>>>>>> >>>>>>>>>> >>>>>>>> wasn't added. >>>>>>>>> >>>>>>>>>> I'll upload my LargeIdentityDictionary implementation to the same >>>>>>>>>>>> >>>>>>>>>>>> place >>>>>>>>>>> >>>>>>>>>> >>>>>>>> this evening, since it's still 2-3 factor faster than other >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> solutionts and >>>>>>>>>>> >>>>>>>>>> >>>>>>>> there seem to be demand for it. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> Levente >>>>>>>>>>> >>>>>>>>>>> "in Squeak the method using primitive 132 directly was renamed to >>>>>>>>>>> >>>>>>>>>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>>>>>>>>>> >>>>>>>>>>>> I do not get the following. Indeed pointTo: looks like >>>>>>>>>>> instVarsInclude: >>>>>>>>>>> now I do not understand the rest of your paragraph. >>>>>>>>>>> >>>>>>>>>>> Stef >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Mariano >>>>>>>>>> http://marianopeck.wordpress.****com<http://marianopeck.** >>>>>>>>>> wordpress.com <http://marianopeck.wordpress.com>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> Mariano >>>>>>> http://marianopeck.wordpress.****com <http://marianopeck.wordpress.** >>>>>>> com <http://marianopeck.wordpress.com>> >>>>>>> >>>>>>> >>>>>> >>>> >>>> -- >>>> Mariano >>>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>>> >>>> >>>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 18, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

2011/12/18 Levente Uzonyi <leves(a)elte.hu> > On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: > > Actually, the long test is >> "Create some bitmaps" >> >> numbers := OrderedCollection new. >> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | >> each negated] ). >> >> "Put them in a dictionary using dict as as key" >> dict := LargeIdentityDictionary new. >> numbers do: [:each | >> dict at: each put: dict size + 1. >> ]. >> >> "Check they are correct" >> numbers doWithIndex: [:each :index | self assert: index == (dict at: >> each)] >> > > Fixed. > > That was FAST :) Thanks Levente. Where can I find the fix? Cheers > > Levente > > >> On Sun, Dec 18, 2011 at 7:17 PM, Mariano Martinez Peck < >> marianopeck(a)gmail.com> wrote: >> >> >>> >>> Now I noticed that SmallIntegers storing is really slow >>>>>> >>>>>> >>>>> This is not entirely true. Every hash table can be made slow if you >>>>> know >>>>> how it's hash function works. Since in your example all numbers are >>>>> congruent to 1 modulo 4096 and the size of the array is also 4096, >>>>> therefore they will be mapped to the same slot (1). Try using random >>>>> (or at >>>>> least more realistic) numbers in your test. If this is a real problem, >>>>> then >>>>> it may be necessary to implement a new hash method, which uses the >>>>> primitive for general objects and does something else with >>>>> SmallIntegers >>>>> and that method shoul be used from these large collections. >>>>> >>>>> >>>> I uploaded a new mcz to the FuelExperiments repository, which is >>>> unaffected by the identity hash differences between Pharo and Squeak and >>>> works better with SmallIntegers. It includes both the dictionary and the >>>> set and (in theory) has slightly better performance for non-SmallInteger >>>> objects. >>>> >>>> >>>> Hi Levente. First let me say I really appreaciate your effort in make >>> it >>> work. It makes definitvly Fuel faster and it would make sense to >>> integrate >>> them also in Squeak and Pharo. >>> Now, I found another bug. To reproduce it: >>> >>> | dict | >>> dict := LargeIdentityDictionary new. >>> dict at: 16941057 put: 1035. >>> >>> Now, it seems 16941057 largeIdentityHash answers 4096 and the comment of >>> #largeIdentityHash says ""Return an integer between 0 and 4095 based on >>> all of my bits. Make sure that for nearby receivers the result will not >>> be >>> nearby with high probability."" >>> >>> The thing is that LargeIdentityHashedCollection permuteHash: hash + 1 >>> answers 4096, and hence, when in #at:put: you do: >>> >>> at: key put: value >>> >>> | hash | >>> (keys at: (hash := key largeIdentityHash + 1)) >>> >>> then you get the out of bounds. >>> >>> Modyfing >>> >>> initialize >>> >>> | rng | >>> rng := Random seed: 664399324. >>> PermutationMap := (1 to: 4096) asArray shuffleBy: rng >>> >>> to 4095 make some tests to pass, but there are some other failing. >>> >>> I will try to continue to see if I find the error. >>> >>> Thanks! >>> >>> >>> Levente >>>> >>>> >>>> >>>> >>>>> Levente >>>>> >>>>> >>>>> numbers := OrderedCollection new. >>>>>> numbers add: 1. >>>>>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>>>>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each >>>>>> | >>>>>> each negated] ). >>>>>> >>>>>> dict := LargeIdentityDictionary new. >>>>>> [numbers do: [:each | >>>>>> dict at: each put: dict size + 1. >>>>>> ]. >>>>>> ] timeToRun >>>>>> -> 12657 >>>>>> >>>>>> >>>>>> whereas with IdentityDictionary it is 37. I guess it could be related >>>>>> to >>>>>> #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it is >>>>>> also >>>>>> also there. >>>>>> I will try to take a look into it during the next week. >>>>>> >>>>>> Thanks in advance >>>>>> >>>>>> >>>>>> >>>>>> Nicolas >>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks for any idea. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>>>>>>>> <stephane.ducasse(a)inria.fr> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>>>>>>>> >>>>>>>>>> Cool. One more thing: in Squeak the method using primitive 132 >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> directly >>>>>>>>>> >>>>>>>>> >>>>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as >>>>>>>> >>>>>>>>> >>>>>>>>>>> expected. If >>>>>>>>>> >>>>>>>>> >>>>>>> this was also added to Pharo, then the #pointsTo: sends should be >>>>>>>> >>>>>>>>> >>>>>>>>>>> changed to >>>>>>>>>> >>>>>>>>> >>>>>>> #instVarsInclude:, otherwise Array can be reported as included even >>>>>>>> >>>>>>>>> >>>>>>>>>>> if it >>>>>>>>>> >>>>>>>>> >>>>>>> wasn't added. >>>>>>>> >>>>>>>>> I'll upload my LargeIdentityDictionary implementation to the same >>>>>>>>>>> >>>>>>>>>>> place >>>>>>>>>> >>>>>>>>> >>>>>>> this evening, since it's still 2-3 factor faster than other >>>>>>>> >>>>>>>>> >>>>>>>>>>> solutionts and >>>>>>>>>> >>>>>>>>> >>>>>>> there seem to be demand for it. >>>>>>>> >>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Levente >>>>>>>>>> >>>>>>>>>> "in Squeak the method using primitive 132 directly was renamed to >>>>>>>>>> >>>>>>>>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>>>>>>>>> >>>>>>>>>>> I do not get the following. Indeed pointTo: looks like >>>>>>>>>> instVarsInclude: >>>>>>>>>> now I do not understand the rest of your paragraph. >>>>>>>>>> >>>>>>>>>> Stef >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Mariano >>>>>>>>> http://marianopeck.wordpress.****com<http://marianopeck.** >>>>>>>>> wordpress.com <http://marianopeck.wordpress.com>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>>> Mariano >>>>>> http://marianopeck.wordpress.****com <http://marianopeck.wordpress.** >>>>>> com <http://marianopeck.wordpress.com>> >>>>>> >>>>>> >>>>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >>> >>> >> >> -- >> Mariano >> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >> > -- Mariano http://marianopeck.wordpress.com

Dec. 18, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Levente Uzonyi

On Sun, 18 Dec 2011, Mariano Martinez Peck wrote: > Actually, the long test is > "Create some bitmaps" > > numbers := OrderedCollection new. > numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. > numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | > each negated] ). > > "Put them in a dictionary using dict as as key" > dict := LargeIdentityDictionary new. > numbers do: [:each | > dict at: each put: dict size + 1. > ]. > > "Check they are correct" > numbers doWithIndex: [:each :index | self assert: index == (dict at: > each)] Fixed. Levente > > On Sun, Dec 18, 2011 at 7:17 PM, Mariano Martinez Peck < > marianopeck(a)gmail.com> wrote: > >> >> >>>>> Now I noticed that SmallIntegers storing is really slow >>>>> >>>> >>>> This is not entirely true. Every hash table can be made slow if you know >>>> how it's hash function works. Since in your example all numbers are >>>> congruent to 1 modulo 4096 and the size of the array is also 4096, >>>> therefore they will be mapped to the same slot (1). Try using random (or at >>>> least more realistic) numbers in your test. If this is a real problem, then >>>> it may be necessary to implement a new hash method, which uses the >>>> primitive for general objects and does something else with SmallIntegers >>>> and that method shoul be used from these large collections. >>>> >>> >>> I uploaded a new mcz to the FuelExperiments repository, which is >>> unaffected by the identity hash differences between Pharo and Squeak and >>> works better with SmallIntegers. It includes both the dictionary and the >>> set and (in theory) has slightly better performance for non-SmallInteger >>> objects. >>> >>> >> Hi Levente. First let me say I really appreaciate your effort in make it >> work. It makes definitvly Fuel faster and it would make sense to integrate >> them also in Squeak and Pharo. >> Now, I found another bug. To reproduce it: >> >> | dict | >> dict := LargeIdentityDictionary new. >> dict at: 16941057 put: 1035. >> >> Now, it seems 16941057 largeIdentityHash answers 4096 and the comment of >> #largeIdentityHash says ""Return an integer between 0 and 4095 based on >> all of my bits. Make sure that for nearby receivers the result will not be >> nearby with high probability."" >> >> The thing is that LargeIdentityHashedCollection permuteHash: hash + 1 >> answers 4096, and hence, when in #at:put: you do: >> >> at: key put: value >> >> | hash | >> (keys at: (hash := key largeIdentityHash + 1)) >> >> then you get the out of bounds. >> >> Modyfing >> >> initialize >> >> | rng | >> rng := Random seed: 664399324. >> PermutationMap := (1 to: 4096) asArray shuffleBy: rng >> >> to 4095 make some tests to pass, but there are some other failing. >> >> I will try to continue to see if I find the error. >> >> Thanks! >> >> >>> Levente >>> >>> >>> >>>> >>>> Levente >>>> >>>> >>>>> numbers := OrderedCollection new. >>>>> numbers add: 1. >>>>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>>>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | >>>>> each negated] ). >>>>> >>>>> dict := LargeIdentityDictionary new. >>>>> [numbers do: [:each | >>>>> dict at: each put: dict size + 1. >>>>> ]. >>>>> ] timeToRun >>>>> -> 12657 >>>>> >>>>> >>>>> whereas with IdentityDictionary it is 37. I guess it could be related >>>>> to >>>>> #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it is >>>>> also >>>>> also there. >>>>> I will try to take a look into it during the next week. >>>>> >>>>> Thanks in advance >>>>> >>>>> >>>>> >>>>> Nicolas >>>>>> >>>>>> >>>>>>>> Thanks for any idea. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>>>>>>> <stephane.ducasse(a)inria.fr> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>>>>>>> >>>>>>>>> Cool. One more thing: in Squeak the method using primitive 132 >>>>>>>>>> >>>>>>>>> directly >>>>>> >>>>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as >>>>>>>>>> >>>>>>>>> expected. If >>>>>> >>>>>>> this was also added to Pharo, then the #pointsTo: sends should be >>>>>>>>>> >>>>>>>>> changed to >>>>>> >>>>>>> #instVarsInclude:, otherwise Array can be reported as included even >>>>>>>>>> >>>>>>>>> if it >>>>>> >>>>>>> wasn't added. >>>>>>>>>> I'll upload my LargeIdentityDictionary implementation to the same >>>>>>>>>> >>>>>>>>> place >>>>>> >>>>>>> this evening, since it's still 2-3 factor faster than other >>>>>>>>>> >>>>>>>>> solutionts and >>>>>> >>>>>>> there seem to be demand for it. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Levente >>>>>>>>> >>>>>>>>> "in Squeak the method using primitive 132 directly was renamed to >>>>>>>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>>>>>>>> >>>>>>>>> I do not get the following. Indeed pointTo: looks like >>>>>>>>> instVarsInclude: >>>>>>>>> now I do not understand the rest of your paragraph. >>>>>>>>> >>>>>>>>> Stef >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Mariano >>>>>>>> http://marianopeck.wordpress.**com<http://marianopeck.wordpress.com> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Mariano >>>>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>>>> >>>> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 18, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

Actually, the long test is "Create some bitmaps" numbers := OrderedCollection new. numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | each negated] ). "Put them in a dictionary using dict as as key" dict := LargeIdentityDictionary new. numbers do: [:each | dict at: each put: dict size + 1. ]. "Check they are correct" numbers doWithIndex: [:each :index | self assert: index == (dict at: each)] On Sun, Dec 18, 2011 at 7:17 PM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > > >>>> Now I noticed that SmallIntegers storing is really slow >>>> >>> >>> This is not entirely true. Every hash table can be made slow if you know >>> how it's hash function works. Since in your example all numbers are >>> congruent to 1 modulo 4096 and the size of the array is also 4096, >>> therefore they will be mapped to the same slot (1). Try using random (or at >>> least more realistic) numbers in your test. If this is a real problem, then >>> it may be necessary to implement a new hash method, which uses the >>> primitive for general objects and does something else with SmallIntegers >>> and that method shoul be used from these large collections. >>> >> >> I uploaded a new mcz to the FuelExperiments repository, which is >> unaffected by the identity hash differences between Pharo and Squeak and >> works better with SmallIntegers. It includes both the dictionary and the >> set and (in theory) has slightly better performance for non-SmallInteger >> objects. >> >> > Hi Levente. First let me say I really appreaciate your effort in make it > work. It makes definitvly Fuel faster and it would make sense to integrate > them also in Squeak and Pharo. > Now, I found another bug. To reproduce it: > > | dict | > dict := LargeIdentityDictionary new. > dict at: 16941057 put: 1035. > > Now, it seems 16941057 largeIdentityHash answers 4096 and the comment of > #largeIdentityHash says ""Return an integer between 0 and 4095 based on > all of my bits. Make sure that for nearby receivers the result will not be > nearby with high probability."" > > The thing is that LargeIdentityHashedCollection permuteHash: hash + 1 > answers 4096, and hence, when in #at:put: you do: > > at: key put: value > > | hash | > (keys at: (hash := key largeIdentityHash + 1)) > > then you get the out of bounds. > > Modyfing > > initialize > > | rng | > rng := Random seed: 664399324. > PermutationMap := (1 to: 4096) asArray shuffleBy: rng > > to 4095 make some tests to pass, but there are some other failing. > > I will try to continue to see if I find the error. > > Thanks! > > >> Levente >> >> >> >>> >>> Levente >>> >>> >>>> numbers := OrderedCollection new. >>>> numbers add: 1. >>>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | >>>> each negated] ). >>>> >>>> dict := LargeIdentityDictionary new. >>>> [numbers do: [:each | >>>> dict at: each put: dict size + 1. >>>> ]. >>>> ] timeToRun >>>> -> 12657 >>>> >>>> >>>> whereas with IdentityDictionary it is 37. I guess it could be related >>>> to >>>> #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it is >>>> also >>>> also there. >>>> I will try to take a look into it during the next week. >>>> >>>> Thanks in advance >>>> >>>> >>>> >>>> Nicolas >>>>> >>>>> >>>>>>> Thanks for any idea. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>>>>>> <stephane.ducasse(a)inria.fr> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>>>>>> >>>>>>>> Cool. One more thing: in Squeak the method using primitive 132 >>>>>>>>> >>>>>>>> directly >>>>> >>>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as >>>>>>>>> >>>>>>>> expected. If >>>>> >>>>>> this was also added to Pharo, then the #pointsTo: sends should be >>>>>>>>> >>>>>>>> changed to >>>>> >>>>>> #instVarsInclude:, otherwise Array can be reported as included even >>>>>>>>> >>>>>>>> if it >>>>> >>>>>> wasn't added. >>>>>>>>> I'll upload my LargeIdentityDictionary implementation to the same >>>>>>>>> >>>>>>>> place >>>>> >>>>>> this evening, since it's still 2-3 factor faster than other >>>>>>>>> >>>>>>>> solutionts and >>>>> >>>>>> there seem to be demand for it. >>>>>>>>> >>>>>>>> >>>>>>>> Levente >>>>>>>> >>>>>>>> "in Squeak the method using primitive 132 directly was renamed to >>>>>>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>>>>>>> >>>>>>>> I do not get the following. Indeed pointTo: looks like >>>>>>>> instVarsInclude: >>>>>>>> now I do not understand the rest of your paragraph. >>>>>>>> >>>>>>>> Stef >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Mariano >>>>>>> http://marianopeck.wordpress.**com<http://marianopeck.wordpress.com> >>>>>>> >>>>>>> >>>>> >>>>> >>>> >>>> -- >>>> Mariano >>>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>>> >>> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

Dec. 18, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

> >>> Now I noticed that SmallIntegers storing is really slow >>> >> >> This is not entirely true. Every hash table can be made slow if you know >> how it's hash function works. Since in your example all numbers are >> congruent to 1 modulo 4096 and the size of the array is also 4096, >> therefore they will be mapped to the same slot (1). Try using random (or at >> least more realistic) numbers in your test. If this is a real problem, then >> it may be necessary to implement a new hash method, which uses the >> primitive for general objects and does something else with SmallIntegers >> and that method shoul be used from these large collections. >> > > I uploaded a new mcz to the FuelExperiments repository, which is > unaffected by the identity hash differences between Pharo and Squeak and > works better with SmallIntegers. It includes both the dictionary and the > set and (in theory) has slightly better performance for non-SmallInteger > objects. > > Hi Levente. First let me say I really appreaciate your effort in make it work. It makes definitvly Fuel faster and it would make sense to integrate them also in Squeak and Pharo. Now, I found another bug. To reproduce it: | dict | dict := LargeIdentityDictionary new. dict at: 16941057 put: 1035. Now, it seems 16941057 largeIdentityHash answers 4096 and the comment of #largeIdentityHash says ""Return an integer between 0 and 4095 based on all of my bits. Make sure that for nearby receivers the result will not be nearby with high probability."" The thing is that LargeIdentityHashedCollection permuteHash: hash + 1 answers 4096, and hence, when in #at:put: you do: at: key put: value | hash | (keys at: (hash := key largeIdentityHash + 1)) then you get the out of bounds. Modyfing initialize | rng | rng := Random seed: 664399324. PermutationMap := (1 to: 4096) asArray shuffleBy: rng to 4095 make some tests to pass, but there are some other failing. I will try to continue to see if I find the error. Thanks! > Levente > > > >> >> Levente >> >> >>> numbers := OrderedCollection new. >>> numbers add: 1. >>> numbers addAll: (1 to: 1 << 29 by: 1 << 14) asArray. >>> numbers addAll: ((1 to: 1 << 29 by: 1 << 14) asArray collect: [:each | >>> each negated] ). >>> >>> dict := LargeIdentityDictionary new. >>> [numbers do: [:each | >>> dict at: each put: dict size + 1. >>> ]. >>> ] timeToRun >>> -> 12657 >>> >>> >>> whereas with IdentityDictionary it is 37. I guess it could be related to >>> #identityHash or #basicIdentityHash. I tried in Squeak 4.3 and it is >>> also >>> also there. >>> I will try to take a look into it during the next week. >>> >>> Thanks in advance >>> >>> >>> >>> Nicolas >>>> >>>> >>>>>> Thanks for any idea. >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >>>>>> <stephane.ducasse(a)inria.fr> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>>>>>> >>>>>>> Cool. One more thing: in Squeak the method using primitive 132 >>>>>>>> >>>>>>> directly >>>> >>>>> was renamed to #instVarsInclude:, so now #pointsTo: works as >>>>>>>> >>>>>>> expected. If >>>> >>>>> this was also added to Pharo, then the #pointsTo: sends should be >>>>>>>> >>>>>>> changed to >>>> >>>>> #instVarsInclude:, otherwise Array can be reported as included even >>>>>>>> >>>>>>> if it >>>> >>>>> wasn't added. >>>>>>>> I'll upload my LargeIdentityDictionary implementation to the same >>>>>>>> >>>>>>> place >>>> >>>>> this evening, since it's still 2-3 factor faster than other >>>>>>>> >>>>>>> solutionts and >>>> >>>>> there seem to be demand for it. >>>>>>>> >>>>>>> >>>>>>> Levente >>>>>>> >>>>>>> "in Squeak the method using primitive 132 directly was renamed to >>>>>>>> #instVarsInclude:, so now #pointsTo: works as expected." >>>>>>>> >>>>>>> I do not get the following. Indeed pointTo: looks like >>>>>>> instVarsInclude: >>>>>>> now I do not understand the rest of your paragraph. >>>>>>> >>>>>>> Stef >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Mariano >>>>>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>>>>> >>>>>> >>>> >>>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >> -- Mariano http://marianopeck.wordpress.com

Dec. 18, 2011