- Pharo-dev - lists.pharo.org

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Nicolas Cellier

2011/12/17 Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com>: > 2011/12/17 Mariano Martinez Peck <marianopeck(a)gmail.com>: >> Hi Levente. I take a deeper look to see why LargeIdentityDictionary is not >> working as expected for us. >> >> The way we are putting objects in the Dict is always this way: >> >> registerIndexesOn: aDictionary >> >> Â Â Â self objects do: [ :instance | aDictionary at: instance put: aDictionary >> size + 1 ]. >> >> >> Soâ¦keys are the objects of the graphs and the values are the internal >> position in the serialization.Â However, when I run a serialization it looks >> like if the follow happens that when we are doing: >> >>>> encodeReferenceTo: anObject >> >> Â Â Â indexCluster >> Â Â Â Â Â Â serialize: (objectsIndexes at: anObject) >> Â Â Â Â Â Â on: stream >> >> >> (objectsIndexes at: anObject) == anObjectÂ Â ->Â true >> >> soÂ (objectsIndexes at: anObject)Â is answering "self" rather than the >> number (which was the dict size + 1 at the serialization moment). >> >> Something interesting is that most tests seem to pass and the problem is >> with benchmarks (where graphs are much bigger), so maybe it could be >> something related to that.Â For example, the following benchmark for bitmaps >> is failing: >> >> Â Â Â Â Â Â Â Â numbers := OrderedCollection new. >> Â Â Â bitmaps := OrderedCollection new. >> Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. >> Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. >> Â Â Â ^ bitmaps >> >> it happens the mentioned problem when trying to serialize the collection >> 'bitmaps'. >> >> >> I tried to reproduce it outside Fuel and I think I succeeded (not sure). The >> following does work with IdentityDictionary but not with >> LargeIdentityDictionary. Basically I am simulating the way we use this dict >> in Fuel: >> >> testLargeIdentityDictionary >> >> Â Â Â | numbers bitmaps dict | >> >> "Create some bitmaps" >> >> Â Â Â numbers := OrderedCollection new. >> Â Â Â bitmaps := OrderedCollection new. >> Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. >> Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. >> >> "Put them in a dictionary using dict as as key" >> Â Â Â dict := LargeIdentityDictionary new. >> Â Â Â bitmaps do: [:each | >> Â Â Â Â Â Â Â dict at: each put: dict size + 1. >> Â Â Â Â Â Â ]. >> >> "Check they are correct" >> Â Â Â bitmaps doWithIndex: [:each :index | self assert: index == (dict at: >> each)] >> >> Of course that test works with IdentityDictionary. >> > > Can't you debug it ? > It took 2 minutes to find this sentence at line 19 of LargeDictionary>>at:put: (values at: hash) at: newIndex put: key Can you replace key with value and retry the tests ? Nicolas >> >> Thanks for any idea. >> >> >> >> On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse >> <stephane.ducasse(a)inria.fr> wrote: >>> >>> >>> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >>> >>> > Cool. One more thing: in Squeak the method using primitive 132 directly >>> > was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> > this was also added to Pharo, then the #pointsTo: sends should be changed to >>> > #instVarsInclude:, otherwise Array can be reported as included even if it >>> > wasn't added. >>> > I'll upload my LargeIdentityDictionary implementation to the same place >>> > this evening, since it's still 2-3 factor faster than other solutionts and >>> > there seem to be demand for it. >>> >>> Levente >>> >>> > "in Squeak the method using primitive 132 directly was renamed to >>> > #instVarsInclude:, so now #pointsTo: works as expected." >>> I do not get the following. Indeed pointTo: looks like instVarsInclude: >>> now I do not understand the rest of your paragraph. >>> >>> Stef >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >>

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Nicolas Cellier

2011/12/17 Mariano Martinez Peck <marianopeck(a)gmail.com>: > Hi Levente. I take a deeper look to see why LargeIdentityDictionary is not > working as expected for us. > > The way we are putting objects in the Dict is always this way: > > registerIndexesOn: aDictionary > > Â Â Â self objects do: [ :instance | aDictionary at: instance put: aDictionary > size + 1 ]. > > > Soâ¦keys are the objects of the graphs and the values are the internal > position in the serialization.Â However, when I run a serialization it looks > like if the follow happens that when we are doing: > >>> encodeReferenceTo: anObject > > Â Â Â indexCluster > Â Â Â Â Â Â serialize: (objectsIndexes at: anObject) > Â Â Â Â Â Â on: stream > > > (objectsIndexes at: anObject) == anObjectÂ Â ->Â true > > soÂ (objectsIndexes at: anObject)Â is answering "self" rather than the > number (which was the dict size + 1 at the serialization moment). > > Something interesting is that most tests seem to pass and the problem is > with benchmarks (where graphs are much bigger), so maybe it could be > something related to that.Â For example, the following benchmark for bitmaps > is failing: > > Â Â Â Â Â Â Â Â numbers := OrderedCollection new. > Â Â Â bitmaps := OrderedCollection new. > Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. > Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. > Â Â Â ^ bitmaps > > it happens the mentioned problem when trying to serialize the collection > 'bitmaps'. > > > I tried to reproduce it outside Fuel and I think I succeeded (not sure). The > following does work with IdentityDictionary but not with > LargeIdentityDictionary. Basically I am simulating the way we use this dict > in Fuel: > > testLargeIdentityDictionary > > Â Â Â | numbers bitmaps dict | > > "Create some bitmaps" > > Â Â Â numbers := OrderedCollection new. > Â Â Â bitmaps := OrderedCollection new. > Â Â Â numbers addAll:Â ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. > Â Â Â numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. > > "Put them in a dictionary using dict as as key" > Â Â Â dict := LargeIdentityDictionary new. > Â Â Â bitmaps do: [:each | > Â Â Â Â Â Â Â dict at: each put: dict size + 1. > Â Â Â Â Â Â ]. > > "Check they are correct" > Â Â Â bitmaps doWithIndex: [:each :index | self assert: index == (dict at: > each)] > > Of course that test works with IdentityDictionary. > Can't you debug it ? > > Thanks for any idea. > > > > On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse > <stephane.ducasse(a)inria.fr> wrote: >> >> >> On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: >> >> > Cool. One more thing: in Squeak the method using primitive 132 directly >> > was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >> > this was also added to Pharo, then the #pointsTo: sends should be changed to >> > #instVarsInclude:, otherwise Array can be reported as included even if it >> > wasn't added. >> > I'll upload my LargeIdentityDictionary implementation to the same place >> > this evening, since it's still 2-3 factor faster than other solutionts and >> > there seem to be demand for it. >> >> Levente >> >> > "in Squeak the method using primitive 132 directly was renamed to >> > #instVarsInclude:, so now #pointsTo: works as expected." >> I do not get the following. Indeed pointTo: looks like instVarsInclude: >> now I do not understand the rest of your paragraph. >> >> Stef > > > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 17, 2011

Re: [Pharo-project] NB Bug?

by Igor Stasenko

On 16 December 2011 13:39, Norbert Hartl <norbert(a)hartl.name> wrote: > > Am 16.12.2011 um 11:47 schrieb Igor Stasenko: > >> On 16 December 2011 11:02, Stefan Marr <smalltalk(a)stefan-marr.de> wrote: >>> >>> On 16 Dec 2011, at 10:49, Igor Stasenko wrote: >>>>> >>>> oh please. that's even less obvious usage :) >>>> there's a lot of different stuff in C syntax which you can play with. >>>> But it doesn't means that you need to routinely support >>>> everything in FFI callout specs. >>>> >>>> as i said, the function signature syntax are close to C syntax, but >>>> not strictly C. >>> >>> Igor, what is your goal with that? >>> Have something working, or have something usable? >>> >>> If your goal is to have something usable, then your direct competitor is LuaJIT2's FFI >>> http://luajit.org/ext_ffi.html >>> and you should show similar ambitions. (Note specifically the 'cut&past part') >>> Otherwise, it will always remain just a nice try, an experiment that never got finished/productized. >>> >>> My understanding is that your boss got other plans for Pharo... >>> >> well, there are things which i would like to have, like integrating >> with Cog (so a call to generated native code >> will be performed directly , instead via primitive), non-blocking and callbacks. >> This is much more important to have than subtleties, how to define a >> function which takes no arguments by using Â '()' or '(void)'.. >> Kind of 'triangle has three corners' vs 'triangle has three edges' discussion. >> > Igor, > > importance lies in the eye of the beholder. You are doing great stuff and you don't want to waste your time with things you consider unnecessary. But you need to decide what is the goal of NativeBoost. If you like to keep the pace and realize cool demos then you shouldn't listen to other people. If you think you like to make it an accepted piece of software that has a lot of users then you need listen to them. People posting in this thread are your potential users. They tell you things the fail over. If this happens more often you'll loose them because they don't want to waste their time with non-working stuff (the term non-working also lies in the eye of the beholder). Basically it is that easy. If you don't care about the (probably stupid) wishes of your users it stays your private pet project. > Yes, i understand what you saying about. :) > Norbert -- Best regards, Igor Stasenko.

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

Hi Levente. I take a deeper look to see why LargeIdentityDictionary is not working as expected for us. The way we are putting objects in the Dict is always this way: registerIndexesOn: aDictionary self objects do: [ :instance | aDictionary at: instance put: aDictionary size + 1 ]. Soâ¦keys are the objects of the graphs and the values are the internal position in the serialization. However, when I run a serialization it looks like if the follow happens that when we are doing: >> encodeReferenceTo: anObject indexCluster serialize: (objectsIndexes at: anObject) on: stream (objectsIndexes at: anObject) == anObject -> true so (objectsIndexes at: anObject) is answering "self" rather than the number (which was the dict size + 1 at the serialization moment). Something interesting is that most tests seem to pass and the problem is with benchmarks (where graphs are much bigger), so maybe it could be something related to that. For example, the following benchmark for bitmaps is failing: numbers := OrderedCollection new. bitmaps := OrderedCollection new. numbers addAll: ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. ^ bitmaps it happens the mentioned problem when trying to serialize the collection 'bitmaps'. I tried to reproduce it outside Fuel and I think I succeeded (not sure). The following does work with IdentityDictionary but not with LargeIdentityDictionary. Basically I am simulating the way we use this dict in Fuel: testLargeIdentityDictionary | numbers bitmaps dict | "Create some bitmaps" numbers := OrderedCollection new. bitmaps := OrderedCollection new. numbers addAll: ( 1 << 30 to:( ( 1 << 30) + 10000) ) asArray. numbers do: [:each | bitmaps add: (Bitmap with: each with: each + 1) ]. "Put them in a dictionary using dict as as key" dict := LargeIdentityDictionary new. bitmaps do: [:each | dict at: each put: dict size + 1. ]. "Check they are correct" bitmaps doWithIndex: [:each :index | self assert: index == (dict at: each)] Of course that test works with IdentityDictionary. Thanks for any idea. On Sat, Dec 17, 2011 at 1:50 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > wrote: > > On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: > > > Cool. One more thing: in Squeak the method using primitive 132 directly > was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If > this was also added to Pharo, then the #pointsTo: sends should be changed > to #instVarsInclude:, otherwise Array can be reported as included even if > it wasn't added. > > I'll upload my LargeIdentityDictionary implementation to the same place > this evening, since it's still 2-3 factor faster than other solutionts and > there seem to be demand for it. > > Levente > > > "in Squeak the method using primitive 132 directly was renamed to > #instVarsInclude:, so now #pointsTo: works as expected." > I do not get the following. Indeed pointTo: looks like instVarsInclude: > now I do not understand the rest of your paragraph. > > Stef > -- Mariano http://marianopeck.wordpress.com

Dec. 17, 2011

Re: [Pharo-project] Instant search

by Benjamin

Should be on the wish list for this new year :) Ben On Dec 17, 2011, at 1:25 PM, StÃ©phane Ducasse wrote: > Nautilus is already better but it relies on rpackage and I got burnt and stopped to work on it sadly. > > >> Hi guys, >> >> What if you enhance search in System Browser so that results will be >> shown instantly and *in the same browser* (not in new window)? That is, >> classes matching criteria and their categories are shown instantly as >> you type in search line. Or with some small delay, like 0.5s. >> >> Or: to drop down results instantly as you type, as you have on Google >> search. After you select some choice, this class is shown immediately in >> the same browser window. >> >> This would be a nice UX improvement for 1.4, in my opinion :) >> >> Of course back button and tabs would be really a big step forward ... >> >> Best regards >> Janko >> >> >> -- >> Janko MivÅ¡ek >> Aida/Web >> Smalltalk Web Application Server >> http://www.aidaweb.si >> > >

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by StÃ©phane Ducasse

On Dec 16, 2011, at 3:28 PM, Levente Uzonyi wrote: > Cool. One more thing: in Squeak the method using primitive 132 directly was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If this was also added to Pharo, then the #pointsTo: sends should be changed to #instVarsInclude:, otherwise Array can be reported as included even if it wasn't added. > I'll upload my LargeIdentityDictionary implementation to the same place this evening, since it's still 2-3 factor faster than other solutionts and there seem to be demand for it. Levente > "in Squeak the method using primitive 132 directly was renamed to #instVarsInclude:, so now #pointsTo: works as expected." I do not get the following. Indeed pointTo: looks like instVarsInclude: now I do not understand the rest of your paragraph. Stef

Dec. 17, 2011

Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?

by StÃ©phane Ducasse

I would love to have that! Stef > > > I also have a LargeIdentityDictionary, which is relatively fast, but not as fast as LargeIdentitySet, because (for some unknown reason) we don't have a primitive that could support it. If we had a primitive like primitive 132 which would return the index of the element if found or 0 if not, then we could have a really fast LargeIdentityDictionary. > > > Levente > >> >> Cheers, >> Henry >> >> >

Dec. 17, 2011

Re: [Pharo-project] NB Bug?

by Serge Stinckwich

2011/12/16 Sean P. DeNigris <sean(a)clipperadams.com>: > > Stefan Marr-3 wrote >> >> LuaJIT2's FFI... specifically the 'cut&past part' >> > > Wow, that's awesome! Why would it ever be any harder than that? Yes really simple to use ! -- Serge Stinckwich UMI UMMISCO 209 (IRD/UPMC), Hanoi, Vietnam Matsuno Laboratory, Kyoto University, Japan (until 12/2011) http://www.mechatronics.me.kyoto-u.ac.jp/ Every DSL ends up being Smalltalk http://doesnotunderstand.org/

Dec. 17, 2011

Re: [Pharo-project] [COTDC] 97 - CRCError

by StÃ©phane Ducasse

you have also the monty python flying circus start with the guy coming out of the see :) Stef On Dec 16, 2011, at 8:29 PM, Sean P. DeNigris wrote: > Hello? Hello? Hello? > Is there anybody in there? > Just nod if you can hear me. > Is there anyone at home? > > http://www.youtube.com/watch?v=ZB1cNkC71vE > > -- > View this message in context: http://forum.world.st/COTDC-97-CRCError-tp4197055p4205923.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >

Dec. 17, 2011

Re: [Pharo-project] NB Bug?

by StÃ©phane Ducasse

On Dec 16, 2011, at 2:05 PM, Sean P. DeNigris wrote: > > Stefan Marr-3 wrote >> >> LuaJIT2's FFI... specifically the 'cut&past part' >> > > Wow, that's awesome! Why would it ever be any harder than that? before starting pharo I was wondering if I should not develop a smalltalk on top of lua. Somedays I think that I should have. Because we still do not have a vm that can be embedded in C and all the restâ¦. :( Stef > -- > View this message in context: http://forum.world.st/NB-Bug-tp4192637p4204556.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >

Dec. 17, 2011