November 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] how should Pier file structures be serialized with Fuel?

by Yanni Chiu

On 17/11/11 1:27 AM, Martin Dias wrote: > > you can implement : > > YourPRFileProxy >> fuelAfterMaterialization > self become: self proxiedPRFile > Anyway, in next version would be nice to provide a faster way of doing > this, without become. Okay #fuelAfterMaterialization - that's something I didn't know about. I think I can use that to re-arrange the content, without using #become.

Nov. 17, 2011

Re: [Pharo-project] how should Pier file structures be serialized with Fuel?

by Mariano Martinez Peck

On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu <yanni(a)rogers.com> wrote: > An earlier version of Fuel did not having the substitution feature. So, I > had custom mapping code to pre-process the structures before passing to > Fuel for serialization, and to post-process the deserialized structures. > The main purpose of the code was to nil out the PUSecurity objects, because > they were linked to too many other Pier objects. > > So you have already solved this one using the substitution hook explained here: http://rmod.lille.inria.fr/web/pier/software/Fuel/Version1.7/Hooks ? > I'm now eliminating the custom mapping code by using Fuel substitution. > The PRFile object was handled by saving the file contents in an instance > variable of the mapping object. I've eliminated the custom mapping object. > Now, how should the file contents be saved/restored, using the substitution > feature? > > Let me see if I understand....PRFile has a reference to a file (on disk). So what you want is to serialize not only the PRFile but also the contents of the file? > I'm thinking of substituting a proxy object, which somehow "becomes" a > PRFile after deserialization. > > Any other suggestions? > > > -- Mariano http://marianopeck.wordpress.com

Nov. 17, 2011

Re: [Pharo-project] how should Pier file structures be serialized with Fuel?

by Mariano Martinez Peck

On Thu, Nov 17, 2011 at 3:27 AM, Martin Dias <tinchodias(a)gmail.com> wrote: > Hello! > > On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu <yanni(a)rogers.com> wrote: > >> An earlier version of Fuel did not having the substitution feature. So, I >> had custom mapping code to pre-process the structures before passing to >> Fuel for serialization, and to post-process the deserialized structures. >> The main purpose of the code was to nil out the PUSecurity objects, because >> they were linked to too many other Pier objects. >> >> I'm now eliminating the custom mapping code by using Fuel substitution. >> The PRFile object was handled by saving the file contents in an instance >> variable of the mapping object. I've eliminated the custom mapping object. >> Now, how should the file contents be saved/restored, using the substitution >> feature? >> >> I'm thinking of substituting a proxy object, which somehow "becomes" a >> PRFile after deserialization. >> > > you can implement : > > YourPRFileProxy >> fuelAfterMaterialization > self become: self proxiedPRFile > But how he replace its original PRFile of the input graph with a YourPRFileProxy instance? Wouldn't something like this make sense: PRFile >> fuelAccept: aVisitor ^ aVisitor visitSubstitution: self PRFile >> fuelSubstitution ^ YourPRFIleProxy for: self ? The problem there is that ALL PRFile will be replaced by proxies and maybe that's not what you want. > > Anyway, in next version would be nice to provide a faster way of doing > this, without become. > > >> Any other suggestions? >> >> >> > -- Mariano http://marianopeck.wordpress.com

Nov. 17, 2011

[Pharo-project] DrGeo pre-release 11.12e

by Hilaire Fernandes

>From your latest DrGeo installation, you can upgrade to DrGeo pre-release 11.12e To do so, select in the DrGeo environment menu "update DrGeo", then save the session to make the update permanent. Feedback and bugs reports appreciated before final release in December. I also appreciate from XO users cool comments at the DrGeo activity page http://activities.sugarlabs.org/sugar/addon/4323 Hilaire NEW FEATURES #13607 Mark symbol for segment #13589 Mutuate point feature (in select mode with [Shift] key) -- Education 0.2 -- http://blog.ofset.org/hilaire

Nov. 17, 2011

Re: [Pharo-project] WideString performance

by Henrik Sperre Johansen

On 17.11.2011 11:11, Henrik Sperre Johansen wrote: > On 17.11.2011 10:38, Henrik Sperre Johansen wrote: >> >> The efficiency drain in this example is, as Dennis points out, >> CSVParser adding scanned characters 1 by 1 to the WideString >> representing current value. >> >> Cheers, >> Henry > There is something strange going on with CogVM's though, on an old VM > the CSVParser use about 17 secs on Data.csv, while it's 131 secs on > Cog 2508... > It looks like primitive 61 has a *really* slow path for variable word > classes? > > Cheers, > Henry ... or not. (Cog) ws := WideString new: 100000. char := Character value: 5021. [1 to: 100000 do: [:i | ws at: i put: char]] timeToRun 10 (Interpreter) ws := WideString new: 100000. char := Character value: 5021. [1 to: 100000 do: [:i | ws at: i put: char]] timeToRun 12 Seems it is instead the becomeForward: call, as each token starts streaming to a ByteString, and thus triggers it :/ A quick hack which will work in your case since all wide tokens usually start with russian character, is replacing (in nextValue and nextQuotedValue) String streamContents: with something like: (stream peek charCode > 255 ifTrue: [WideString] ifFalse: [String]) streamContents: I don't see an easy/elegant way to avoid it in the general case though :( Cheers, Henry

Nov. 17, 2011

Re: [Pharo-project] * Smalltalk User Group

by Norbert Hartl

Ha ha, I was asking that myself earlier in this century. AFAIK there are not that much smalltalkers in Berlin. But Potsdam and the HPI are close. I wanted to create a Berlin user group but now I don't live there anymore (and I helped creating one in cologne instead). But I'm inclined to attend anyway even now I'm closer to Lille than Berlin :) good luck, Norbert Am 17.11.2011 um 12:53 schrieb StÃ©phane Ducasse: > > > Begin forwarded message: > >> From: Johannes Rasche <johannesrasche(a)mac.com> >> Subject: * Smalltalk User Group >> Date: November 17, 2011 12:27:15 PM GMT+01:00 >> To: pharo-project-owner(a)lists.gforge.inria.fr >> >> as Sven just announced the Belgian Smalltalk User Group (BSUG) meeting: >> >> are there Pharo people in Berlin (I didn't realize it up to now)? >> >> It would be great to have a local community with the opportunity for face-to-face exchange >> and potential cooperation >> >> Johannes >> >> -- >> Johannes Rasche Hagelberger Str. 56 10965 Berlin mail johannesrasche(a)mac.com mobile +49 1577 175 04 75 >

Nov. 17, 2011

Re: [Pharo-project] WideString performance

by Levente Uzonyi

On Thu, 17 Nov 2011, Henrik Sperre Johansen wrote: > On 17.11.2011 11:53, Levente Uzonyi wrote: >> On Thu, 17 Nov 2011, Henrik Sperre Johansen wrote: >> >>> On 17.11.2011 11:26, Levente Uzonyi wrote: >>>> On Thu, 17 Nov 2011, Henrik Sperre Johansen wrote: >>>> >>>>> On 16.11.2011 21:45, Nicolas Cellier wrote: >>>>>> I think that there are several issues: >>>>>> >>>>>> 1) Character with codePoint< 256 are unique, but Wider are not. >>>>>> The characters are not stored in the WideString (which instead stores >>>>>> the code points), but any enumeration using at: rather than wordAt: >>>>>> will create an overhead. I presume that it would be better to have a >>>>>> #codePointAt: defined as #byteAt: in ByteString and #wordAt: in >>>>>> WideString, so we could still define generic methods in String. >>>>>> 2) A lot of ByteString methods are optimized with ad hoc optional >>>>>> primitives, but very few WideString methods are. >>>>>> 3) Some of these optimized operations work with 256 element masks and >>>>>> cannot be generalized to WideString that easily >>>>>> 4) Some Stream operations that copy one Collection to the stream >>>>>> buffer or vice versa are also not optimized for mixed byte/word >>>>>> collections >>>>>> 5) UTF8 is optimized for ASCII anyway... And Squeak/Pharo converters >>>>>> also are (because they can use more primitives in this case) >>>>>> 6) Pharo ripped #squeakToUtf8 and #utf8ToSqueak which were maybe >>>>>> questionable from quality POV, but were not replaced from efficiency >>>>>> POV. >>>>>> >>>>>> I would say that a new VM with immediate Character values might help, >>>>>> but that won't be enough. >>>>>> >>>>>> Nicolas >>>>> If I may object to point 6: >>>>> (Squeak) >>>>> [FileStream oldFileNamed: 'Data.csv' do: [:stream | text := stream >>>>> contents]] timeToRun 6753 >>>>> (Pharo) >>>>> [FileStream oldFileNamed: 'Data.csv' do: [:stream | text := stream >>>>> contents]] timeToRun 7027 >>>>> Reading without conversion, then converting with utf8ToSqueak: >>>>> [FileStream oldFileNamed: 'C:\Users\Rydier\Dropbox\Pharo 1.4 latest >>>>> jenkins\Data.csv' >>>>> do: [:stream | stream converter: Latin1TextConverter >>>>> new. >>>>> text := stream contents >>>>> utf8ToSqueak ]] timeToRun 9752 >>>> >>>> That's not "reading without conversion", you explicitly setting >>>> Latin1TextConverter there, try StandardFileStream instead. ;) >>>> >>>> >>>> Levente >>> Latin1TextConverter does not do any conversion, no? ;) >>> >>> [StandardFileStream oldFileNamed: 'C:\Users\Rydier\Dropbox\Pharo 1.4 >>> latest jenkins\Data.csv' >>> do: [:stream | >>> text := stream contents >>> utf8ToSqueak ]] timeToRun 9221 >>> >>> Better? ;) >> >> Better, but not good enough :). I got the following in Squeak: >> >> [ FileStream readOnlyFileNamed: 'Data.csv' do: [ :file | >> file contents ] ] timeToRun. >> 2429 >> >> [ StandardFileStream readOnlyFileNamed: 'Data.csv' do: [ :file | >> file contents utf8ToSqueak ] ] timeToRun. >> 2288 >> >> Significant amount of time is spent finding the current leadingChar in both >> cases. Using 0 as leadingChar in Unicode >> #value: gives the following >> results: >> >> [ FileStream readOnlyFileNamed: 'Data.csv' do: [ :file | >> file contents ] ] timeToRun. >> 1152 >> >> [ StandardFileStream readOnlyFileNamed: 'Data.csv' do: [ :file | >> file contents utf8ToSqueak ] ] timeToRun. >> 1247 >> >> >> Levente >> >> P.S.: To avoid measuring disk latency, the file was already in memory >> during the tests. > I fail to se how even those values are significant enough to warrant using > utf8ToSqueak rather than let the converter take care of it, which was the > point I contended in the first place. It's not worth using #utf8ToSqueak in this case to get better performance. This is also true for all other cases, because of the optimizations done to text converters (in Squeak UTF8TextConverter uses the same method as #utf8ToSqueak when possible). The only thing you can save with it performance wise is the creation of the converter and the optional lookup of the converter class, which is insignificant in most cases. Levente > > Cheers, > Henry > >

Nov. 17, 2011

[Pharo-project] Fwd: * Smalltalk User Group

by StÃ©phane Ducasse

Begin forwarded message: > From: Johannes Rasche <johannesrasche(a)mac.com> > Subject: * Smalltalk User Group > Date: November 17, 2011 12:27:15 PM GMT+01:00 > To: pharo-project-owner(a)lists.gforge.inria.fr > > as Sven just announced the Belgian Smalltalk User Group (BSUG) meeting: > > are there Pharo people in Berlin (I didn't realize it up to now)? > > It would be great to have a local community with the opportunity for face-to-face exchange > and potential cooperation > > Johannes > > -- > Johannes Rasche Hagelberger Str. 56 10965 Berlin mail johannesrasche(a)mac.com mobile +49 1577 175 04 75

Nov. 17, 2011

Re: [Pharo-project] WideString performance

by Henrik Sperre Johansen

On 17.11.2011 11:53, Levente Uzonyi wrote: > On Thu, 17 Nov 2011, Henrik Sperre Johansen wrote: > >> On 17.11.2011 11:26, Levente Uzonyi wrote: >>> On Thu, 17 Nov 2011, Henrik Sperre Johansen wrote: >>> >>>> On 16.11.2011 21:45, Nicolas Cellier wrote: >>>>> I think that there are several issues: >>>>> >>>>> 1) Character with codePoint< 256 are unique, but Wider are not. >>>>> The characters are not stored in the WideString (which instead stores >>>>> the code points), but any enumeration using at: rather than wordAt: >>>>> will create an overhead. I presume that it would be better to have a >>>>> #codePointAt: defined as #byteAt: in ByteString and #wordAt: in >>>>> WideString, so we could still define generic methods in String. >>>>> 2) A lot of ByteString methods are optimized with ad hoc optional >>>>> primitives, but very few WideString methods are. >>>>> 3) Some of these optimized operations work with 256 element masks and >>>>> cannot be generalized to WideString that easily >>>>> 4) Some Stream operations that copy one Collection to the stream >>>>> buffer or vice versa are also not optimized for mixed byte/word >>>>> collections >>>>> 5) UTF8 is optimized for ASCII anyway... And Squeak/Pharo converters >>>>> also are (because they can use more primitives in this case) >>>>> 6) Pharo ripped #squeakToUtf8 and #utf8ToSqueak which were maybe >>>>> questionable from quality POV, but were not replaced from efficiency >>>>> POV. >>>>> >>>>> I would say that a new VM with immediate Character values might help, >>>>> but that won't be enough. >>>>> >>>>> Nicolas >>>> If I may object to point 6: >>>> (Squeak) >>>> [FileStream oldFileNamed: 'Data.csv' do: [:stream | text := stream >>>> contents]] timeToRun 6753 >>>> (Pharo) >>>> [FileStream oldFileNamed: 'Data.csv' do: [:stream | text := stream >>>> contents]] timeToRun 7027 >>>> Reading without conversion, then converting with utf8ToSqueak: >>>> [FileStream oldFileNamed: 'C:\Users\Rydier\Dropbox\Pharo 1.4 latest >>>> jenkins\Data.csv' >>>> do: [:stream | stream converter: >>>> Latin1TextConverter new. >>>> text := stream contents >>>> utf8ToSqueak ]] timeToRun 9752 >>> >>> That's not "reading without conversion", you explicitly setting >>> Latin1TextConverter there, try StandardFileStream instead. ;) >>> >>> >>> Levente >> Latin1TextConverter does not do any conversion, no? ;) >> >> [StandardFileStream oldFileNamed: 'C:\Users\Rydier\Dropbox\Pharo 1.4 >> latest jenkins\Data.csv' >> do: [:stream | >> text := stream contents >> utf8ToSqueak ]] timeToRun 9221 >> >> Better? ;) > > Better, but not good enough :). I got the following in Squeak: > > [ FileStream readOnlyFileNamed: 'Data.csv' do: [ :file | > file contents ] ] timeToRun. > 2429 > > [ StandardFileStream readOnlyFileNamed: 'Data.csv' do: [ :file | > file contents utf8ToSqueak ] ] timeToRun. > 2288 > > Significant amount of time is spent finding the current leadingChar in > both cases. Using 0 as leadingChar in Unicode >> #value: gives the > following results: > > [ FileStream readOnlyFileNamed: 'Data.csv' do: [ :file | > file contents ] ] timeToRun. > 1152 > > [ StandardFileStream readOnlyFileNamed: 'Data.csv' do: [ :file | > file contents utf8ToSqueak ] ] timeToRun. > 1247 > > > Levente > > P.S.: To avoid measuring disk latency, the file was already in memory > during the tests. I fail to se how even those values are significant enough to warrant using utf8ToSqueak rather than let the converter take care of it, which was the point I contended in the first place. Cheers, Henry

Nov. 17, 2011

Re: [Pharo-project] WideString performance

by Levente Uzonyi

On Thu, 17 Nov 2011, Henrik Sperre Johansen wrote: > On 17.11.2011 11:26, Levente Uzonyi wrote: >> On Thu, 17 Nov 2011, Henrik Sperre Johansen wrote: >> >>> On 16.11.2011 21:45, Nicolas Cellier wrote: >>>> I think that there are several issues: >>>> >>>> 1) Character with codePoint< 256 are unique, but Wider are not. >>>> The characters are not stored in the WideString (which instead stores >>>> the code points), but any enumeration using at: rather than wordAt: >>>> will create an overhead. I presume that it would be better to have a >>>> #codePointAt: defined as #byteAt: in ByteString and #wordAt: in >>>> WideString, so we could still define generic methods in String. >>>> 2) A lot of ByteString methods are optimized with ad hoc optional >>>> primitives, but very few WideString methods are. >>>> 3) Some of these optimized operations work with 256 element masks and >>>> cannot be generalized to WideString that easily >>>> 4) Some Stream operations that copy one Collection to the stream >>>> buffer or vice versa are also not optimized for mixed byte/word >>>> collections >>>> 5) UTF8 is optimized for ASCII anyway... And Squeak/Pharo converters >>>> also are (because they can use more primitives in this case) >>>> 6) Pharo ripped #squeakToUtf8 and #utf8ToSqueak which were maybe >>>> questionable from quality POV, but were not replaced from efficiency >>>> POV. >>>> >>>> I would say that a new VM with immediate Character values might help, >>>> but that won't be enough. >>>> >>>> Nicolas >>> If I may object to point 6: >>> (Squeak) >>> [FileStream oldFileNamed: 'Data.csv' do: [:stream | text := stream >>> contents]] timeToRun 6753 >>> (Pharo) >>> [FileStream oldFileNamed: 'Data.csv' do: [:stream | text := stream >>> contents]] timeToRun 7027 >>> Reading without conversion, then converting with utf8ToSqueak: >>> [FileStream oldFileNamed: 'C:\Users\Rydier\Dropbox\Pharo 1.4 latest >>> jenkins\Data.csv' >>> do: [:stream | stream converter: Latin1TextConverter >>> new. >>> text := stream contents >>> utf8ToSqueak ]] timeToRun 9752 >> >> That's not "reading without conversion", you explicitly setting >> Latin1TextConverter there, try StandardFileStream instead. ;) >> >> >> Levente > Latin1TextConverter does not do any conversion, no? ;) > > [StandardFileStream oldFileNamed: 'C:\Users\Rydier\Dropbox\Pharo 1.4 latest > jenkins\Data.csv' > do: [:stream | > text := stream contents > utf8ToSqueak ]] timeToRun 9221 > > Better? ;) Better, but not good enough :). I got the following in Squeak: [ FileStream readOnlyFileNamed: 'Data.csv' do: [ :file | file contents ] ] timeToRun. 2429 [ StandardFileStream readOnlyFileNamed: 'Data.csv' do: [ :file | file contents utf8ToSqueak ] ] timeToRun. 2288 Significant amount of time is spent finding the current leadingChar in both cases. Using 0 as leadingChar in Unicode >> #value: gives the following results: [ FileStream readOnlyFileNamed: 'Data.csv' do: [ :file | file contents ] ] timeToRun. 1152 [ StandardFileStream readOnlyFileNamed: 'Data.csv' do: [ :file | file contents utf8ToSqueak ] ] timeToRun. 1247 Levente P.S.: To avoid measuring disk latency, the file was already in memory during the tests. > > Cheers, > Henry > >

Nov. 17, 2011