- Pharo-dev - lists.pharo.org

Re: [Pharo-project] WideString performance

by Henrik Sperre Johansen

On 17.11.2011 10:38, Henrik Sperre Johansen wrote: > > The efficiency drain in this example is, as Dennis points out, > CSVParser adding scanned characters 1 by 1 to the WideString > representing current value. > > Cheers, > Henry There is something strange going on with CogVM's though, on an old VM the CSVParser use about 17 secs on Data.csv, while it's 131 secs on Cog 2508... It looks like primitive 61 has a *really* slow path for variable word classes? Cheers, Henry

Nov. 17, 2011

Re: [Pharo-project] WideString performance

by Henrik Sperre Johansen

On 16.11.2011 21:45, Nicolas Cellier wrote: > I think that there are several issues: > > 1) Character with codePoint< 256 are unique, but Wider are not. > The characters are not stored in the WideString (which instead stores > the code points), but any enumeration using at: rather than wordAt: > will create an overhead. I presume that it would be better to have a > #codePointAt: defined as #byteAt: in ByteString and #wordAt: in > WideString, so we could still define generic methods in String. > 2) A lot of ByteString methods are optimized with ad hoc optional > primitives, but very few WideString methods are. > 3) Some of these optimized operations work with 256 element masks and > cannot be generalized to WideString that easily > 4) Some Stream operations that copy one Collection to the stream > buffer or vice versa are also not optimized for mixed byte/word > collections > 5) UTF8 is optimized for ASCII anyway... And Squeak/Pharo converters > also are (because they can use more primitives in this case) > 6) Pharo ripped #squeakToUtf8 and #utf8ToSqueak which were maybe > questionable from quality POV, but were not replaced from efficiency > POV. > > I would say that a new VM with immediate Character values might help, > but that won't be enough. > > Nicolas If I may object to point 6: (Squeak) [FileStream oldFileNamed: 'Data.csv' do: [:stream | text := stream contents]] timeToRun 6753 (Pharo) [FileStream oldFileNamed: 'Data.csv' do: [:stream | text := stream contents]] timeToRun 7027 Reading without conversion, then converting with utf8ToSqueak: [FileStream oldFileNamed: 'C:\Users\Rydier\Dropbox\Pharo 1.4 latest jenkins\Data.csv' do: [:stream | stream converter: Latin1TextConverter new. text := stream contents utf8ToSqueak ]] timeToRun 9752 The efficiency drain in this example is, as Dennis points out, CSVParser adding scanned characters 1 by 1 to the WideString representing current value. Cheers, Henry

Nov. 17, 2011

[Pharo-project] How can I find all objects that reference myObject?

by niko.schwarz＠googlemail.com

Hi, How can I find all objects that reference myObject? http://stackoverflow.com/questions/8164418/how-can-i-find-all-objects-that-… Niko -- http://scg.unibe.ch/staff/Schwarz twitter.com/nes1983 Tel: +41786126354

Nov. 17, 2011

Re: [Pharo-project] WideString performance

by Dennis Schetinin

There's no need for :) 2011/11/17 Denis Kudriashov <dionisiydk(a)gmail.com> > Hello, > > Where is converting strings from UTF8 encoding? > > > 2011/11/17 Dennis Schetinin <chaetal(a)gmail.com> > >> Sure. >> >> I've loaded CSVParser from http://www.squeaksource.com/CSVParser.htmland patched one method there: >> >> nextRow >> | row | >> row := OrderedCollection new. >> stream skipSeparators. >> [self atEndOfLine] >> whileFalse: [row add: self nextValue]. >> *"stream skip: -1."* >> stream next = $, ifTrue: [row add: '']. >> ^ row >> >> This is how I test it: >> >> stream := FileStream readOnlyFileNamed: 'Data.csv'. >> [(CSVParser on: stream) >> useDelimiter: $;; >> rows ] ensure: [stream close]. >> >> >> >> I attach the bigger data file. The shorter one can be produced by >> deleting rows. >> >> 2011/11/16 Sven Van Caekenberghe <sven(a)beta9.be> >> >>> Dennis, >>> >>> On 16 Nov 2011, at 19:30, Dennis Schetinin wrote: >>> >>> > Hi! >>> > >>> > Parsing a UTF-8 CSV file with Russian symbols (using CSVParser), I >>> found it really slow. It takes about 5 seconds to read a 53Kb file (136 >>> lines) and several minutes to parse a 3.4Mb file (about 8K lines). >>> > >>> > Profiler shows that nearly all the time is eaten by WideString >> >>> at:put: which is invoked from WriteStream>>nextPut:. If I'm not mistaken, >>> this means that <primitive: 66> there does not work in this caseâ¦ So, to be >>> short: is there a simple way to improve the performance? TIA >>> > >>> > -- >>> > Dennis Schetinin >>> >>> I am interested in this problem. >>> >>> Would it be possible to provide a test file (not that I can read >>> Russian) and your test code ? >>> >>> I would like to try this myself. >>> >>> Sven >>> >> >> >> >> -- >> Dennis Schetinin >> > > -- Dennis Schetinin

Nov. 17, 2011

Re: [Pharo-project] OpenGL in Pharo?

by Serge Stinckwich

On Thu, Nov 17, 2011 at 4:52 PM, Michele Lanza <michele.lanza(a)usi.ch> wrote: > Dear all, > I wanted to ask whether there are some news I missed with respect to having a decent OpenGL framework in Pharo. I checked the following page: > > http://book.pharo-project.org/book/LanguageAndLibraries/3DGraphicsAndOpenGL/ > > ..but it sounds all a bit outdated and not something I can rely on in the long run. Ideally there should be something like the Jun framework (available for VisualWorks), but when I met the Jun maker at ICSE and I asked him whether there were any plans of porting Jun to Pharo he told me that it was not in their plans. Long talk, short sense: Is anyone working on OpenGL for Pharo? Did you talk to Aoki Atsushi ? I meet him 2 days ago in Kyoto and i show him a quick demo of Pharo. He shows me very interesting demonstrations of haptic device & 3D visualisation with Cincom Smalltalk. Regards, -- Serge Stinckwich UMI UMMISCO 209 (IRD/UPMC), Hanoi, Vietnam Matsuno Laboratory, Kyoto University, Japan (until 12/2011) http://www.mechatronics.me.kyoto-u.ac.jp/ Every DSL ends up being Smalltalk http://doesnotunderstand.org/

Nov. 17, 2011

[Pharo-project] OpenGL in Pharo?

by Michele Lanza

Dear all, I wanted to ask whether there are some news I missed with respect to having a decent OpenGL framework in Pharo. I checked the following page: http://book.pharo-project.org/book/LanguageAndLibraries/3DGraphicsAndOpenGL/ ..but it sounds all a bit outdated and not something I can rely on in the long run. Ideally there should be something like the Jun framework (available for VisualWorks), but when I met the Jun maker at ICSE and I asked him whether there were any plans of porting Jun to Pharo he told me that it was not in their plans. Long talk, short sense: Is anyone working on OpenGL for Pharo? Cheers Michele

Nov. 17, 2011

Re: [Pharo-project] WideString performance

by Sven Van Caekenberghe

I will have a look somewhere in the next couple of days. On 17 Nov 2011, at 05:15, Dennis Schetinin wrote: > Sure. > > I've loaded CSVParser from http://www.squeaksource.com/CSVParser.html and patched one method there: > > nextRow > | row | > row := OrderedCollection new. > stream skipSeparators. > [self atEndOfLine] > whileFalse: [row add: self nextValue]. > "stream skip: -1." > stream next = $, ifTrue: [row add: '']. > ^ row > > This is how I test it: > > stream := FileStream readOnlyFileNamed: 'Data.csv'. > [(CSVParser on: stream) > useDelimiter: $;; > rows ] ensure: [stream close]. > > > > I attach the bigger data file. The shorter one can be produced by deleting rows. >

Nov. 17, 2011

Re: [Pharo-project] how should Pier file structures be serialized with Fuel?

by Lukas Renggli

Just out of curiosity: Why are PUSecurity linked to too many other objects? Is there a bug leaking memory? Lukas On Thursday, 17 November 2011, Martin Dias <tinchodias(a)gmail.com> wrote: > Hello! > On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu <yanni(a)rogers.com> wrote: >> >> An earlier version of Fuel did not having the substitution feature. So, I had custom mapping code to pre-process the structures before passing to Fuel for serialization, and to post-process the deserialized structures. The main purpose of the code was to nil out the PUSecurity objects, because they were linked to too many other Pier objects. >> >> I'm now eliminating the custom mapping code by using Fuel substitution. The PRFile object was handled by saving the file contents in an instance variable of the mapping object. I've eliminated the custom mapping object. Now, how should the file contents be saved/restored, using the substitution feature? >> >> I'm thinking of substituting a proxy object, which somehow "becomes" a PRFile after deserialization. > > you can implement : > YourPRFileProxy >> fuelAfterMaterialization > self become: self proxiedPRFile > > Anyway, in next version would be nice to provide a faster way of doing this, without become. >> >> Any other suggestions? >> >> > > -- Lukas Renggli www.lukas-renggli.ch

Nov. 17, 2011

Re: [Pharo-project] how should Pier file structures be serialized with Fuel?

by Martin Dias

Hello! On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu <yanni(a)rogers.com> wrote: > An earlier version of Fuel did not having the substitution feature. So, I > had custom mapping code to pre-process the structures before passing to > Fuel for serialization, and to post-process the deserialized structures. > The main purpose of the code was to nil out the PUSecurity objects, because > they were linked to too many other Pier objects. > > I'm now eliminating the custom mapping code by using Fuel substitution. > The PRFile object was handled by saving the file contents in an instance > variable of the mapping object. I've eliminated the custom mapping object. > Now, how should the file contents be saved/restored, using the substitution > feature? > > I'm thinking of substituting a proxy object, which somehow "becomes" a > PRFile after deserialization. > you can implement : YourPRFileProxy >> fuelAfterMaterialization self become: self proxiedPRFile Anyway, in next version would be nice to provide a faster way of doing this, without become. > Any other suggestions? > > >

Nov. 17, 2011

Re: [Pharo-project] WideString performance

by Denis Kudriashov

Hello, Where is converting strings from UTF8 encoding? 2011/11/17 Dennis Schetinin <chaetal(a)gmail.com> > Sure. > > I've loaded CSVParser from http://www.squeaksource.com/CSVParser.html and > patched one method there: > > nextRow > | row | > row := OrderedCollection new. > stream skipSeparators. > [self atEndOfLine] > whileFalse: [row add: self nextValue]. > *"stream skip: -1."* > stream next = $, ifTrue: [row add: '']. > ^ row > > This is how I test it: > > stream := FileStream readOnlyFileNamed: 'Data.csv'. > [(CSVParser on: stream) > useDelimiter: $;; > rows ] ensure: [stream close]. > > > > I attach the bigger data file. The shorter one can be produced by deleting > rows. > > 2011/11/16 Sven Van Caekenberghe <sven(a)beta9.be> > >> Dennis, >> >> On 16 Nov 2011, at 19:30, Dennis Schetinin wrote: >> >> > Hi! >> > >> > Parsing a UTF-8 CSV file with Russian symbols (using CSVParser), I >> found it really slow. It takes about 5 seconds to read a 53Kb file (136 >> lines) and several minutes to parse a 3.4Mb file (about 8K lines). >> > >> > Profiler shows that nearly all the time is eaten by WideString >> >> at:put: which is invoked from WriteStream>>nextPut:. If I'm not mistaken, >> this means that <primitive: 66> there does not work in this caseâ¦ So, to be >> short: is there a simple way to improve the performance? TIA >> > >> > -- >> > Dennis Schetinin >> >> I am interested in this problem. >> >> Would it be possible to provide a test file (not that I can read Russian) >> and your test code ? >> >> I would like to try this myself. >> >> Sven >> > > > > -- > Dennis Schetinin >

Nov. 17, 2011