- Pharo-users - lists.pharo.org

Fastest matrix implementation?
by HernÃ¡n Morales Durand Dec. 7, 2010

Dec. 7, 2010

Hi list In the context of a scientific project here we are building big matrices for later processing, mostly exporting to custom file formats for PLINK, HaploView, etc (bioinformatics tools). I've tested one of our scripts in both Pharo 1.1 (not CogVM) with the corresponding Python 2.6 implementation (without PyPy), and the performance in Python was superior, about 8x faster than ST. So I wonder if anyone knows the fastest (or a faster) implementation of Matrix than the included by default in Collections? Cheers, -- HernÃ¡n Morales Information Technology Manager, Institute of Veterinary Genetics. National Scientific and Technical Research Council (CONICET). La Plata (1900), Buenos Aires, Argentina. Telephone: +54 (0221) 421-1799. Internal: 422 Fax: 425-7980 or 421-1799.

4 8

Re: [Pharo-users] Fastest matrix implementation?
by HernÃ¡n Morales Durand Dec. 7, 2010

Dec. 7, 2010

Hi Benoit, Thanks for your suggestions! I did some metrics again with a subset of my files with fewer lines, and here are the results Lines Milliseconds 116 59 2784 1415 63936 18675 175840 48216 534760 149812 After implementing your improvements: Lines Milliseconds 116 44 2784 1067 63936 13991 175840 37521 534760 112906 Since I can assume my CSV files doesn't include quotes, I've removed the qouted character checking subclassing a special parser for files with qoutes: Lines Milliseconds 116 37 2784 888 63936 11521 175840 31905 534760 96549 which is an acceptable improvement considering files of millions of lines. If I can make some time for implementing a primitive, I will update with more results. Best regards, 2010/12/7 Benoit St-Jean <bstjean(a)hotmail.ca>: > Hi Hernan, > > I had a look at the Text Parser package and here's a little suggestion... > > In class STextParser, there seems to be place for optimization (I know, I > *hate* using this word with Smalltalk, I've always preferred simplicity over > harder-to-read-code) in one particular method, namely #nextInLine. > > A quick test in a workspace shows that removing message sends for methods > #cr and #lf (making the class variables instead) as well as using #== > instead of #= make this method 3 to 5 times faster.Â Since it's probably > called millions and millions of times in your case, this might help... > > So it would look like: > > nextInLine > Â Â Â | next | > > Â Â Â next := stream next. > Â Â Â (next == MyCr or: [next == MyLf]) > Â Â Â Â Â Â ifTrue:Â Â Â [stream skip: -1. > Â Â Â Â Â Â Â Â Â Â Â Â next := nil]. > Â Â Â ^ next > > > The original method executes in 23.45 seconds > Removing the message sends (Character cr and Character lf) and replacing > them with class vars brings that down to 10.9 seconds. > Then, replacing #= by #== brings the average time to 4.25 seconds. > > Tests were executed for 10000000 characters. > > Hope this helps! > > Keep me posted! > > > ----------------- > Benoit St-Jean > A standpoint is an intellectual horizon of radius zero. > (Albert Einstein) > > > > >> Date: Tue, 7 Dec 2010 03:10:20 -0300 >> From: hernan.morales(a)gmail.com >> To: pharo-users(a)lists.gforge.inria.fr >> Subject: Re: [Pharo-users] Fastest matrix implementation? >> >> I cannot send the CSV files because they are private data currently >> being used for research, just to give a hint I'm parsing files from >> 800,000 to 19 millions of lines. >> >> I'm using http://www.squeaksource.com/SimpleTextParser.html which is >> based in http://www.squeaksource.com/CSV.html plus some useful >> additions (for me). >> >> 2010/12/7 Benoit St-Jean <bstjean(a)hotmail.ca>: >> > What are you using to read those CSV files?Â Do you have a file so we >> > can >> > have a look at it and possibly speed up the reading of the CSV file? >> > >> > ----------------- >> > Benoit St-Jean >> > A standpoint is an intellectual horizon of radius zero. >> > (Albert Einstein) >> > >> > >> > >> > >> >> Date: Mon, 6 Dec 2010 16:54:37 -0300 >> >> From: hernan.morales(a)gmail.com >> >> To: pharo-users(a)lists.gforge.inria.fr >> >> Subject: Re: [Pharo-users] Fastest matrix implementation? >> >> >> >> Hi Benoit, >> >> >> >> I've loaded the package but it seems the port is not complete, i.e. if >> >> you evaluate: >> >> >> >> DhbMatrix new: 10 >> >> >> >> you will get a MessageNotUnderstood: Interval>>asVector because >> >> extension methods were not ported. I uploaded to the SqueakSource a >> >> new version including extension methods and now most tests pass. >> >> >> >> Concerning the performance issues, I've narrowed my code to only >> >> measure the writing and reading of a matrix of 710500 elements, >> >> resulting in 58239 milliseconds for the native Matrix implementation >> >> and 56920 for DhbMatrix. >> >> It seems my performance problem involves reading and parsing a "CSV" >> >> file >> >> >> >> Elements Matrix DhbMatrix >> >> 53400 18274 17329 >> >> 175960 61043 60722 >> >> 710500 379276 385278 >> >> >> >> I will check if it's worth to implement a primitive for very fast >> >> parsing of CSV files. >> >> Cheers, >> >> >> >> 2010/12/5 Benoit St-Jean <bstjean(a)hotmail.ca>: >> >> > Have you tried the matrix implementation in the numerical package >> >> > from >> >> > Didier H. Besset? >> >> > >> >> > http://squeaksource.com/@Q45T_l348Ag07gGT/VMsGzidC >> >> > >> >> > >> >> > >> >> > >> >> > ----------------- >> >> > Benoit St-Jean >> >> > A standpoint is an intellectual horizon of radius zero. >> >> > (Albert Einstein) >> >> > >> >> > >> >> > >> >> > >> >> >> Date: Sun, 5 Dec 2010 17:33:17 -0300 >> >> >> From: hernan.morales(a)gmail.com >> >> >> To: pharo-users(a)lists.gforge.inria.fr >> >> >> Subject: [Pharo-users] Fastest matrix implementation? >> >> >> >> >> >> Hi list >> >> >> >> >> >> In the context of a scientific project here we are building big >> >> >> matrices for later processing, mostly exporting to custom file >> >> >> formats >> >> >> for PLINK, HaploView, etc (bioinformatics tools). I've tested one of >> >> >> our scripts in both Pharo 1.1 (not CogVM) with the corresponding >> >> >> Python 2.6 implementation (without PyPy), and the performance in >> >> >> Python was superior, about 8x faster than ST. >> >> >> So I wonder if anyone knows the fastest (or a faster) implementation >> >> >> of Matrix than the included by default in Collections? >> >> >> >> >> >> Cheers, >> >> >> >> >> >> >> -- >> >> HernÃ¡n Morales >> >> Information Technology Manager, >> >> Institute of Veterinary Genetics. >> >> National Scientific and Technical Research Council (CONICET). >> >> La Plata (1900), Buenos Aires, Argentina. >> >> Telephone: +54 (0221) 421-1799. >> >> Internal: 422 >> >> Fax: 425-7980 or 421-1799. >> >> >> > >> >> >> >> -- >> HernÃ¡n Morales >> Information Technology Manager, >> Institute of Veterinary Genetics. >> National Scientific and Technical Research Council (CONICET). >> La Plata (1900), Buenos Aires, Argentina. >> Telephone: +54 (0221) 421-1799. >> Internal: 422 >> Fax: 425-7980 or 421-1799. >> >

1 0

Pharocasts: access SQLite databases through ODBC
by laurent laffont Dec. 7, 2010

Dec. 7, 2010

See how to setup and access an SQLite database through ODBC on Windows. Recorded by Torsten Bergmann. Enjoy: http://www.pharocasts.com/2010/12/access-database-through-odbc.html Laurent Laffont Pharo Smalltalk Screencasts: http://www.pharocasts.com/ Blog: http://magaloma.blogspot.com/

2 1

SMTPClient issue
by Daniel Klein Dec. 6, 2010

Dec. 6, 2010

I can't seem to get: SMTPClient example to work. I changed the 'deliverMailFrom:', 'to:' and 'usingServer:' arguments to valid entries but I keep getting a 'TelnetProtocolError'. Could someone else please test this out cos I think it's broken. Please prove me wrong ;-) I'm using: Pharo-1.1.1-- Latest update: #11414 on Windows XP Pro SP3. Dan

3 5

[Pharo] Checkbox
by alexandre bp Dec. 3, 2010

Dec. 3, 2010

Hi, I have a problem with checkboxes. I'm trying to add a checkbox at each line of a table. I have tried several things(the last tries are in comments): addActionColumn columns add: ((WAReportColumn new) title: ''; valueBlock: [ :row :html | html span with:[ "checked ifTrue:[BibliList bibliCollection add: row]." html checkbox value: checked; onTrue:[BibliList bibliCollection add: row] onFalse:[]. "onClick: [checked := true]" "callback:[:value | value ifTrue: [BibliList bibliCollection add: row]]" ]]). Unfortunately, nothing seems to work...when I click on the checkbox, nothing happen and my list bibliCollection (which is an OrderedCollection) is still empty. Thx for your help alex

1 0

Pharo integration in enterprise feedback
by laurent laffont Dec. 3, 2010

Dec. 3, 2010

Hi, I'm looking for feedback on Pharo integration in enterprise. Especially how you have managed to put Pharo in your enterprise. For example, I'm involved in web application development in other languages. For functional testing I'm able to script Selenium using Pharo thanks to WebTester. So Pharo has made a small step at my work. I'm looking for such stories. Cheers, Laurent Laffont Pharo Smalltalk Screencasts: http://www.pharocasts.com/ Blog: http://magaloma.blogspot.com/

2 3

[ANN] New Smalltalk VM Beginners mailing list
by Mariano Martinez Peck Dec. 1, 2010

Dec. 1, 2010

Hi everybody. We want to announce that we've created a mailing list called "Smalltalk VM Beginners" where all the people interested in VM can share knowledge. The aim of this mailing list is create a good and positive energy and spread the knowledge around the Squeak VM. If you always wanted to know something on the VM but never dare to ask, join the mailing-list, may be we will not know it but we will try hard to learn it together. The idea is that we can help each other, welcome people to learn about the VM, and increase and share the knowledge about it in the community. Everybody can ask questions, everybody can help. This mailing list has more or less the same idea of pharo-users or squeak-beginners. Now, to clarify the objectives of the mailing list, we propose some requirements: - Be polite, respectful. - No question is stupid and all questions are welcome. We welcome the most basic questions someone can ask about the VM like, "is it written in C?" "How can I download the code?" - This mailing list should be useful for people that are just interested in knowing a little about the VM, to those who want to know how certain things are done, and specially for those that are learning about the VM because they need to extend or modify it. - If you have a heavy question, you will probably not get the answer in this mailing-list but this is not a problem you can ask in the Squeak VM mailing list. - The idea is that we can help each other. So, the link to the mailing list is: http://lists.squeakfoundation.org/mailman/listinfo/vm-beginners Address: vm-beginners(a)lists.squeakfoundation.org. Cheers

1 0

Re: [Pharo-users] pharo 1.1. headless server startup and shutdown
by Paul DeBruicker Dec. 1, 2010

Dec. 1, 2010

On 12/01/2010 06:04 AM, pharo-users-request(a)lists.gforge.inria.fr wrote: > Message: 1 > Date: Tue, 30 Nov 2010 09:58:07 -0500 > From: Jon Hancock<jhancock(a)shellshadow.com> > Subject: > To:pharo-users@lists.gforge.inria.fr > Message-ID:<4CF510FF.9070909(a)shellshadow.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > I've found a few tidbits on how to start pharo headless. > I'm using the most recent Pharo-1.1-OneClick.app which works fine for > dev on OS X. When I try to move this to linux, I get troubles. > > To keep things simple, I've copied the pharo.sh script as pharo-headless.sh. > > Here is the script: > --- BEGIN --- > #!/bin/sh > > # path > ROOT=`dirname $0` > BASE="$ROOT/Contents/Linux" > > # execute > exec "$BASE/squeakvm" \ > -plugins "$BASE" -mmap 100m -encoding latin1 \ > -vm-display-X11 -vm-sound-null -headless \ I think the preceding line should be -vm-display-null -vm-sound-null > "$ROOT/Contents/Resources/pharo.image" > > --- END --- > > On ubuntu 10.04 32-bit I get this: > jhancock@pi1:~/Pharo-1.1-OneClick.app$ ./pharo-headless.sh > libSM.so.6: cannot open shared object file: No such file or directory > could not find module vm-display-X11 > Aborted > > On ubuntu 8.04 64-bit I get this: > knr@us1:~/Pharo-1.1-OneClick.app$ ./pharo-headless.sh > exec: 15: ./Contents/Linux/squeakvm: not found make sure you have the package ia32-libs installed on the 64bit system. > I really need to get it running on the ubuntu 8.04 64-bit system. > Once I get past this problem, what is the best method to kill/shutdown > pharo from outside the dev environ? I don't require to save the image, > just safely kill it so it doesn't hold on to resources like socket > listener, etc... > > thanks, Jon > > > > > > > > > ------------------------------

1 0

pharo 1.1. headless server startup and shutdown
by Jon Hancock Nov. 30, 2010

Nov. 30, 2010

I've found a few tidbits on how to start pharo headless. I'm using the most recent Pharo-1.1-OneClick.app which works fine for dev on OS X. When I try to move this to linux, I get troubles. To keep things simple, I've copied the pharo.sh script as pharo-headless.sh. Here is the script: --- BEGIN --- #!/bin/sh # path ROOT=`dirname $0` BASE="$ROOT/Contents/Linux" # execute exec "$BASE/squeakvm" \ -plugins "$BASE" -mmap 100m -encoding latin1 \ -vm-display-X11 -vm-sound-null -headless \ "$ROOT/Contents/Resources/pharo.image" --- END --- On ubuntu 10.04 32-bit I get this: jhancock@pi1:~/Pharo-1.1-OneClick.app$ ./pharo-headless.sh libSM.so.6: cannot open shared object file: No such file or directory could not find module vm-display-X11 Aborted On ubuntu 8.04 64-bit I get this: knr@us1:~/Pharo-1.1-OneClick.app$ ./pharo-headless.sh exec: 15: ./Contents/Linux/squeakvm: not found I really need to get it running on the ubuntu 8.04 64-bit system. Once I get past this problem, what is the best method to kill/shutdown pharo from outside the dev environ? I don't require to save the image, just safely kill it so it doesn't hold on to resources like socket listener, etc... thanks, Jon

3 2

Changesets with LF/CRLF instead of CR?
by Stefan Marr Nov. 29, 2010

Nov. 29, 2010

Hi: Is there an easy way to have change sets filed out with LF instead of the horribly incompatible CR? And well, how do I convince the import to accept my change sets? Related to that, what is the status of this GSoC Project http://gsoc2010.esug.org/projects/git-mercurial is that related to http://www.squeaksource.com/Git? Thanks a lot Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525

1 0