December 2010 - Pharo-users

Re: [Pharo-users] Feature requests

by HernÃ¡n Morales Durand

2010/12/8 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: >> >> I know AndrÃ©s did a lot of good work in and for Smalltalk, he >> shouldn't need to present any credentials here. And besides there must >> be a nice way to make people listened and to considerate their claims. > > I know him too and this is not the problem of trust or achievement as a programmer. > We consider claim and expectations this is why we sent calls in the past. > > Now the key question is: is it productive to discuss features if by construction we know that we will not have > the manpower to do something? Considering the extreme and embarrassing success of, for example, Python (even with their crappy syntax and methodologies) over Smalltalk, I think yes it is productive to learn from succesfully software projects. What's wrong with the revisions and improvements process described in my previous mail? Why nobody considerate it? See for example: What is a PEP? ============== PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python or its processes or environment. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as text files in a versioned repository, their revision history is the historical record of the feature proposal (from http://svn.python.org/projects/peps/trunk/pep-0001.txt) > We can dream and talk but at the end of the day if we say ok it costs 4 man months and changing all the vms (which I would love to see happening). > > I have a huge wish list, believe me and the result is gorgeous now the only thing that I can do > is to take one of the item and beat it to death and doing it and go to the next one. And I frustrate a lot > because these are a lot of tiny steps. > Ok, what I'm saying is why don't start organizing the proposals, evaluations, coordination, decisions, etc. and then the people will come. Cheers, -- HernÃ¡n Morales Information Technology Manager, Institute of Veterinary Genetics. National Scientific and Technical Research Council (CONICET). La Plata (1900), Buenos Aires, Argentina. Telephone: +54 (0221) 421-1799. Internal: 422 Fax: 425-7980 or 421-1799.

Dec. 9, 2010

Re: [Pharo-users] Feature requests

by andres

> Now the key question is: is it productive to discuss features if by construction we know that we will not have > the manpower to do something? I think that if you never put the list in public you will never know; if you have your list and I have mine and so does everybody we will always feel like we are the only ones who want that. If we find out that 20 o 30 people want the same thing then maybe we can find a way of making it happen. > We can dream and talk but at the end of the day if we say ok it costs 4 man months and changing all the vms (which I would love to see happening). I think that there are a lot of things out there that don't mean that you have to change the VMs. AndrÃ©s

Dec. 8, 2010

Re: [Pharo-users] Feature requests

by George Herolyants

2010/12/8 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: > Now the key question is: is it productive to discuss features if by construction we know that we will not have > the manpower to do something? I think it is. Who knows, maybe such a wishlist could influence somebody to make a contribution or some research? George

Dec. 8, 2010

Re: [Pharo-users] Feature requests

by Igor Stasenko

On 8 December 2010 15:46, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: >> >> I know AndrÃ©s did a lot of good work in and for Smalltalk, he >> shouldn't need to present any credentials here. And besides there must >> be a nice way to make people listened and to considerate their claims. > > I know him too and this is not the problem of trust or achievement as a programmer. > We consider claim and expectations this is why we sent calls in the past. > > Now the key question is: is it productive to discuss features if by construction we know that we will not have > the manpower to do something? > We can dream and talk but at the end of the day if we say ok it costs 4 man months and changing all the vms (which I would love to see happening). > > I have a huge wish list, believe me and the result is gorgeous now the only thing that I can do > is to take one of the item and beat it to death and doing it and go to the next one. And I frustrate a lot > because these are a lot of tiny steps. > Amen. The better native windows support is pending in my head for 2 years (i think). At least that was a first time when i first thought about it and took some effort to experiment with it and implemented a prototypes. But then i left it, because i had other things to do, and because there was not much interest from people to do that anyways :) So, really, if there's anyone who willing to invest good lump of money in it, it could be done in multiple months. Otherwise, the best what you can do is to check/ask about the current state of art then form a strategy and step by step do things which will bring the day when you have your feature request fulfilled closer. > Stef > -- Best regards, Igor Stasenko AKA sig.

Dec. 8, 2010

Re: [Pharo-users] Feature requests

by StÃ©phane Ducasse

> > I know AndrÃ©s did a lot of good work in and for Smalltalk, he > shouldn't need to present any credentials here. And besides there must > be a nice way to make people listened and to considerate their claims. I know him too and this is not the problem of trust or achievement as a programmer. We consider claim and expectations this is why we sent calls in the past. Now the key question is: is it productive to discuss features if by construction we know that we will not have the manpower to do something? We can dream and talk but at the end of the day if we say ok it costs 4 man months and changing all the vms (which I would love to see happening). I have a huge wish list, believe me and the result is gorgeous now the only thing that I can do is to take one of the item and beat it to death and doing it and go to the next one. And I frustrate a lot because these are a lot of tiny steps. Stef

Dec. 8, 2010

Re: [Pharo-users] Feature requests

by HernÃ¡n Morales Durand

Hi, Thanks for bringing this topic to the list. I like the proposal, but I don't like it in the sense that these kind of mails are "good idea" which requires work and we-all-are-always-busy. Here is what I've seen: This is a distributed community. We may learn from other open source software communities. How they specify revisions and improvements, see for example: Community Formal Process Python -> PEP (http://www.python.org/dev/peps/) Jabber -> XEP (http://xmpp.org/extensions/) Plone -> PLIP (http://plone.org/documentation/glossary/plip) In Python this process includes as activities: 1) Documentation 2) Discussion (which is the most important to keep a high rate of participation) 3) Implementation Maybe that's a valid path too. I read the IRC meeting yesterday. I was suprised in the sense almost no comments were made to what we need to learn about the current Pharo users, about the energy spent in peer-reviewing code or proposals, the division of labor, how many active contributors have the Core or Dev packages, why some of them abandoned or joined, etc. IMHO sooner or later, the core team should have a better view of social dynamics here (I know it requires hard work but there are social scientists out there!). How participants participate, to understand the complexities of motivation, involvement, roles, group calibration, etc. Meta-discussing about our participation is not very comfortable or somewhat controversial, because exposes hidden structures of power or implicit social rules. Which are inevitable. But you will gain insight distinguishing between: Proposals Evaluations Clarifications Synthesis Coordination Decisions and see the most common patterns Proposals -> Evaluations -> Agreement/Disagreement Clarifications -> Clarifications Synthesis -> Agreement/Proposal Coordination -> Coordination Decisions -> Coordination I know AndrÃ©s did a lot of good work in and for Smalltalk, he shouldn't need to present any credentials here. And besides there must be a nice way to make people listened and to considerate their claims. Cheers. 2010/12/7 andres <andres(a)lifia.info.unlp.edu.ar>: > Ok, so here goes an attempt of defining how I think feature requests could > be handled and why I think that the meritocracy is not always the best > choice. > > UserVoice and similar sites > > The idea is to let people ask for features they want in a quick way. I think > my question about native window support could be a good example: someone ask > if a feature exists. If it doesn't, he can just go and record somewhere > "hey, I would like X". If other people also want X they can put a vote (or > more than one) into the feature. Since votes are finite per user you have to > think where are you putting your vote. > > Generally the product owner would choose some of the top ranked requests and > decide to implement them. How many requests are selected per release or what > is the criteria for choosing them is up to the product owner (e.g. you may > choose #1 and #3 in the ranking, since #2 may be too complex for a given > release). I guess that as long as the rules are clear (e.g. choose at least > 1 feature request from the top five or whatever the dev team feels is ok for > them) things would run smooth. > > Of course you can consider alternatives, like giving contributors more votes > to encourage participation, get an extra vote for each bug you found, etc. > > Advantages: Its easy, it welcomes newcomers, it allows to re-think your vote > or change it if you see that someone posted a feature that you also require, > it allows the dev team to measure the user's expectations. > Disadvantages: if you don't have enough population the requests may not > reflect the real community needs, as a dev team you may end up implementing > something you don't consider relevant. > > About Meritocracy > > While I understand the idea behind "first do something, then ask for > something" I don't think it is always the best choice. For someone to commit > to do something he must feel he is inside a community, and that takes some > time. Of course I'm not saying that someone who just downloaded Pharo should > dictate what will be included in the next release, I'm just saying that his > voice shouldn't be banned because he hasn't yet contributed. And I am > convinced that it is more likely to have a newcomer commited to a cause if > you listen to him rather than if you say "first do something, then I may > listen to your requests". > > But what concerned me the most was this: > > The only influence we deeply understand is ***Contributions*** with either > videos, pdf, ads, promotion actions or ST code. > So if you want something to happen for real in pharo then you should > consider how you can make this happens. > > So how do you measure someone's contribution and the rights he has? Talking > about pharo at my university counts for changing a line of code, writing > some slides for a method, creating a bunch of screencasts for some classes? > Also, contributing to Smalltalk in general counts as a contribution to > Pharo? Up to what extent? And if I contributed and you never knew? Should I > present my credentials asking for your reconsideration? My point is that, > while contributions are fine and I also like for people to give in return > for what they received, it is sometimes hard to measure and be fair to > everybody. As you explained it seems that the decision rests in the > subjective view of the core developers. That model is of course a valid one, > I just consider it is not how healthy communities are built. > > Best regards, > AndrÃ©s > > -- HernÃ¡n Morales Information Technology Manager, Institute of Veterinary Genetics. National Scientific and Technical Research Council (CONICET). La Plata (1900), Buenos Aires, Argentina. Telephone: +54 (0221) 421-1799. Internal: 422 Fax: 425-7980 or 421-1799.

Dec. 8, 2010

Re: [Pharo-users] IRC meeting notes

by Sean P. DeNigris

Log at http://pastebin.com/Y7MFaddK -- View this message in context: http://forum.world.st/IRC-meeting-notes-tp3077265p3077350.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Dec. 7, 2010

Feature requests

by andres

Ok, so here goes an attempt of defining how I think feature requests could be handled and why I think that the meritocracy is not always the best choice. UserVoice and similar sites The idea is to let people ask for features they want in a quick way. I think my question about native window support could be a good example: someone ask if a feature exists. If it doesn't, he can just go and record somewhere "hey, I would like X". If other people also want X they can put a vote (or more than one) into the feature. Since votes are finite per user you have to think where are you putting your vote. Generally the product owner would choose some of the top ranked requests and decide to implement them. How many requests are selected per release or what is the criteria for choosing them is up to the product owner (e.g. you may choose #1 and #3 in the ranking, since #2 may be too complex for a given release). I guess that as long as the rules are clear (e.g. choose at least 1 feature request from the top five or whatever the dev team feels is ok for them) things would run smooth. Of course you can consider alternatives, like giving contributors more votes to encourage participation, get an extra vote for each bug you found, etc. Advantages: Its easy, it welcomes newcomers, it allows to re-think your vote or change it if you see that someone posted a feature that you also require, it allows the dev team to measure the user's expectations. Disadvantages: if you don't have enough population the requests may not reflect the real community needs, as a dev team you may end up implementing something you don't consider relevant. About Meritocracy While I understand the idea behind "first do something, then ask for something" I don't think it is always the best choice. For someone to commit to do something he must feel he is inside a community, and that takes some time. Of course I'm not saying that someone who just downloaded Pharo should dictate what will be included in the next release, I'm just saying that his voice shouldn't be banned because he hasn't yet contributed. And I am convinced that it is more likely to have a newcomer commited to a cause if you listen to him rather than if you say "first do something, then I may listen to your requests". But what concerned me the most was this: The only influence we deeply understand is ***Contributions*** with either videos, pdf, ads, promotion actions or ST code. So if you want something to happen for real in pharo then you should consider how you can make this happens. So how do you measure someone's contribution and the rights he has? Talking about pharo at my university counts for changing a line of code, writing some slides for a method, creating a bunch of screencasts for some classes? Also, contributing to Smalltalk in general counts as a contribution to Pharo? Up to what extent? And if I contributed and you never knew? Should I present my credentials asking for your reconsideration? My point is that, while contributions are fine and I also like for people to give in return for what they received, it is sometimes hard to measure and be fair to everybody. As you explained it seems that the decision rests in the subjective view of the core developers. That model is of course a valid one, I just consider it is not how healthy communities are built. Best regards, AndrÃ©s

Dec. 7, 2010

Re: [Pharo-users] Native Windows in Pharo

by andres

Hi Laurent, thanks for your response. While I agree that in many cases the "do-it-yourself" style may apply, I think it would be kind of weird to set up a user voice account for Pharo without the leaders/developers/board being ok with it. Of course is not a matter of laziness, it would be actually less work doing it that writing a couple of mails, it's just that I would feel like taking attributions that don't belong to me. Besides, I don't see anything good coming out of it if the board is not ok with it. Cheers, AndrÃ©s laurent laffont escribiÃ³: > On Tue, Dec 7, 2010 at 1:41 PM, andres <andres(a)lifia.info.unlp.edu.ar>wrote: > >> Stef, Doru: >> first of all I was *asking if* such a model exists, not >> demanding one. Having just entered the Pharo world I wanted to know how you >> guys handle the feature requests. I see now. >> >> On the other hand if you really want to discuss the question of "who has >> the rights for asking for X" in the open source model I'm open, but just to >> be sure I will first ask: Is it ok to open such a thread? Or should I first >> contribute to Pharo before asking? >> > > Asking is a form of contribution and is important because this is what > creates a community and how we can learn. It enables communication and this > is good. And your question is a valuable question. (well, unless people keep > asking over and over the same question without searching in the mailing-list > archive....) > > And actually the best answer I think is: if you want feature X, you have two > ways to get it: > A/ pay someone for doing it > B/ do it yourself and share so other people may help you > > Today in Pharo the most used way is B (and I think this is true for most > open source project). But may be you have (lot of) money :) > > So if you think something like UserVoice may sounds good, just try it and > share. Darwinism will tell whether it is a good idea or not. > > Honestly I'm curious about what ideas can people put in a voting system and > what they can vote on. Maybe newcomers won't be shy to vote (comparing to > sending an email). Now whether the ideas will be implemented is just a > matter of choice between A and B. > > Cheers, > > Laurent > > > >> AndrÃ©s >> >> Tudor Girba escribiÃ³: >> >> +1. >>> Doru >>> >>> >>> On 7 Dec 2010, at 12:39, StÃ©phane Ducasse wrote: >>> >>> Hi andres >>>>> Thanks Mariano! I was looking for something more "formal", to let user's >>>>> vote for features and a commitment of the dev team to include N of the most >>>>> voted per release (of course N depends on the release cycle and the feature >>>>> complexity). >>>>> >>>> Well well well >>>> When people are not able to pay 10 Euros for pharo (we did a poll and got >>>> 5 positive answers) why would they have the right to vote for something that >>>> we would be forced to do? This is an interesting question, no? >>>> >>>> We are not in the consumer model here. We (the people around pharo core) >>>> are fixing the infrastructure so that everybody benefit from it, we can be >>>> influenced >>>> but not much. The only influence we deeply understand is >>>> ***Contributions*** with either videos, pdf, ads, promotion actions or ST >>>> code. >>>> >>>> So if you want something to happen for real in pharo then you should >>>> consider how you can make this happens. >>>> >>>> Stef >>>> >>>> >>>> >>>> >>> -- >>> www.tudorgirba.com >>> >>> "What is more important: To be happy, or to make happy?" >>> >>> >>> >>> >

Dec. 7, 2010

IRC meeting notes

by Adrian Lienhard

This evening the announced IRC meeting took place. I've published the notes as a blog post on our website: http://pharo-project.org/news?article=irc-meeting-notes Cheers, Adrian ___________________ http://www.adrian-lienhard.ch/

Dec. 7, 2010

Re: [Pharo-users] Fastest matrix implementation?

by HernÃ¡n Morales Durand

Hi Sven, Thanks for your comments, my matrix includes names, floats and characters, and the matrix isn't squared because I need to build a transposed matrix while reading the CSV, however I will take a look the SqNumberParser. In case anyone is wondering how this is done, this is the code for a matrix with 20 rows: First to make some predefined columns for which I have to repeat values, so I used the Generator ported by Lukas Renggli. matrix := Matrix rows: 29044 columns: 20. firstCol := ( Generator on: [: g | rowCount timesRepeat: [ g yield: 1 ] ] ) upToEnd. matrix atColumn: 1 put: fCol. "... fill remaining 5 columns ..." index := 0. parserResult rowsDo: [: rs | | rowInd colInd | matrix at: ( rowInd := index \\ 20 + 1 ) at: ( colInd := index // 20 + 7 ) put: 'test'. index := index + 1 ]. 1 to: rowCount do: [: rIndex | 1 to: colCount do: [: cIndex | outputFile nextPutAll: ( matrix at: rIndex at: cIndex ) asString; tab ]. outputFile cr ]. Hope it helps someone, Cheers 2010/12/7 Sven Van Caekenberghe <sven(a)beta9.be>: > Hi HernÃ¡n, > On 06 Dec 2010, at 20:54, HernÃ¡n Morales Durand wrote: > > It seems my performance problem involves reading and parsing a "CSV" file > > Elements Matrix DhbMatrix > 53400 18274 17329 > 175960 61043 60722 > 710500 379276 385278 > > I will check if it's worth to implement a primitive for very fastÂ parsing of > CSV files. > > I think that instead of going native, it would be worthwhile to try to > optimize in Smalltalk first (it certainly is more fun). > I thought that this was an interesting problem so I tried writing some code > myself, assuming the main problem is getting a CSV matrix in and out of > Smalltalk as fast as possible. I simplified further by making the matrix > square and containing only Numbers. I also preallocate the matrix and use > the fact that I know the row/column sizes. > These are my results: > SizeÂ ElementsÂ Read Write > 250 Â 62500 Â Â 1013 7858 > 500 Â 250000 Â 4185 31007 > 750 Â 562500 Â 9858 71434 > I think this is faster, but it is hard to compare. I am still a bit puzzled > as to why the writing is slower than the reading though. > The code is available atÂ http://www.squeaksource.com/ADayAtTheBeach.htmlÂ in > the package 'Smalltalk-Hacking-Sven', class NumberMatrix. > This is the write loop: > writeCsvTo: stream > 1 to: size do: [ :row | > 1 to: size do: [ :column | > column ~= 1 ifTrue: [ stream nextPut: $, ]. > stream print: (self at: row at: column) ]. > stream nextPut: Character lf ] > And this is the read loop: > readCsvFrom: stream > | numberParser | > numberParser := SqNumberParser on: stream. > 1 to: size do: [ :row | > 1 to: size do: [ :column | > self at: row at: column put: numberParser nextNumber. > column ~= size ifTrue: [ stream peekFor: $, ] ]. > row ~= size ifTrue: [ stream peekFor: Character lf ] ] > I am of course cheating a little bit, but should your CSV file be different, > I am sure you can adapt the code (for example to deal with quoting). I am > also advancing the stream under the SqNumberParser to avoid allocation a new > one every time. I think this code generates little garbage. > What do you think ? > Sven > -- HernÃ¡n Morales Information Technology Manager, Institute of Veterinary Genetics. National Scientific and Technical Research Council (CONICET). La Plata (1900), Buenos Aires, Argentina. Telephone: +54 (0221) 421-1799. Internal: 422 Fax: 425-7980 or 421-1799.

Dec. 7, 2010

Re: [Pharo-users] Fastest matrix implementation?

by HernÃ¡n Morales Durand

Hi Mariano, 2010/12/7 Mariano Martinez Peck <marianopeck(a)gmail.com>: > > > On Mon, Dec 6, 2010 at 8:54 PM, HernÃ¡n Morales Durand > <hernan.morales(a)gmail.com> wrote: >> >> Hi Benoit, >> >> I've loaded the package but it seems the port is not complete, i.e. if >> you evaluate: >> >> DhbMatrix new: 10 >> >> you will get a MessageNotUnderstood: Interval>>asVector because >> extension methods were not ported. I uploaded to the SqueakSource a >> new version including extension methods and now most tests pass. >> >> Concerning the performance issues, I've narrowed my code to only >> measure the writing and reading of a matrix of 710500 elements, >> resulting in 58239 milliseconds for the native Matrix implementation >> and 56920 for DhbMatrix. >> It seems my performance problem involves reading and parsing a "CSV" file >> > > > which file stream are you using ?Â do you need encoding? > I'm using just FileStream, how do I know if I need encoding? > You can use the Alexandre Profiler to detect where the problem is. > You mean the PetitProfiler or just Spy? Cheers, > Cheers > > Mariano > >> -- HernÃ¡n Morales Information Technology Manager, Institute of Veterinary Genetics. National Scientific and Technical Research Council (CONICET). La Plata (1900), Buenos Aires, Argentina. Telephone: +54 (0221) 421-1799. Internal: 422 Fax: 425-7980 or 421-1799.

Dec. 7, 2010

Re: [Pharo-users] Fastest matrix implementation?

by HernÃ¡n Morales Durand

Hi Benoit, Thanks for your suggestions! I did some metrics again with a subset of my files with fewer lines, and here are the results Lines Milliseconds 116 59 2784 1415 63936 18675 175840 48216 534760 149812 After implementing your improvements: Lines Milliseconds 116 44 2784 1067 63936 13991 175840 37521 534760 112906 Since I can assume my CSV files doesn't include quotes, I've removed the qouted character checking subclassing a special parser for files with qoutes: Lines Milliseconds 116 37 2784 888 63936 11521 175840 31905 534760 96549 which is an acceptable improvement considering files of millions of lines. If I can make some time for implementing a primitive, I will update with more results. Best regards, 2010/12/7 Benoit St-Jean <bstjean(a)hotmail.ca>: > Hi Hernan, > > I had a look at the Text Parser package and here's a little suggestion... > > In class STextParser, there seems to be place for optimization (I know, I > *hate* using this word with Smalltalk, I've always preferred simplicity over > harder-to-read-code) in one particular method, namely #nextInLine. > > A quick test in a workspace shows that removing message sends for methods > #cr and #lf (making the class variables instead) as well as using #== > instead of #= make this method 3 to 5 times faster.Â Since it's probably > called millions and millions of times in your case, this might help... > > So it would look like: > > nextInLine > Â Â Â | next | > > Â Â Â next := stream next. > Â Â Â (next == MyCr or: [next == MyLf]) > Â Â Â Â Â Â ifTrue:Â Â Â [stream skip: -1. > Â Â Â Â Â Â Â Â Â Â Â Â next := nil]. > Â Â Â ^ next > > > The original method executes in 23.45 seconds > Removing the message sends (Character cr and Character lf) and replacing > them with class vars brings that down to 10.9 seconds. > Then, replacing #= by #== brings the average time to 4.25 seconds. > > Tests were executed for 10000000 characters. > > Hope this helps! > > Keep me posted! > > > ----------------- > Benoit St-Jean > A standpoint is an intellectual horizon of radius zero. > (Albert Einstein) > > > > >> Date: Tue, 7 Dec 2010 03:10:20 -0300 >> From: hernan.morales(a)gmail.com >> To: pharo-users(a)lists.gforge.inria.fr >> Subject: Re: [Pharo-users] Fastest matrix implementation? >> >> I cannot send the CSV files because they are private data currently >> being used for research, just to give a hint I'm parsing files from >> 800,000 to 19 millions of lines. >> >> I'm using http://www.squeaksource.com/SimpleTextParser.html which is >> based in http://www.squeaksource.com/CSV.html plus some useful >> additions (for me). >> >> 2010/12/7 Benoit St-Jean <bstjean(a)hotmail.ca>: >> > What are you using to read those CSV files?Â Do you have a file so we >> > can >> > have a look at it and possibly speed up the reading of the CSV file? >> > >> > ----------------- >> > Benoit St-Jean >> > A standpoint is an intellectual horizon of radius zero. >> > (Albert Einstein) >> > >> > >> > >> > >> >> Date: Mon, 6 Dec 2010 16:54:37 -0300 >> >> From: hernan.morales(a)gmail.com >> >> To: pharo-users(a)lists.gforge.inria.fr >> >> Subject: Re: [Pharo-users] Fastest matrix implementation? >> >> >> >> Hi Benoit, >> >> >> >> I've loaded the package but it seems the port is not complete, i.e. if >> >> you evaluate: >> >> >> >> DhbMatrix new: 10 >> >> >> >> you will get a MessageNotUnderstood: Interval>>asVector because >> >> extension methods were not ported. I uploaded to the SqueakSource a >> >> new version including extension methods and now most tests pass. >> >> >> >> Concerning the performance issues, I've narrowed my code to only >> >> measure the writing and reading of a matrix of 710500 elements, >> >> resulting in 58239 milliseconds for the native Matrix implementation >> >> and 56920 for DhbMatrix. >> >> It seems my performance problem involves reading and parsing a "CSV" >> >> file >> >> >> >> Elements Matrix DhbMatrix >> >> 53400 18274 17329 >> >> 175960 61043 60722 >> >> 710500 379276 385278 >> >> >> >> I will check if it's worth to implement a primitive for very fast >> >> parsing of CSV files. >> >> Cheers, >> >> >> >> 2010/12/5 Benoit St-Jean <bstjean(a)hotmail.ca>: >> >> > Have you tried the matrix implementation in the numerical package >> >> > from >> >> > Didier H. Besset? >> >> > >> >> > http://squeaksource.com/@Q45T_l348Ag07gGT/VMsGzidC >> >> > >> >> > >> >> > >> >> > >> >> > ----------------- >> >> > Benoit St-Jean >> >> > A standpoint is an intellectual horizon of radius zero. >> >> > (Albert Einstein) >> >> > >> >> > >> >> > >> >> > >> >> >> Date: Sun, 5 Dec 2010 17:33:17 -0300 >> >> >> From: hernan.morales(a)gmail.com >> >> >> To: pharo-users(a)lists.gforge.inria.fr >> >> >> Subject: [Pharo-users] Fastest matrix implementation? >> >> >> >> >> >> Hi list >> >> >> >> >> >> In the context of a scientific project here we are building big >> >> >> matrices for later processing, mostly exporting to custom file >> >> >> formats >> >> >> for PLINK, HaploView, etc (bioinformatics tools). I've tested one of >> >> >> our scripts in both Pharo 1.1 (not CogVM) with the corresponding >> >> >> Python 2.6 implementation (without PyPy), and the performance in >> >> >> Python was superior, about 8x faster than ST. >> >> >> So I wonder if anyone knows the fastest (or a faster) implementation >> >> >> of Matrix than the included by default in Collections? >> >> >> >> >> >> Cheers, >> >> >> >> >> >> >> -- >> >> HernÃ¡n Morales >> >> Information Technology Manager, >> >> Institute of Veterinary Genetics. >> >> National Scientific and Technical Research Council (CONICET). >> >> La Plata (1900), Buenos Aires, Argentina. >> >> Telephone: +54 (0221) 421-1799. >> >> Internal: 422 >> >> Fax: 425-7980 or 421-1799. >> >> >> > >> >> >> >> -- >> HernÃ¡n Morales >> Information Technology Manager, >> Institute of Veterinary Genetics. >> National Scientific and Technical Research Council (CONICET). >> La Plata (1900), Buenos Aires, Argentina. >> Telephone: +54 (0221) 421-1799. >> Internal: 422 >> Fax: 425-7980 or 421-1799. >> >

Dec. 7, 2010

Re: [Pharo-users] Native Windows in Pharo

by StÃ©phane Ducasse

On Dec 7, 2010, at 1:41 PM, andres wrote: > Stef, Doru: > first of all I was *asking if* such a model exists, not demanding one. Having just entered the Pharo world I wanted to know how you guys handle the feature requests. I see now. > > On the other hand if you really want to discuss the question of "who has the rights for asking for X" in the open source model I'm open, but just to be sure I will first ask: Is it ok to open such a thread? sure > Or should I first contribute to Pharo before asking? You can but you should know that the chance of getting what you want are low. In the past we ask if esug would support the work of getting multiple native windows. Now apparently the community was not really in need and probably the vm had to be fixed and this was not clear. Stef

Dec. 7, 2010

Re: [Pharo-users] Native Windows in Pharo

by S Krish

If we want to contribute where and how do we contribute...? Perhaps as you state just get a proper DVD for the contribution given @1 per 10 Euro's.. like we used to get from ESUG. -Skrish On Tue, Dec 7, 2010 at 5:09 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>wrote: > Hi andres > > > Thanks Mariano! I was looking for something more "formal", to let user's > vote for features and a commitment of the dev team to include N of the most > voted per release (of course N depends on the release cycle and the feature > complexity). > > Well well well > When people are not able to pay 10 Euros for pharo (we did a poll and got 5 > positive answers) why would they have the right to vote for something that > we would be forced to do? This is an interesting question, no? > > We are not in the consumer model here. We (the people around pharo core) > are fixing the infrastructure so that everybody benefit from it, we can be > influenced > but not much. The only influence we deeply understand is > ***Contributions*** with either videos, pdf, ads, promotion actions or ST > code. > > So if you want something to happen for real in pharo then you should > consider how you can make this happens. > > > Stef > > > >

Dec. 7, 2010

Re: [Pharo-users] Native Windows in Pharo

by laurent laffont

On Tue, Dec 7, 2010 at 1:41 PM, andres <andres(a)lifia.info.unlp.edu.ar>wrote: > Stef, Doru: > first of all I was *asking if* such a model exists, not > demanding one. Having just entered the Pharo world I wanted to know how you > guys handle the feature requests. I see now. > > On the other hand if you really want to discuss the question of "who has > the rights for asking for X" in the open source model I'm open, but just to > be sure I will first ask: Is it ok to open such a thread? Or should I first > contribute to Pharo before asking? > Asking is a form of contribution and is important because this is what creates a community and how we can learn. It enables communication and this is good. And your question is a valuable question. (well, unless people keep asking over and over the same question without searching in the mailing-list archive....) And actually the best answer I think is: if you want feature X, you have two ways to get it: A/ pay someone for doing it B/ do it yourself and share so other people may help you Today in Pharo the most used way is B (and I think this is true for most open source project). But may be you have (lot of) money :) So if you think something like UserVoice may sounds good, just try it and share. Darwinism will tell whether it is a good idea or not. Honestly I'm curious about what ideas can people put in a voting system and what they can vote on. Maybe newcomers won't be shy to vote (comparing to sending an email). Now whether the ideas will be implemented is just a matter of choice between A and B. Cheers, Laurent > AndrÃ©s > > Tudor Girba escribiÃ³: > > +1. >> >> Doru >> >> >> On 7 Dec 2010, at 12:39, StÃ©phane Ducasse wrote: >> >> Hi andres >>> >>>> Thanks Mariano! I was looking for something more "formal", to let user's >>>> vote for features and a commitment of the dev team to include N of the most >>>> voted per release (of course N depends on the release cycle and the feature >>>> complexity). >>>> >>> Well well well >>> When people are not able to pay 10 Euros for pharo (we did a poll and got >>> 5 positive answers) why would they have the right to vote for something that >>> we would be forced to do? This is an interesting question, no? >>> >>> We are not in the consumer model here. We (the people around pharo core) >>> are fixing the infrastructure so that everybody benefit from it, we can be >>> influenced >>> but not much. The only influence we deeply understand is >>> ***Contributions*** with either videos, pdf, ads, promotion actions or ST >>> code. >>> >>> So if you want something to happen for real in pharo then you should >>> consider how you can make this happens. >>> >>> Stef >>> >>> >>> >>> >> -- >> www.tudorgirba.com >> >> "What is more important: To be happy, or to make happy?" >> >> >> >> >

Dec. 7, 2010

Re: [Pharo-users] Native Windows in Pharo

by andres

Stef, Doru: first of all I was *asking if* such a model exists, not demanding one. Having just entered the Pharo world I wanted to know how you guys handle the feature requests. I see now. On the other hand if you really want to discuss the question of "who has the rights for asking for X" in the open source model I'm open, but just to be sure I will first ask: Is it ok to open such a thread? Or should I first contribute to Pharo before asking? AndrÃ©s Tudor Girba escribiÃ³: > +1. > > Doru > > > On 7 Dec 2010, at 12:39, StÃ©phane Ducasse wrote: > >> Hi andres >> >>> Thanks Mariano! I was looking for something more "formal", to let user's vote for features and a commitment of the dev team to include N of the most voted per release (of course N depends on the release cycle and the feature complexity). >> Well well well >> When people are not able to pay 10 Euros for pharo (we did a poll and got 5 positive answers) why would they have the right to vote for something that we would be forced to do? This is an interesting question, no? >> >> We are not in the consumer model here. We (the people around pharo core) are fixing the infrastructure so that everybody benefit from it, we can be influenced >> but not much. The only influence we deeply understand is ***Contributions*** with either videos, pdf, ads, promotion actions or ST code. >> >> So if you want something to happen for real in pharo then you should consider how you can make this happens. >> >> >> Stef >> >> >> > > -- > www.tudorgirba.com > > "What is more important: To be happy, or to make happy?" > > >

Dec. 7, 2010

Re: [Pharo-users] Native Windows in Pharo

by andres

Thanks Sean! Sean P. DeNigris escribiÃ³: > http://forum.world.st/Pharo-Wishlist-was-Native-Windows-in-Pharo-td3075089.…

Dec. 7, 2010

Re: [Pharo-users] Native Windows in Pharo

by Tudor Girba

+1. Doru On 7 Dec 2010, at 12:39, StÃ©phane Ducasse wrote: > Hi andres > >> Thanks Mariano! I was looking for something more "formal", to let user's vote for features and a commitment of the dev team to include N of the most voted per release (of course N depends on the release cycle and the feature complexity). > > Well well well > When people are not able to pay 10 Euros for pharo (we did a poll and got 5 positive answers) why would they have the right to vote for something that we would be forced to do? This is an interesting question, no? > > We are not in the consumer model here. We (the people around pharo core) are fixing the infrastructure so that everybody benefit from it, we can be influenced > but not much. The only influence we deeply understand is ***Contributions*** with either videos, pdf, ads, promotion actions or ST code. > > So if you want something to happen for real in pharo then you should consider how you can make this happens. > > > Stef > > > -- www.tudorgirba.com "What is more important: To be happy, or to make happy?"

Dec. 7, 2010

Re: [Pharo-users] Native Windows in Pharo

by StÃ©phane Ducasse

Hi andres > Thanks Mariano! I was looking for something more "formal", to let user's vote for features and a commitment of the dev team to include N of the most voted per release (of course N depends on the release cycle and the feature complexity). Well well well When people are not able to pay 10 Euros for pharo (we did a poll and got 5 positive answers) why would they have the right to vote for something that we would be forced to do? This is an interesting question, no? We are not in the consumer model here. We (the people around pharo core) are fixing the infrastructure so that everybody benefit from it, we can be influenced but not much. The only influence we deeply understand is ***Contributions*** with either videos, pdf, ads, promotion actions or ST code. So if you want something to happen for real in pharo then you should consider how you can make this happens. Stef

Dec. 7, 2010

Re: [Pharo-users] [Pharo-project] Pharocasts: access SQLite databases through ODBC

by StÃ©phane Ducasse

Excellent! On Dec 5, 2010, at 10:25 PM, laurent laffont wrote: > See how to setup and access an SQLite database through ODBC on Windows. > > Recorded by Torsten Bergmann. > > Enjoy: http://www.pharocasts.com/2010/12/access-database-through-odbc.html > > > > Laurent Laffont > > Pharo Smalltalk Screencasts: http://www.pharocasts.com/ > Blog: http://magaloma.blogspot.com/

Dec. 7, 2010

Re: [Pharo-users] Fastest matrix implementation?

by Sven Van Caekenberghe

Hi HernÃ¡n, On 06 Dec 2010, at 20:54, HernÃ¡n Morales Durand wrote: > It seems my performance problem involves reading and parsing a "CSV" file > > Elements Matrix DhbMatrix > 53400 18274 17329 > 175960 61043 60722 > 710500 379276 385278 > > I will check if it's worth to implement a primitive for very fast parsing of CSV files. I think that instead of going native, it would be worthwhile to try to optimize in Smalltalk first (it certainly is more fun). I thought that this was an interesting problem so I tried writing some code myself, assuming the main problem is getting a CSV matrix in and out of Smalltalk as fast as possible. I simplified further by making the matrix square and containing only Numbers. I also preallocate the matrix and use the fact that I know the row/column sizes. These are my results: Size Elements Read Write 250 62500 1013 7858 500 250000 4185 31007 750 562500 9858 71434 I think this is faster, but it is hard to compare. I am still a bit puzzled as to why the writing is slower than the reading though. The code is available at http://www.squeaksource.com/ADayAtTheBeach.html in the package 'Smalltalk-Hacking-Sven', class NumberMatrix. This is the write loop: writeCsvTo: stream 1 to: size do: [ :row | 1 to: size do: [ :column | column ~= 1 ifTrue: [ stream nextPut: $, ]. stream print: (self at: row at: column) ]. stream nextPut: Character lf ] And this is the read loop: readCsvFrom: stream | numberParser | numberParser := SqNumberParser on: stream. 1 to: size do: [ :row | 1 to: size do: [ :column | self at: row at: column put: numberParser nextNumber. column ~= size ifTrue: [ stream peekFor: $, ] ]. row ~= size ifTrue: [ stream peekFor: Character lf ] ] I am of course cheating a little bit, but should your CSV file be different, I am sure you can adapt the code (for example to deal with quoting). I am also advancing the stream under the SqNumberParser to avoid allocation a new one every time. I think this code generates little garbage. What do you think ? Sven

Dec. 7, 2010

Re: [Pharo-users] Fastest matrix implementation?

by Mariano Martinez Peck

On Mon, Dec 6, 2010 at 8:54 PM, HernÃ¡n Morales Durand < hernan.morales(a)gmail.com> wrote: > Hi Benoit, > > I've loaded the package but it seems the port is not complete, i.e. if > you evaluate: > > DhbMatrix new: 10 > > you will get a MessageNotUnderstood: Interval>>asVector because > extension methods were not ported. I uploaded to the SqueakSource a > new version including extension methods and now most tests pass. > > Concerning the performance issues, I've narrowed my code to only > measure the writing and reading of a matrix of 710500 elements, > resulting in 58239 milliseconds for the native Matrix implementation > and 56920 for DhbMatrix. > It seems my performance problem involves reading and parsing a "CSV" file > > which file stream are you using ? do you need encoding? You can use the Alexandre Profiler to detect where the problem is. Cheers Mariano > Elements Matrix DhbMatrix > 53400 18274 17329 > 175960 61043 60722 > 710500 379276 385278 > > I will check if it's worth to implement a primitive for very fast > parsing of CSV files. > Cheers, > > 2010/12/5 Benoit St-Jean <bstjean(a)hotmail.ca>: > > Have you tried the matrix implementation in the numerical package from > > Didier H. Besset? > > > > http://squeaksource.com/@Q45T_l348Ag07gGT/VMsGzidC > > > > > > > > > > ----------------- > > Benoit St-Jean > > A standpoint is an intellectual horizon of radius zero. > > (Albert Einstein) > > > > > > > > > >> Date: Sun, 5 Dec 2010 17:33:17 -0300 > >> From: hernan.morales(a)gmail.com > >> To: pharo-users(a)lists.gforge.inria.fr > >> Subject: [Pharo-users] Fastest matrix implementation? > >> > >> Hi list > >> > >> In the context of a scientific project here we are building big > >> matrices for later processing, mostly exporting to custom file formats > >> for PLINK, HaploView, etc (bioinformatics tools). I've tested one of > >> our scripts in both Pharo 1.1 (not CogVM) with the corresponding > >> Python 2.6 implementation (without PyPy), and the performance in > >> Python was superior, about 8x faster than ST. > >> So I wonder if anyone knows the fastest (or a faster) implementation > >> of Matrix than the included by default in Collections? > >> > >> Cheers, > >> > > -- > HernÃ¡n Morales > Information Technology Manager, > Institute of Veterinary Genetics. > National Scientific and Technical Research Council (CONICET). > La Plata (1900), Buenos Aires, Argentina. > Telephone: +54 (0221) 421-1799. > Internal: 422 > Fax: 425-7980 or 421-1799. > >

Dec. 7, 2010

Re: [Pharo-users] Fastest matrix implementation?

by HernÃ¡n Morales Durand

I cannot send the CSV files because they are private data currently being used for research, just to give a hint I'm parsing files from 800,000 to 19 millions of lines. I'm using http://www.squeaksource.com/SimpleTextParser.html which is based in http://www.squeaksource.com/CSV.html plus some useful additions (for me). 2010/12/7 Benoit St-Jean <bstjean(a)hotmail.ca>: > What are you using to read those CSV files?Â Do you have a file so we can > have a look at it and possibly speed up the reading of the CSV file? > > ----------------- > Benoit St-Jean > A standpoint is an intellectual horizon of radius zero. > (Albert Einstein) > > > > >> Date: Mon, 6 Dec 2010 16:54:37 -0300 >> From: hernan.morales(a)gmail.com >> To: pharo-users(a)lists.gforge.inria.fr >> Subject: Re: [Pharo-users] Fastest matrix implementation? >> >> Hi Benoit, >> >> I've loaded the package but it seems the port is not complete, i.e. if >> you evaluate: >> >> DhbMatrix new: 10 >> >> you will get a MessageNotUnderstood: Interval>>asVector because >> extension methods were not ported. I uploaded to the SqueakSource a >> new version including extension methods and now most tests pass. >> >> Concerning the performance issues, I've narrowed my code to only >> measure the writing and reading of a matrix of 710500 elements, >> resulting in 58239 milliseconds for the native Matrix implementation >> and 56920 for DhbMatrix. >> It seems my performance problem involves reading and parsing a "CSV" file >> >> Elements Matrix DhbMatrix >> 53400 18274 17329 >> 175960 61043 60722 >> 710500 379276 385278 >> >> I will check if it's worth to implement a primitive for very fast >> parsing of CSV files. >> Cheers, >> >> 2010/12/5 Benoit St-Jean <bstjean(a)hotmail.ca>: >> > Have you tried the matrix implementation in the numerical package from >> > Didier H. Besset? >> > >> > http://squeaksource.com/@Q45T_l348Ag07gGT/VMsGzidC >> > >> > >> > >> > >> > ----------------- >> > Benoit St-Jean >> > A standpoint is an intellectual horizon of radius zero. >> > (Albert Einstein) >> > >> > >> > >> > >> >> Date: Sun, 5 Dec 2010 17:33:17 -0300 >> >> From: hernan.morales(a)gmail.com >> >> To: pharo-users(a)lists.gforge.inria.fr >> >> Subject: [Pharo-users] Fastest matrix implementation? >> >> >> >> Hi list >> >> >> >> In the context of a scientific project here we are building big >> >> matrices for later processing, mostly exporting to custom file formats >> >> for PLINK, HaploView, etc (bioinformatics tools). I've tested one of >> >> our scripts in both Pharo 1.1 (not CogVM) with the corresponding >> >> Python 2.6 implementation (without PyPy), and the performance in >> >> Python was superior, about 8x faster than ST. >> >> So I wonder if anyone knows the fastest (or a faster) implementation >> >> of Matrix than the included by default in Collections? >> >> >> >> Cheers, >> >> >> >> -- >> HernÃ¡n Morales >> Information Technology Manager, >> Institute of Veterinary Genetics. >> National Scientific and Technical Research Council (CONICET). >> La Plata (1900), Buenos Aires, Argentina. >> Telephone: +54 (0221) 421-1799. >> Internal: 422 >> Fax: 425-7980 or 421-1799. >> > -- HernÃ¡n Morales Information Technology Manager, Institute of Veterinary Genetics. National Scientific and Technical Research Council (CONICET). La Plata (1900), Buenos Aires, Argentina. Telephone: +54 (0221) 421-1799. Internal: 422 Fax: 425-7980 or 421-1799.

Dec. 7, 2010

Re: [Pharo-users] Fastest matrix implementation?

by Benoit St-Jean

What are you using to read those CSV files? Do you have a file so we can have a look at it and possibly speed up the reading of the CSV file? ----------------- Benoit St-Jean A standpoint is an intellectual horizon of radius zero. (Albert Einstein) > Date: Mon, 6 Dec 2010 16:54:37 -0300 > From: hernan.morales(a)gmail.com > To: pharo-users(a)lists.gforge.inria.fr > Subject: Re: [Pharo-users] Fastest matrix implementation? > > Hi Benoit, > > I've loaded the package but it seems the port is not complete, i.e. if > you evaluate: > > DhbMatrix new: 10 > > you will get a MessageNotUnderstood: Interval>>asVector because > extension methods were not ported. I uploaded to the SqueakSource a > new version including extension methods and now most tests pass. > > Concerning the performance issues, I've narrowed my code to only > measure the writing and reading of a matrix of 710500 elements, > resulting in 58239 milliseconds for the native Matrix implementation > and 56920 for DhbMatrix. > It seems my performance problem involves reading and parsing a "CSV" file > > Elements Matrix DhbMatrix > 53400 18274 17329 > 175960 61043 60722 > 710500 379276 385278 > > I will check if it's worth to implement a primitive for very fast > parsing of CSV files. > Cheers, > > 2010/12/5 Benoit St-Jean <bstjean(a)hotmail.ca>: > > Have you tried the matrix implementation in the numerical package from > > Didier H. Besset? > > > > http://squeaksource.com/@Q45T_l348Ag07gGT/VMsGzidC > > > > > > > > > > ----------------- > > Benoit St-Jean > > A standpoint is an intellectual horizon of radius zero. > > (Albert Einstein) > > > > > > > > > >> Date: Sun, 5 Dec 2010 17:33:17 -0300 > >> From: hernan.morales(a)gmail.com > >> To: pharo-users(a)lists.gforge.inria.fr > >> Subject: [Pharo-users] Fastest matrix implementation? > >> > >> Hi list > >> > >> In the context of a scientific project here we are building big > >> matrices for later processing, mostly exporting to custom file formats > >> for PLINK, HaploView, etc (bioinformatics tools). I've tested one of > >> our scripts in both Pharo 1.1 (not CogVM) with the corresponding > >> Python 2.6 implementation (without PyPy), and the performance in > >> Python was superior, about 8x faster than ST. > >> So I wonder if anyone knows the fastest (or a faster) implementation > >> of Matrix than the included by default in Collections? > >> > >> Cheers, > >> > > -- > HernÃ¡n Morales > Information Technology Manager, > Institute of Veterinary Genetics. > National Scientific and Technical Research Council (CONICET). > La Plata (1900), Buenos Aires, Argentina. > Telephone: +54 (0221) 421-1799. > Internal: 422 > Fax: 425-7980 or 421-1799. >

Dec. 7, 2010