- Pharo-users - lists.pharo.org

Re: [Pharo-users] Roassal question

by Werner Kassens

Hi Pierre, perhaps i should show what i intend to do, i attached a screenshot and i would like to have the axis' on the picture to the right. On 02/23/2015 04:38 PM, Werner Kassens wrote: > Hi Pierre, > i still have a problem since i can't place my RTBoxes at the position i > need them to be. obviously they dont understand #translate: or > #translateTo: like the things in TRCanvas. is there a way around this > restriction? > werner > > >

Feb. 23, 2015

Re: [Pharo-users] Roassal question

by Werner Kassens

Hi Pierre, i still have a problem since i can't place my RTBoxes at the position i need them to be. obviously they dont understand #translate: or #translateTo: like the things in TRCanvas. is there a way around this restriction? werner

Feb. 23, 2015

Re: [Pharo-users] GSOC 2015 Call for Ideas

by stepharo

> I am sorry, I must have misunderstood the purpose of this thread. I > read "Even if you have a vague idea, you can contribute." and tried to > give a couple of vague ideas. no this is good :) We should get influenced and ideas from other people. Now you should consider that since you know better than us certain domains, you could start to have a little project to improve the situation (you know one brick at a time) and doing so you will become an accomplished and zen pharoer :) May be with something as simple as one hour per week you can really improve the situation. Stef

Feb. 23, 2015

Re: [Pharo-users] GSOC 2015 Call for Ideas

by stepharo

> I am sorry if the previous messages came off as too harsh. The Neo > tools are perfectly fine for their intended use. > > What I was trying to say is that a good idea for a SoC project would > be to develop a framework for data analysis that would be useful for > data scientists, and in particular this would include something to > import unstructured data more freely. yes we need that for agile visulization book and distributions. Stef

Feb. 23, 2015

Re: [Pharo-users] [Pharo-dev] GSOC 2015 Call for Ideas

by stepharo

I will add these ideas to the list of Pharo topics (if serge did not do it yet) but after the result of GSOC are announced to avoid to break the web site. Stef Le 19/2/15 09:36, Andrea Ferretti a Ã©crit : > Hi Serge, > > as I said I do not really have the time now to get involved in a gsoc > proposal, but I can give you my perspective. There are two sides to > the story. > > The first one is complementary to SciSmalltalk: in order to analize > data, you need to get data in first. So, one may want to read - say - > a CSV, and have a number of heuristics, such as: > > - autodetection of encoding > - autodetection of quotes and delimiter > - autodetection of columns containing numbers or dates > - the possibility to indicate that some markers, such as "N/A", > represent missing values > - the possibility to indicate a replacement for missing values, such > as 0, or "", or the average or the minimum of the other values in the > colums > > See http://pandas.pydata.org/pandas-docs/version/0.15.2/io.html#csv-text-files > for some examples. > > It may be worth to consider making this into a sequence that is read > and processed lazily, to deal with CSV files bigger than memory. > > When data is finally in, usually the first task is doing some > processing, inspection or visualization. The Smalltalk collections are > good for processing (although some lazy variants might help), and > Roassal and the inspectors are perfect for visualization and browsing. > > The second part comes the time when one wants to run some algorithm. > While there is no need to have the fanciest ones, there should be some > of the basics, such as: > > - some form or regression (linear, logistic...) > - some form of clustering (kmeans, dbscan, canopy...) > - SVM > > Another thing which would be useful is support for linear algebra, > leveraging native libraries such as BLAS or LAPACK. > > In short: just copying R, or numpy + pandas + scikit-learn would > already be a giant leap forward. > > Actually, some of the things I have mentioned above are already (I > think) in SciSmalltalk, which brings me to the next point: > documentation. There is really no point in having all these tools if > people do not know they are there. > > For this to become useful, there should be a dedicated site, > highlighting what is already available, in what state (experimental, > partial, stable...) and how to use it. > > Ideally, I would include also some tutorials, for instance for dealing > with standard problems such as Kaggle competitions. Here I think > Smalltalk would have an edge, since these tutorial could be in the > form of Prof Stef. Still, it would be nice if some form of the > tutorials was also on the web, which makes it discoverable. > > Best, > Andrea > > 2015-02-18 11:14 GMT+01:00 Serge Stinckwich <serge.stinckwich(a)gmail.com>: >> On Wed, Feb 18, 2015 at 11:01 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> OK, try making a proposal then, http://gsoc.pharo.org has the instructions and the current list, you probably know more about data science than I do. >>> >>>> On 18 Feb 2015, at 10:53, Andrea Ferretti <ferrettiandrea(a)gmail.com> wrote: >>>> >>>> I am sorry if the previous messages came off as too harsh. The Neo >>>> tools are perfectly fine for their intended use. >>>> >>>> What I was trying to say is that a good idea for a SoC project would >>>> be to develop a framework for data analysis that would be useful for >>>> data scientists, and in particular this would include something to >>>> import unstructured data more freely. >> Sorry Andrea. I didn't see you message because I'm not pharo-users >> mailing-list, only on pharo-dev. >> I'm also really interested to have a gsoc project to develop data >> analysis framework. >> Please let's talk together in order to discuss about a proposal. >> >> Regards, >> -- >> Serge Stinckwich >> UCBN & UMI UMMISCO 209 (IRD/UPMC) >> Every DSL ends up being Smalltalk >> http://www.doesnotunderstand.org/ >> >

Feb. 23, 2015

Re: [Pharo-users] Conditional CSV parsing

by stepharo

I will add that to the topic list! Stef Le 22/2/15 22:15, Sven Van Caekenberghe a Ã©crit : > There are some cool ideas here: > > https://github.com/BurntSushi/xsv > >> On 31 Jan 2015, at 14:26, stepharo <stepharo(a)free.fr> wrote: >> >> hernan >> >> if you need some help you can also find a smart student and ask ESUG to sponsor him during a summertalk. >> >> Stef >> >> Le 26/1/15 21:03, HernÃ¡n Morales Durand a Ã©crit : >>> >>> 2015-01-26 9:01 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >>> HernÃ¡n, >>> >>>> On 26 Jan 2015, at 08:00, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >>>> >>>> It is possible :) >>>> I work with DNA sequences, there could be millions of common SNPs in a genome. >>> Still weird for CSV. How many record are there then ? >>> >>> We genotyped few individuals (24 records) but now we have a genotyping platform (GeneTitan) with array plates allowing up to 96 samples, which is up to 2.6 million of markers. The first run I completed generated CSVs of 1 million of records (see attach). Sadly the high-level analysis of this data (annotation, clustering, discrimination) now is performed with R with packages like SNPolisher. >>> >>> And this is microarray analysis, NGS platforms produce larger volumes of data in a shorter period of time (several genomes in a day). See http://www.slideshare.net/allenday/renaissance-in-medicine-strata-nosql-and… for the 2014-2020 predictions. >>> >>> Feel free to contact me if you want to experiment with metrics. >>> >>> I assume they all have the same number of fields ? >>> >>> Yes, never seen CSV file with variable number of fields (in this domain) >>> >>> Anyway, could you point me to the specification of the format you want to read ? >>> >>> Actually I have no rush for this, I want to avoid awk, sed and shell scripts in the next run. I would like to avoid Python but spreads like a virus. >>> >>> I will be working mostly with CSV's from Axiom annotation files [1] and genotyping results. Other file formats I use are genotype file formats for programs like PLINK [2] (PED files, column 7 onwards) and HaploView. Is worst than you might think, because you have to transpose the output generated by genotyping platforms (millions of records), and then filter & cut them by chromosome because those Java programs cannot deal with all chromosomes at the same time. >>> >>> And to the older the that you used to use ? >>> >>> >>> http://www.smalltalkhub.com/#!/~hernan/CSV >>> >>> Cheers, >>> HernÃ¡n >>> >>> >>> [1] http://www.affymetrix.com/support/technical/annotationfilesmain.affx >>> [2] http://pngu.mgh.harvard.edu/~purcell/plink/data.shtml#ped >>> >>> >>> >>> Thx, >>> >>> Sven >>> >>>> Cheers, >>>> >>>> HernÃ¡n >>>> >>>> >>>> 2015-01-26 3:33 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >>>> >>>>> On 26 Jan 2015, at 06:32, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >>>>> >>>>> >>>>> >>>>> 2015-01-23 18:00 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >>>>> >>>>>> On 23 Jan 2015, at 20:53, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >>>>>> >>>>>> Hi Sven, >>>>>> >>>>>> 2015-01-23 16:06 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >>>>>> Hi HernÃ¡n, >>>>>> >>>>>>> On 23 Jan 2015, at 19:50, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >>>>>>> >>>>>>> I used to use a CSV parser from Squeak where I could attach conditional iterations: >>>>>>> >>>>>>> csvParser rowsSkipFirst: 2 do: [: row | " some action ignoring first 2 fields on each row " ]. >>>>>>> csvParser rowsSkipLast: 2 do: [: row | " some action ignoring last 2 fields on each row " ]. >>>>>> With NeoCSVParser you can describe how each field is read and converted, using the same mechanism you can ignore fields. Have a look at the senders of #addIgnoredField from the unit tests. >>>>>> >>>>>> >>>>>> I am trying to understand the implementation, I see you included #addIgnoredFields: for consecutive fields in Neo-CSV-Core-SvenVanCaekenberghe.21 >>>>>> A question about usage then, adding ignored field(s) requires adding field types on all other remaining fields? >>>>> Yes, like this: >>>>> >>>>> testReadWithIgnoredField >>>>> | input | >>>>> input := (String crlf join: #( '1,2,a,3' '1,2,b,3' '1,2,c,3' '')). >>>>> self >>>>> assert: ((NeoCSVReader on: input readStream) >>>>> addIntegerField; >>>>> addIntegerField; >>>>> addIgnoredField; >>>>> addIntegerField; >>>>> upToEnd) >>>>> equals: { >>>>> #(1 2 3). >>>>> #(1 2 3). >>>>> #(1 2 3).} >>>>> >>>>> >>>>> >>>>> May be you like to know if you make a pass to NeoCSV, for some data sets I have 1 million of columns, it would be nice an addFieldsInterval: or such. >>>> 1 million columns ? How is that possible, useful ? >>>> >>>> The reader is like a builder. You could try to do this yourself by writing a little loop or two. >>>> >>>> But still, 1 million ? >>>> >>>>> Thank you. >>>>> >>>>> HernÃ¡n >>>>> >>>> >>>> >>> >>> > > >

Feb. 23, 2015

Re: [Pharo-users] Conditional CSV parsing

by Sven Van Caekenberghe

There are some cool ideas here: https://github.com/BurntSushi/xsv > On 31 Jan 2015, at 14:26, stepharo <stepharo(a)free.fr> wrote: > > hernan > > if you need some help you can also find a smart student and ask ESUG to sponsor him during a summertalk. > > Stef > > Le 26/1/15 21:03, HernÃ¡n Morales Durand a Ã©crit : >> >> >> 2015-01-26 9:01 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >> HernÃ¡n, >> >> > On 26 Jan 2015, at 08:00, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >> > >> > It is possible :) >> > I work with DNA sequences, there could be millions of common SNPs in a genome. >> >> Still weird for CSV. How many record are there then ? >> >> We genotyped few individuals (24 records) but now we have a genotyping platform (GeneTitan) with array plates allowing up to 96 samples, which is up to 2.6 million of markers. The first run I completed generated CSVs of 1 million of records (see attach). Sadly the high-level analysis of this data (annotation, clustering, discrimination) now is performed with R with packages like SNPolisher. >> >> And this is microarray analysis, NGS platforms produce larger volumes of data in a shorter period of time (several genomes in a day). See http://www.slideshare.net/allenday/renaissance-in-medicine-strata-nosql-and… for the 2014-2020 predictions. >> >> Feel free to contact me if you want to experiment with metrics. >> >> I assume they all have the same number of fields ? >> >> Yes, never seen CSV file with variable number of fields (in this domain) >> >> Anyway, could you point me to the specification of the format you want to read ? >> >> Actually I have no rush for this, I want to avoid awk, sed and shell scripts in the next run. I would like to avoid Python but spreads like a virus. >> >> I will be working mostly with CSV's from Axiom annotation files [1] and genotyping results. Other file formats I use are genotype file formats for programs like PLINK [2] (PED files, column 7 onwards) and HaploView. Is worst than you might think, because you have to transpose the output generated by genotyping platforms (millions of records), and then filter & cut them by chromosome because those Java programs cannot deal with all chromosomes at the same time. >> >> And to the older the that you used to use ? >> >> >> http://www.smalltalkhub.com/#!/~hernan/CSV >> >> Cheers, >> HernÃ¡n >> >> >> [1] http://www.affymetrix.com/support/technical/annotationfilesmain.affx >> [2] http://pngu.mgh.harvard.edu/~purcell/plink/data.shtml#ped >> >> >> >> Thx, >> >> Sven >> >> > Cheers, >> > >> > HernÃ¡n >> > >> > >> > 2015-01-26 3:33 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >> > >> > > On 26 Jan 2015, at 06:32, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >> > > >> > > >> > > >> > > 2015-01-23 18:00 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >> > > >> > > > On 23 Jan 2015, at 20:53, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >> > > > >> > > > Hi Sven, >> > > > >> > > > 2015-01-23 16:06 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >> > > > Hi HernÃ¡n, >> > > > >> > > > > On 23 Jan 2015, at 19:50, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >> > > > > >> > > > > I used to use a CSV parser from Squeak where I could attach conditional iterations: >> > > > > >> > > > > csvParser rowsSkipFirst: 2 do: [: row | " some action ignoring first 2 fields on each row " ]. >> > > > > csvParser rowsSkipLast: 2 do: [: row | " some action ignoring last 2 fields on each row " ]. >> > > > >> > > > With NeoCSVParser you can describe how each field is read and converted, using the same mechanism you can ignore fields. Have a look at the senders of #addIgnoredField from the unit tests. >> > > > >> > > > >> > > > I am trying to understand the implementation, I see you included #addIgnoredFields: for consecutive fields in Neo-CSV-Core-SvenVanCaekenberghe.21 >> > > > A question about usage then, adding ignored field(s) requires adding field types on all other remaining fields? >> > > >> > > Yes, like this: >> > > >> > > testReadWithIgnoredField >> > > | input | >> > > input := (String crlf join: #( '1,2,a,3' '1,2,b,3' '1,2,c,3' '')). >> > > self >> > > assert: ((NeoCSVReader on: input readStream) >> > > addIntegerField; >> > > addIntegerField; >> > > addIgnoredField; >> > > addIntegerField; >> > > upToEnd) >> > > equals: { >> > > #(1 2 3). >> > > #(1 2 3). >> > > #(1 2 3).} >> > > >> > > >> > > >> > > May be you like to know if you make a pass to NeoCSV, for some data sets I have 1 million of columns, it would be nice an addFieldsInterval: or such. >> > >> > 1 million columns ? How is that possible, useful ? >> > >> > The reader is like a builder. You could try to do this yourself by writing a little loop or two. >> > >> > But still, 1 million ? >> > >> > > Thank you. >> > > >> > > HernÃ¡n >> > > >> > >> > >> > >> >> >> >

Feb. 22, 2015

Re: [Pharo-users] Git (gitfiletree) on Pharo vs Windows

by HernÃ¡n Morales Durand

Thierry, The ConfigurationOfProcessWrapper is available from the Configuration Browser in Pharo 3. The StHub page is http://www.smalltalkhub.com/#!/~hernan/ProcessWrapper I have no permission to write to MetaRepoForPharo40 otherwise I would upload it. Cheers, HernÃ¡n 2015-02-22 16:45 GMT-03:00 Thierry Goubier <thierry.goubier(a)gmail.com>: > Hi Hernan, > > thanks, but ProcessWrapper isn't available on Pharo 3 nor Pharo 4. Could > you tell which configuration or metacello script we need to load it? > > Thierry > > Le 22/02/2015 17:52, HernÃ¡n Morales Durand a Ã©crit : > >> >> ProcessWrapper new >> useStdout; >> startWithShellCommand: 'echo hello'; >> upToEnd >> >> >> HernÃ¡n >> >> Because this is where we are now: GitFileTree can run git commands >> on windows (and they do run correctly), it's just that the command >> output isn't recovered. >> >> Help, please! >> >> Thierry >> >> >> Pharo image if you need: >> http://www.mediafire.com/__download/zsguw7suit4vdu7/15-__ >> 02-22_10.25.zip >> <http://www.mediafire.com/download/zsguw7suit4vdu7/15- >> 02-22_10.25.zip> >> >> >> Jan >> >> >> Thierry Goubier wrote >> >> Le 21/02/2015 21:50, Jan B. a Ã©crit : >> >> Hi >> >> Thanks for reply. >> >> When I run this load command on Pharo 3, I get exception >> "FileDoesNotExist: >> @ \tmp\github-__dalehenrichfiletreeissue142.__zip" >> >> >> Ok. I have uploaded the latest version of >> MonticelloFileTree-Git on >> smalltalkhub; >> >> Gofer new >> url: >> 'https://smalltalkhub.com/mc/__Pharo/MetaRepoForPharo30/main >> <https://smalltalkhub.com/mc/Pharo/MetaRepoForPharo30/main> >> '__; >> >> configurationOf: 'GitFileTree'; >> loadDevelopment >> >> should pick it up >> >> Thierry >> >> On Pharo 4, it loads. But when I try to add gitfiletree >> repository now >> (ie. >> clean pharo 4 platfrom -> your >> issue 142 load command -> adding gitfiletree >> repository), I get exception >> "MessageNotUnderstood: ExternalWindowsOSProcess >> class>>defaultShellPath >> >> When I afterwards try to manually load latest OSProcess >> packages, I get >> the >> same exception like before, without this issue 142 >> loading: >> "MessageNotUnderstood: receiver of "addFirst:" is nil >> >> Jan >> >> >> Thierry Goubier wrote >> >> Hi Jan, >> >> ok, the latest cod for GitFileTree is available on a >> dedicated branch on >> filetree. The way to load it is: >> >> Metacello new >> baseline: 'FileTree'; >> repository: >> 'github://dalehenrich/__ >> filetree:issue_142/repository'__; >> load: 'Git' >> >> It rely on OSProcess, and I'm not entirely sure of >> OSProcess on windows, >> but I've simplified the commands, in part to ease an >> eventual Windows >> support. So, if you happen to be able to test? >> >> I would suggest pharo3.0 as your platform. I think >> changes to filetree >> to handle pharo4.0 are still on a separate branch. >> >> Thierry >> >> -- >> View this message in context: >> http://forum.world.st/Git-__gitfiletree-on-Pharo-vs-__ >> Windows-tp4806791p4806824.html >> <http://forum.world.st/Git-gitfiletree-on-Pharo-vs- >> Windows-tp4806791p4806824.html> >> Sent from the Pharo Smalltalk Users mailing list archive >> at Nabble.com. >> >> >> >> >> >> >> >> >> -- >> View this message in context: >> http://forum.world.st/Git-__gitfiletree-on-Pharo-vs-__ >> Windows-tp4806791p4806857.html >> <http://forum.world.st/Git-gitfiletree-on-Pharo-vs- >> Windows-tp4806791p4806857.html> >> Sent from the Pharo Smalltalk Users mailing list archive at >> Nabble.com. >> >> >> >> >> >> >> > >

Feb. 22, 2015

Re: [Pharo-users] Git (gitfiletree) on Pharo vs Windows

by Thierry Goubier

Hi Hernan, thanks, but ProcessWrapper isn't available on Pharo 3 nor Pharo 4. Could you tell which configuration or metacello script we need to load it? Thierry Le 22/02/2015 17:52, HernÃ¡n Morales Durand a Ã©crit : > > ProcessWrapper new > useStdout; > startWithShellCommand: 'echo hello'; > upToEnd > > > HernÃ¡n > > Because this is where we are now: GitFileTree can run git commands > on windows (and they do run correctly), it's just that the command > output isn't recovered. > > Help, please! > > Thierry > > > Pharo image if you need: > http://www.mediafire.com/__download/zsguw7suit4vdu7/15-__02-22_10.25.zip > <http://www.mediafire.com/download/zsguw7suit4vdu7/15-02-22_10.25.zip> > > > Jan > > > Thierry Goubier wrote > > Le 21/02/2015 21:50, Jan B. a Ã©crit : > > Hi > > Thanks for reply. > > When I run this load command on Pharo 3, I get exception > "FileDoesNotExist: > @ \tmp\github-__dalehenrichfiletreeissue142.__zip" > > > Ok. I have uploaded the latest version of > MonticelloFileTree-Git on > smalltalkhub; > > Gofer new > url: > 'https://smalltalkhub.com/mc/__Pharo/MetaRepoForPharo30/main > <https://smalltalkhub.com/mc/Pharo/MetaRepoForPharo30/main>'__; > configurationOf: 'GitFileTree'; > loadDevelopment > > should pick it up > > Thierry > > On Pharo 4, it loads. But when I try to add gitfiletree > repository now > (ie. > clean pharo 4 platfrom -> your > issue 142 load command -> adding gitfiletree > repository), I get exception > "MessageNotUnderstood: ExternalWindowsOSProcess > class>>defaultShellPath > > When I afterwards try to manually load latest OSProcess > packages, I get > the > same exception like before, without this issue 142 loading: > "MessageNotUnderstood: receiver of "addFirst:" is nil > > Jan > > > Thierry Goubier wrote > > Hi Jan, > > ok, the latest cod for GitFileTree is available on a > dedicated branch on > filetree. The way to load it is: > > Metacello new > baseline: 'FileTree'; > repository: > 'github://dalehenrich/__filetree:issue_142/repository'__; > load: 'Git' > > It rely on OSProcess, and I'm not entirely sure of > OSProcess on windows, > but I've simplified the commands, in part to ease an > eventual Windows > support. So, if you happen to be able to test? > > I would suggest pharo3.0 as your platform. I think > changes to filetree > to handle pharo4.0 are still on a separate branch. > > Thierry > > -- > View this message in context: > http://forum.world.st/Git-__gitfiletree-on-Pharo-vs-__Windows-tp4806791p480… > <http://forum.world.st/Git-gitfiletree-on-Pharo-vs-Windows-tp4806791p4806824…> > Sent from the Pharo Smalltalk Users mailing list archive > at Nabble.com. > > > > > > > > > -- > View this message in context: > http://forum.world.st/Git-__gitfiletree-on-Pharo-vs-__Windows-tp4806791p480… > <http://forum.world.st/Git-gitfiletree-on-Pharo-vs-Windows-tp4806791p4806857…> > Sent from the Pharo Smalltalk Users mailing list archive at > Nabble.com. > > > > > >

Feb. 22, 2015

Re: [Pharo-users] [Pharo-dev] GSOC 2015 Call for Ideas

by Werner Kassens

On 02/21/2015 01:44 PM, stepharo wrote: > Indeed these are nice to have, now they will not magically happen :) > There is a 400 pages book on SciTalk. > Le 19/2/15 09:36, Andrea Ferretti a Ã©crit : >> Actually, some of the things I have mentioned above are already (I >> think) in SciSmalltalk, which brings me to the next point: >> documentation. There is really no point in having all these tools if >> people do not know they are there. >> >> For this to become useful, there should be a dedicated site, >> highlighting what is already available, in what state (experimental, >> partial, stable...) and how to use it. Hi Stefane, hi Andrea, some docu for sciTalk can be found here: https://github.com/SergeStinckwich/SciSmalltalk/wiki/SciSmalltalk-Contents werner

Feb. 22, 2015