May 2017 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Fwd: Re: 11/05/17 - Tabular Data Structures for Data Analysis - Oleksandr Zaytsev

by Stephane Ducasse

I'm interested to help for such new "containers". May be we should proceed that way: On Tue, May 16, 2017 at 7:44 PM, phil(a)highoctane.be <phil(a)highoctane.be> wrote: > We may also use Discord and do something "somewhat live" > > Phil > > On Tue, May 16, 2017 at 7:23 PM, <serge.stinckwich(a)gmail.com> wrote: > >> I was asking Philippe but hope to see you also at ESUG ! >> >> EnvoyÃ© de mon iPhone >> >> Le 16 mai 2017 Ã 19:02, Oleksandr Zaytsev <olk.zaytsev(a)gmail.com> a >> Ã©crit : >> >> I would love to, but to go to Lille from my country I would need a visa. >> Which is not that easy to acquire. >> So maybe I will come to PharoDays 2018. >> And I will definitely try to come to ESUG Conference in September. >> >> Oleks >> >> On Tue, May 16, 2017 at 7:26 PM, <serge.stinckwich(a)gmail.com> wrote: >> >>> >>> >>> EnvoyÃ© de mon iPhone >>> >>> Le 11 mai 2017 Ã 11:43, "phil(a)highoctane.be" <phil(a)highoctane.be> a >>> Ã©crit : >>> >>> ---------- Message transfÃ©rÃ© ---------- >>> De : "phil(a)highoctane.be" <phil(a)highoctane.be> >>> Date : 11 mai 2017 10:54 >>> Objet : Re: 11/05/17 - Tabular Data Structures for Data Analysis - >>> Oleksandr Zaytsev >>> Ã : "Nick Papoylias" <npapoylias(a)gmail.com> >>> Cc : >>> >>> >>> >>> On Thu, May 11, 2017 at 10:20 AM, Nick Papoylias <npapoylias(a)gmail.com> >>> wrote: >>> >>>> >>>> >>>> On Thu, May 11, 2017 at 5:24 AM, Oleksandr Zaytsev < >>>> olk.zaytsev(a)gmail.com> wrote: >>>> >>>>> >>>>> *A. Work done* >>>>> >>>>> - Downloaded the threaded VM as suggested by Esteban Lorenzano to >>>>> make Iceberg work. And it does! I have successfully pushed my NeuralNetwork >>>>> code to GitHub: https://github.com/olekscode/MLNeuralNetwork >>>>> - Joined the PolyMath organization on GitHub >>>>> - Created a repository for the TabularDataset project >>>>> https://github.com/PolyMathOrg/TabularDataset >>>>> <https://github.com/PolyMathOrg/TabularDataset> as a part of >>>>> PolyMath organization on GitHub >>>>> - Fixed a PolyMath issue #25 and made a PR >>>>> - Read an article from Wolfram Mathematica documentation regarding >>>>> Dataset. It was one of the reading suggestions sent to me by Nick Papoylias >>>>> >>>>> >>>>> *B. Next steps* >>>>> >>>>> - Fix more issues of PolyMath, using Iceberg. I have to get used >>>>> to it by the time the coding phase starts >>>>> - Read the rest of Nick Papoylias's suggestions >>>>> >>>>> >>>>> *C. Help needed* >>>>> >>>>> - The Dataset in Wolfram, as well as Pandas in Python, has a very >>>>> advanced indexing system. Smalltalk has its own special conventions for >>>>> indexing, so I think that it would be great if I got familiar with them. >>>>> Could you suggest me some reading on this topic (what are the indexing >>>>> conventions in Smalltalk?). >>>>> For example, in Wolfram, I can write *dataset[[-1]]* to extract >>>>> the last row. But in Pharo indexes can not be negative. In Pharo I would >>>>> say *dataset last*. But how about *dataset[[-5]]*? >>>>> >>>>> This would be a good exercise for you ;) In Pharo you can easily add >>>> negative indexing yourself. >>>> >>>> *Hint:* You know the index of the last element, since this is the size >>>> of the collection, so... ;) >>>> >>>> No need for changes, this exists already. >>> >>> Use atWrap: index put: value and atWrap: with negative indexes. >>> 'hello' atWrap: -2 >>> >>> There is a specific version for Array using a primitive. >>> #[ 10 20 30 40 ] atWrap: -1 >>> >>> atWrap:0 gives you the last item. >>> atWrap: -1 gives 30 >>> >>> This is different from 0 based index languages. >>> >>> The interesing thing about atWrap: is that it uses modulo interally so >>> you do not need to care about that. >>> >>> ($/ split: 'abc/def/ghi/jkl') atWrap: -1 >>> --> 'ghi' >>> >>> The Matrix class has a bunch of things API wise but the class is highly >>> inefficient, doing copies all the time etc. It would be nice to have some >>> kind of futures/copy on write style things in there. >>> >>> I miss cbind and rbind. These are useful. I have some half baked super >>> inefficient implementations of these things for Matrix. >>> >>> https://stat.ethz.ch/R-manual/R-devel/library/base/html/cbind.html >>> >>> The ability to name columns is also nice to have. >>> >>> In R one does: >>> >>> df <- dataframe() >>> cbind(df, c(1,2,3)) >>> cbind(df, c(4,5,6)) >>> names(df)<-("C1", "C2", "C3") >>> names can be found back with: >>> >>> names(df) >>> >>> A Smalltalkish style would be welcome. >>> >>> >>> >>> >>> Interesting ! Are you coming to PharoDays ? We can talk about that if we >>> found time. >>> >>> Maybe looking at the Voyage queries can be helpful. >>> >>> Phil >>> >>> >>> >>>> Try adding an extention method to Ordrered or SequenceableCollection. >>>> >>>> If the Pharo by example chapter is not enough or the MOOC, read the >>>> source >>>> itself in the core, to see how basic methods are implemented (it is >>>> less scary, >>>> than it sounds). >>>> >>>> You can also try Chapters 9, 10, 11 of the blue book (some API changes >>>> may apply): >>>> >>>> <http://goog_1902892863> >>>> http://sdmeta.gforge.inria.fr/FreeBooks/BlueBook/Bluebook.pdf >>>> >>>> >>>>> - Or what is the best way of implementing this index: >>>>> *dataset[["name"]]* (extracts a named row), *dataset[[1]*] >>>>> (extracts the first row)? Should I create two separate messages: *dataset >>>>> rowNamed: 'name'* and *dataset rowAt: 1*? >>>>> >>>>> rowNamed: >>> rowAt: >>> >>> yes, look like it. >>> >>> But if we want to model things like R dataframes for example, this has >>> to be seen as a vectorized operation, so you can to use row slices, column >>> slices, and logical indexes. >>> >>> Check this out: >>> >>> http://www.r-tutor.com/r-introduction/data-frame/data-frame-row-slice >>> https://www.r-bloggers.com/working-with-data-frames/ >>> >>> >>> >>>> The internal representation of your data-structure can be anything at >>>> the moment, *as long as you encapsulate it.* >>>> >>>> (ie it can be nested OrderedCollections with meta-data for column-names >>>> to indexes, or dictionary of collections etc). >>>> >>>> *If you don't expose it to the user* (ie return it from the public >>>> api, or expect knowledge of it in argument passing), >>>> we can easily change it later. So *first make it work, and we optimize >>>> later ;)* >>>> >>>> For your case it will be a little bit trickier because *you also have >>>> the notions of a) rows and b) columns*, which >>>> are exposed to the user. So *you would need to create abstractions* >>>> for these too. >>>> >>>> Cheers, >>>> >>>> Nick >>>> >>>>> >>>>> - >>>>> >>>>> >>>>> If someone else is having problems with Iceberg on Linux, try >>>>> downloading the threaded VM: >>>>> >>>>> wget -O- get.pharo.org/vmT60 | bash >>>>> >>>>> And use SSH (not HTTPS) remote URL. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Pharo Google Summer of Code" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to pharo-gsoc+unsubscribe(a)googlegroups.com. >>>>> To post to this group, send email to pharo-gsoc(a)googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/pharo-gsoc/CAEp0Uzu-8fw3dA >>>>> 6ezVoj-QptvLcB8cWPHvZ1tfLg1Ce8qkTqfQ%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/pharo-gsoc/CAEp0Uzu-8fw3dA6ezVoj-QptvLcB8…> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Pharo Google Summer of Code" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to pharo-gsoc+unsubscribe(a)googlegroups.com. >>>> To post to this group, send email to pharo-gsoc(a)googlegroups.com. >>>> To view this discussion on the web visit https://groups.google.com/d/ms >>>> gid/pharo-gsoc/CACEStOgLC6HbYJ8HBLHWfs5%2BwqN3ib_kdVGuVizx7G >>>> h1c0sM%3DA%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/pharo-gsoc/CACEStOgLC6HbYJ8HBLHWfs5%2BwqN…> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >> >

May 17, 2017

Re: [Pharo-users] Loading Glorp+PostgreSQL

by Esteban A. Maringolo

2017-05-17 11:04 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > >> On 17 May 2017, at 16:01, Pierce Ng <pierce(a)samadhiweb.com> wrote: >> >> On Tue, May 16, 2017 at 11:58:50AM +0200, Sven Van Caekenberghe wrote: >>> I am bringing my Reddit example back up to date (https://medium.com/concerning-pharo/reddit-st-in-10-cool-pharo-classes-1b53…) for an upcoming Pharo Booklet. >> I've adapted your Reddit application to work with Glorp-SQLite. It was pretty >> much simply subclassing RedditDatabaseResource and wiring up #newGlorpSession. >> It's a nice demo of Glorp's pluggability. > > That is cool. We should definitively keep a reference to that. Thanks. And it's better to get a non devops savvy user to get started, because it avoids installing a PgSQL server. Regards! Esteban A. Maringolo

May 17, 2017

Re: [Pharo-users] Ston feature idea?

by Cyril Ferlicot D.

On 17/05/2017 16:33, Sven Van Caekenberghe wrote: > > > You change the representation of MooseEntity to the above short form, then you just use MooseEntity instances inside other objects. All other instance variables of MooseEntity will be skipped. > > Just like Date, Time, or DateAndTime have short representations, that you see when you include these as values inside other objects. > > We must be talking about something different because I really don't see the problem. > > > Now I see why I did not understood. I wanted the container to change his variables containing the moose entities. So I though that you talked about the container of the MooseEntity and not about the moose entity itself. For now I wanted to avoid modifying the ston properties of MooseEntity because we have other ston exporters that export some of them with hacks and I did not wanted to update all of them at the same time. (It risk to break some features and we try to stabilize our tools) But I think I will need to do it. -- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques PrÃ©vert 01, 59650 Villeneuve d'ascq France

May 17, 2017

Re: [Pharo-users] Ston feature idea?

by Sven Van Caekenberghe

> On 17 May 2017, at 16:27, Cyril Ferlicot D. <cyril.ferlicot(a)gmail.com> wrote: > > On 17/05/2017 16:25, Sven Van Caekenberghe wrote: >> >> >> OK. >> >> So there is some existing method that gives you the entity given its name ? >> >> MooseEntity class>>fromEntityName: 'mooseEntityName' >> >> Right ? >> >> So why can't you use the example that I gave you earlier, replace the UUID string with the mooseEntityName string, and call the above method instead of the #fromId: in my example ? >> >> In STON, you will then see >> >> MooseEntity['mooseEntityName'] >> >> as value. >> > > But it would not export/import all the other variables no? You change the representation of MooseEntity to the above short form, then you just use MooseEntity instances inside other objects. All other instance variables of MooseEntity will be skipped. Just like Date, Time, or DateAndTime have short representations, that you see when you include these as values inside other objects. We must be talking about something different because I really don't see the problem. >> >> > > > -- > Cyril Ferlicot > https://ferlicot.fr > > http://www.synectique.eu > 2 rue Jacques PrÃ©vert 01, > 59650 Villeneuve d'ascq France >

May 17, 2017

Re: [Pharo-users] Ston feature idea?

by Cyril Ferlicot D.

On 17/05/2017 16:25, Sven Van Caekenberghe wrote: > > > OK. > > So there is some existing method that gives you the entity given its name ? > > MooseEntity class>>fromEntityName: 'mooseEntityName' > > Right ? > > So why can't you use the example that I gave you earlier, replace the UUID string with the mooseEntityName string, and call the above method instead of the #fromId: in my example ? > > In STON, you will then see > > MooseEntity['mooseEntityName'] > > as value. > But it would not export/import all the other variables no? > > -- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques PrÃ©vert 01, 59650 Villeneuve d'ascq France

May 17, 2017

Re: [Pharo-users] Ston feature idea?

by Sven Van Caekenberghe

> On 17 May 2017, at 16:15, Cyril Ferlicot D. <cyril.ferlicot(a)gmail.com> wrote: > > On 17/05/2017 16:02, Sven Van Caekenberghe wrote: >> >> OK, that is an understandable example. >> >> But why exactly do you want Currency to be serialised differently ? You don't want too many instances ? You want all instances to be #== ? Is Currency too big ? You want to allow humans to edit the STON file ? What ? Why ? >> >> You can easily make it Currency['Euro'] or even Currency[#Euro] like it is already done for a number of built-in classes. >> >> The last example that I gave solved the 'too many instances', make them #== issue. >> >> If it is size, that got covered too, but you must solve the problem of how the receiving end will resolve the reference then. >> >> But since that is all no good, there must still be another requirement. > > Here it is a simple example to show what I want. > > The real use case is that in Moose, entities are really connected to > their model, and if one of our classes contains an entity it would > produce a really really really big ston file since it would export > millions of entities. > > But there is a way to retrieve an entity of this model from its moose name. > > Thus, I would like to export only the moose name of the entities, then > get the corresponding entity from a Model the user would have load > before during the ston reading. OK. So there is some existing method that gives you the entity given its name ? MooseEntity class>>fromEntityName: 'mooseEntityName' Right ? So why can't you use the example that I gave you earlier, replace the UUID string with the mooseEntityName string, and call the above method instead of the #fromId: in my example ? In STON, you will then see MooseEntity['mooseEntityName'] as value. >> >> > > > -- > Cyril Ferlicot > https://ferlicot.fr > > http://www.synectique.eu > 2 rue Jacques PrÃ©vert 01, > 59650 Villeneuve d'ascq France

May 17, 2017

Re: [Pharo-users] Ston feature idea?

by Cyril Ferlicot D.

On 17/05/2017 16:02, Sven Van Caekenberghe wrote: > > OK, that is an understandable example. > > But why exactly do you want Currency to be serialised differently ? You don't want too many instances ? You want all instances to be #== ? Is Currency too big ? You want to allow humans to edit the STON file ? What ? Why ? > > You can easily make it Currency['Euro'] or even Currency[#Euro] like it is already done for a number of built-in classes. > > The last example that I gave solved the 'too many instances', make them #== issue. > > If it is size, that got covered too, but you must solve the problem of how the receiving end will resolve the reference then. > > But since that is all no good, there must still be another requirement. Here it is a simple example to show what I want. The real use case is that in Moose, entities are really connected to their model, and if one of our classes contains an entity it would produce a really really really big ston file since it would export millions of entities. But there is a way to retrieve an entity of this model from its moose name. Thus, I would like to export only the moose name of the entities, then get the corresponding entity from a Model the user would have load before during the ston reading. > > -- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques PrÃ©vert 01, 59650 Villeneuve d'ascq France

May 17, 2017

Re: [Pharo-users] Loading Glorp+PostgreSQL

by Sven Van Caekenberghe

> On 17 May 2017, at 16:01, Pierce Ng <pierce(a)samadhiweb.com> wrote: > > On Tue, May 16, 2017 at 11:58:50AM +0200, Sven Van Caekenberghe wrote: >> I am bringing my Reddit example back up to date (https://medium.com/concerning-pharo/reddit-st-in-10-cool-pharo-classes-1b53…) for an upcoming Pharo Booklet. > > Hi Sven, > > I've adapted your Reddit application to work with Glorp-SQLite. It was pretty > much simply subclassing RedditDatabaseResource and wiring up #newGlorpSession. > It's a nice demo of Glorp's pluggability. That is cool. We should definitively keep a reference to that. Thanks. > Pierce >

May 17, 2017

Re: [Pharo-users] Ston feature idea?

by Sven Van Caekenberghe

> On 17 May 2017, at 15:47, Cyril Ferlicot D. <cyril.ferlicot(a)gmail.com> wrote: > > On 17/05/2017 15:17, Sven Van Caekenberghe wrote: >> >> >> OK. >> >> But don't go too far in writing actual code. I want to understand the use case at a high level (like how you started), then I can try to find how/if STON can do what you want. Right now, you are already too deep into a possible solution. >> >> > > If I have to give a really simple use case. > > Imagine an invoice. > > The invoice has the name of the customer and the currency used as a class. > > During the export of the invoice you do not want to export the currency > as a class but as a String. > > So if I have an Invoice instance like this: > > username: 'user' > currency: Currency[Euro] > > I want to transform the currency to a string during the export. (and > keep the username as strings). > > Result: > > Invoice { > #username : 'user', > #currency : 'eur' > } > > Instead of: > > Invoice { > #username : 'user', > #currency : Currency { > #unit: 'Euro' > } > } > > Then during the import I want to transform the currency into a class > again. (For example `Currency fromAbbreviation: stonInput`). OK, that is an understandable example. But why exactly do you want Currency to be serialised differently ? You don't want too many instances ? You want all instances to be #== ? Is Currency too big ? You want to allow humans to edit the STON file ? What ? Why ? You can easily make it Currency['Euro'] or even Currency[#Euro] like it is already done for a number of built-in classes. The last example that I gave solved the 'too many instances', make them #== issue. If it is size, that got covered too, but you must solve the problem of how the receiving end will resolve the reference then. But since that is all no good, there must still be another requirement. > -- > Cyril Ferlicot > https://ferlicot.fr > > http://www.synectique.eu > 2 rue Jacques PrÃ©vert 01, > 59650 Villeneuve d'ascq France >

May 17, 2017

Re: [Pharo-users] Loading Glorp+PostgreSQL

by Pierce Ng

On Tue, May 16, 2017 at 11:58:50AM +0200, Sven Van Caekenberghe wrote: > I am bringing my Reddit example back up to date (https://medium.com/concerning-pharo/reddit-st-in-10-cool-pharo-classes-1b53…) for an upcoming Pharo Booklet. Hi Sven, I've adapted your Reddit application to work with Glorp-SQLite. It was pretty much simply subclassing RedditDatabaseResource and wiring up #newGlorpSession. It's a nice demo of Glorp's pluggability. Pierce

May 17, 2017