- Pharo-users - lists.pharo.org

Re: [Pharo-users] Dictionary whose values are dictionaries

by Stephane Ducasse

Tx peter. It is great to see you around!. It is on my todo to produce a booklet on Soup and your HTML scrapping tutorial :). You will see it will be nice. Stef On Sat, May 27, 2017 at 10:41 PM, PBKResearch <peter(a)pbkresearch.co.uk> wrote: > Stef > > > > Cyril is right; there is a class KeyedTree which is a subclass of > Dictionary and is in effect a set of nested dictionaries. There was a brief > discussion of this in April, initiated by Markus BÃ¶hm, who wanted a neat > way of looking up entries in nested dictionaries. KeyedTree uses the idea > of a path, which is an array of the successive keys needed to locate an > entry in an embedded dictionary. Sven also saw this, and has extended > NeoJSONObject to provide the same embedding and path lookup. To see Svenâs > brief explanation, find the original thread, entitled â [Pharo-users] Hot > to retrieve values from Nested Dictionariesâ (note the typo in the original > post) starting on 24 April 2017, or see the class comment for NeoJSONObject. > > > > Hope this helps. > > > > Peter Kenny > > > > > > *From:* Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] *On > Behalf Of *Cyril Ferlicot > *Sent:* 27 May 2017 19:42 > *To:* Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > *Subject:* Re: [Pharo-users] Dictionary whose values are dictionaries > > > > > > On sam. 27 mai 2017 at 20:35, Cyril Ferlicot <cyril.ferlicot(a)gmail.com> > wrote: > > > > On sam. 27 mai 2017 at 20:29, Stephane Ducasse <stepharo.self(a)gmail.com> > wrote: > > Hi > > > > I'm looking for an implementation of dictionary whose value is also a > dictionary. > > > > Stef > > Hi! > > > > I don't remember the exact name but I know that an algo at synectique use > such a collection. The name is something like Ktree or something like this > but I don't have a computer to check this week end. > > > > â > > From my phone > > Cyril > > -- > > Cheers > > Cyril Ferlicot > > > > > > I think the exact name was KeyedTree. > > > > â > > From my phone > > Cyril > > -- > > Cheers > > Cyril Ferlicot >

May 28, 2017

Re: [Pharo-users] [Pharo-dev] Chrome DevTools Protocol and Pharo

by Stephane Ducasse

Thanks Alistair I really like your attitude! Bring ideas to reality, create solutions,... Let us invent our future On Sun, May 28, 2017 at 8:44 AM, Alistair Grant <akgrant0710(a)gmail.com> wrote: > (Moving to pharo-users as Serge suggested) > > Sending the email also made me realise that it would be possible to > check for the presence of a particular node while waiting for Chrome to > load the page, which would avoid the long timeout. I'll look at adding > this later. > > Cheers, > Alistair > > > On Sun, May 28, 2017 at 10:49:14AM +0700, serge.stinckwich(a)gmail.com > wrote: > > Thank you guys for your contributions, but I think that this kind of > > discussions should be done on pharo-users mailing-list, not here. > > > > pharo-dev should be use only if you contribute to core-pharo. > > > > Thank you > > > > Envoy? de mon iPhone > > > > On May 27, 2017, at 3:23 PM, Alistair Grant [via Smalltalk] > > <[hidden email] > > > wrote: > > > > > > Hi Torsten and All, > > > > Quick Introduction for those not familiar with Pharo-Chrome: > > > > Pharo-Chrome enables Pharo to control and query Chrome / > > Chromium, in particular to retrieve the DOM of a page. This > > is useful as many modern pages are just a template which then > > loads some javascript to asynchronously build the DOM, meaning > > that the ZnEasy / Soap combination doesn't get the bulk of the > > information on a page. > > > > > > > > Pharo-Chrome is now mostly working, i.e. it is possible to > > open a connection to Chrome, navigate to a requested URL, wait > > for it to load, retrieve the DOM and then navigate the DOM > > using a subset of the Soap API, e.g. #findAllStrings:, > > #findAllTags:, attributeAt:, etc.. > > > > GoogleChrome class>>exampleNavigation has been updated to > > retrieve the DOM from http://pharo.org. > > > > GoogleChrome class>>get: is analogous to ZnEasy class>>get:, > > although it returns a ChromeNode, not an html string. > > > > I wasn't able to get rid of the delay while waiting for the > > page to finish loading. This actually makes sense, since, as > > mentioned above, many modern pages build the DOM > > asynchronously, so there's no clear indication of when it is > > complete. The default delay is currently 2000 milliseconds, > > which is about twice the maximum I saw needed (983ms), but > > this can be changed (ChromeTabPage>>pageLoadDelay:). > > > > I had three use cases for this library: one which works with > > ZnEasy+Soap, one that used to work with ZnEasy+Soap, but > > doesn't due to a page redesign, and one which I hadn't got > > working before. All three are working now. > > > > Unlike Soap, I've currently modelled the nodes as a single > > class, and have only implemented a subset of Soap's methods, > > but is enough for what I need. > > > > I've introduced a dependency on the Beacon logging framework. > > I find it useful, but can remove it if you don't want the > > additional dependency. (I'm planning to add some GoogleChrome > > specific logging classes and use those to better understand > > what pageLoadDelay should be). > > > > I was focussed on trying to understand the events that Chrome > > generates, so documentation is still lacking (read "missing" > > :-)). > > > > I'll generate a pull request after some more testing, tweaking > > and documenting, but if you would like to take a look, the > > code is available at: > > > > https://github.com/akgrant43/Pharo-Chrome/tree/development > > > > I haven't yet updated BaselineOfChrome with the Beacon > > dependency. I did merge in your two commits from May 23. > > > > If you, or anyone else, finds this useful, I welcome any > > feedback. > > > > P.S. I've just realised that I need to tidy up #sendMessage:, > > #sendMessageDictionary and #sendMessageDictionary:wait:. I'll > > do that as part of the genral tidy up. > > > > Cheers, Alistair > > > > > > On Sun, May 21, 2017 at 09:37:56PM +0000, > > Alistair Grant wrote: > > > > > Hi Torsten, > > > > > > On Fri, May 19, 2017 at 09:20:48PM +0000, Alistair Grant > > > wrote: > > > > > > > > On Fri, May 19, 2017 at 10:50:41PM +0200, Torsten Bergmann > > > > wrote: > > > > > Hi Alistair, > > > > > > > > > > cant look right now but two things: > > > > > > > > > > - there are also events in the protocol - if we could > > > > > hook Pharo into them > > > > > this would solve the problem without abusing delay > > > > > (because then you will > > > > > get informed when the page loading is finished) > > > > > > > > That would be great. It will be a while before I get a > > > > chance to look > > > > at this (I want to finish some proposed changes to the > > > > FileSystem packages first), but I'll try and include it > > > > then. > > > > > > I've got basic event listening working. It requires that > > > all messages > > > are read asynchronously, so I'll need to change the > > > interface to handle that. > > > > > > Knowing when a page has finished loading isn't quite as > > > simple as looking for an event - a page can consist of > > > multiple frames, and notifications are delivered for each > > > frame. The page I'm interested in > > > has around 25 frames. > > > > > > If anyone has a good design pattern for writing an > > > asynchronous WebSocket client please let me know, I don't > > > have anything concrete in mind. > > > > > > Thanks, Alistair > >

May 28, 2017

Re: [Pharo-users] [Pharo-dev] Chrome DevTools Protocol and Pharo

by Alistair Grant

(Moving to pharo-users as Serge suggested) Sending the email also made me realise that it would be possible to check for the presence of a particular node while waiting for Chrome to load the page, which would avoid the long timeout. I'll look at adding this later. Cheers, Alistair On Sun, May 28, 2017 at 10:49:14AM +0700, serge.stinckwich(a)gmail.com wrote: > Thank you guys for your contributions, but I think that this kind of > discussions should be done on pharo-users mailing-list, not here. > > pharo-dev should be use only if you contribute to core-pharo. > > Thank you > > Envoy? de mon iPhone > > On May 27, 2017, at 3:23 PM, Alistair Grant [via Smalltalk] > <[hidden email] > > wrote: > > > Hi Torsten and All, > > Quick Introduction for those not familiar with Pharo-Chrome: > > Pharo-Chrome enables Pharo to control and query Chrome / > Chromium, in particular to retrieve the DOM of a page. This > is useful as many modern pages are just a template which then > loads some javascript to asynchronously build the DOM, meaning > that the ZnEasy / Soap combination doesn't get the bulk of the > information on a page. > > > > Pharo-Chrome is now mostly working, i.e. it is possible to > open a connection to Chrome, navigate to a requested URL, wait > for it to load, retrieve the DOM and then navigate the DOM > using a subset of the Soap API, e.g. #findAllStrings:, > #findAllTags:, attributeAt:, etc.. > > GoogleChrome class>>exampleNavigation has been updated to > retrieve the DOM from http://pharo.org. > > GoogleChrome class>>get: is analogous to ZnEasy class>>get:, > although it returns a ChromeNode, not an html string. > > I wasn't able to get rid of the delay while waiting for the > page to finish loading. This actually makes sense, since, as > mentioned above, many modern pages build the DOM > asynchronously, so there's no clear indication of when it is > complete. The default delay is currently 2000 milliseconds, > which is about twice the maximum I saw needed (983ms), but > this can be changed (ChromeTabPage>>pageLoadDelay:). > > I had three use cases for this library: one which works with > ZnEasy+Soap, one that used to work with ZnEasy+Soap, but > doesn't due to a page redesign, and one which I hadn't got > working before. All three are working now. > > Unlike Soap, I've currently modelled the nodes as a single > class, and have only implemented a subset of Soap's methods, > but is enough for what I need. > > I've introduced a dependency on the Beacon logging framework. > I find it useful, but can remove it if you don't want the > additional dependency. (I'm planning to add some GoogleChrome > specific logging classes and use those to better understand > what pageLoadDelay should be). > > I was focussed on trying to understand the events that Chrome > generates, so documentation is still lacking (read "missing" > :-)). > > I'll generate a pull request after some more testing, tweaking > and documenting, but if you would like to take a look, the > code is available at: > > https://github.com/akgrant43/Pharo-Chrome/tree/development > > I haven't yet updated BaselineOfChrome with the Beacon > dependency. I did merge in your two commits from May 23. > > If you, or anyone else, finds this useful, I welcome any > feedback. > > P.S. I've just realised that I need to tidy up #sendMessage:, > #sendMessageDictionary and #sendMessageDictionary:wait:. I'll > do that as part of the genral tidy up. > > Cheers, Alistair > > > On Sun, May 21, 2017 at 09:37:56PM +0000, > Alistair Grant wrote: > > > Hi Torsten, > > > > On Fri, May 19, 2017 at 09:20:48PM +0000, Alistair Grant > > wrote: > > > > > > On Fri, May 19, 2017 at 10:50:41PM +0200, Torsten Bergmann > > > wrote: > > > > Hi Alistair, > > > > > > > > cant look right now but two things: > > > > > > > > - there are also events in the protocol - if we could > > > > hook Pharo into them > > > > this would solve the problem without abusing delay > > > > (because then you will > > > > get informed when the page loading is finished) > > > > > > That would be great. It will be a while before I get a > > > chance to look > > > at this (I want to finish some proposed changes to the > > > FileSystem packages first), but I'll try and include it > > > then. > > > > I've got basic event listening working. It requires that > > all messages > > are read asynchronously, so I'll need to change the > > interface to handle that. > > > > Knowing when a page has finished loading isn't quite as > > simple as looking for an event - a page can consist of > > multiple frames, and notifications are delivered for each > > frame. The page I'm interested in > > has around 25 frames. > > > > If anyone has a good design pattern for writing an > > asynchronous WebSocket client please let me know, I don't > > have anything concrete in mind. > > > > Thanks, Alistair

May 28, 2017

Re: [Pharo-users] [Pharo-dev] Chrome DevTools Protocol and Pharo

by Alistair Grant

Hi Aik-Siong, (Moving to pharo-users as Serge suggestion) On Sat, May 27, 2017 at 08:22:27PM -0700, askoh wrote: > Wonderful contribution. Having full control of an internet browser will give > Smalltalk more freedom to innovate. > > Now, can we make Pharo be a web server on demand? Then we can give anyone a URL > and they can see the app or info we want to present. Pharo can communicate with > anyone using a browser either automatically or under developer control. Pharo includes the Zinc framework, which includes a basic server (ZnServer). There's also Seaside, which is more complete for making web applications. Cheers, Alistair

May 28, 2017

Re: [Pharo-users] Dictionary whose values are dictionaries

by monty

You could just use a regular Dictionary, but with association keys: dict at: outerKey -> innerKey put: innerValue

May 28, 2017

Re: [Pharo-users] Dictionary whose values are dictionaries

by Bernardo Ezequiel Contreras

hi, i'm using a multi-valued dictionary https://github.com/vonbecmann/multi-valued-dictionary in the following way itemsByKey := MultiValuedDictionary dictionary: IdentityDictionary collection: IdentityDictionary so i can check the existence of an item by its key (an ordered pair) (itemsByKey at: anItem key first ifAbsent: nil) ifNotNil: [ :dict | (dict at: anItem key second ifAbsent: nil) ifNotNil: [ :found | ^ found ] ]. and i add an item like this itemsByKey at: anItem key first add: (Association basicNew key: anItem key second value: anItem). hth On Sat, May 27, 2017 at 3:28 PM, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: > Hi > > I'm looking for an implementation of dictionary whose value is also a > dictionary. > > Stef > -- Bernardo E.C. Sent from a cheap desktop computer in South America.

May 27, 2017

Re: [Pharo-users] Dictionary whose values are dictionaries

by PBKResearch

Stef Cyril is right; there is a class KeyedTree which is a subclass of Dictionary and is in effect a set of nested dictionaries. There was a brief discussion of this in April, initiated by Markus BÃ¶hm, who wanted a neat way of looking up entries in nested dictionaries. KeyedTree uses the idea of a path, which is an array of the successive keys needed to locate an entry in an embedded dictionary. Sven also saw this, and has extended NeoJSONObject to provide the same embedding and path lookup. To see Svenâs brief explanation, find the original thread, entitled â [Pharo-users] Hot to retrieve values from Nested Dictionariesâ (note the typo in the original post) starting on 24 April 2017, or see the class comment for NeoJSONObject. Hope this helps. Peter Kenny From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of Cyril Ferlicot Sent: 27 May 2017 19:42 To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> Subject: Re: [Pharo-users] Dictionary whose values are dictionaries On sam. 27 mai 2017 at 20:35, Cyril Ferlicot <cyril.ferlicot(a)gmail.com <mailto:cyril.ferlicot@gmail.com> > wrote: On sam. 27 mai 2017 at 20:29, Stephane Ducasse <stepharo.self(a)gmail.com <mailto:stepharo.self@gmail.com> > wrote: Hi I'm looking for an implementation of dictionary whose value is also a dictionary. Stef Hi! I don't remember the exact name but I know that an algo at synectique use such a collection. The name is something like Ktree or something like this but I don't have a computer to check this week end. â >From my phone Cyril -- Cheers Cyril Ferlicot I think the exact name was KeyedTree. â >From my phone Cyril -- Cheers Cyril Ferlicot

May 27, 2017

Re: [Pharo-users] Dictionary whose values are dictionaries

by Cyril Ferlicot

On sam. 27 mai 2017 at 20:35, Cyril Ferlicot <cyril.ferlicot(a)gmail.com> wrote: > > On sam. 27 mai 2017 at 20:29, Stephane Ducasse <stepharo.self(a)gmail.com> > wrote: > >> Hi >> >> I'm looking for an implementation of dictionary whose value is also a >> dictionary. >> >> Stef >> > Hi! > > I don't remember the exact name but I know that an algo at synectique use > such a collection. The name is something like Ktree or something like this > but I don't have a computer to check this week end. > > â > From my phone > Cyril > -- > Cheers > Cyril Ferlicot > I think the exact name was KeyedTree. â >From my phone Cyril -- Cheers Cyril Ferlicot

May 27, 2017

Re: [Pharo-users] Dictionary whose values are dictionaries

by phil＠highoctane.be

Looks like a JsonObject On Sat, May 27, 2017 at 8:28 PM, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: > Hi > > I'm looking for an implementation of dictionary whose value is also a > dictionary. > > Stef >

May 27, 2017

Re: [Pharo-users] Dictionary whose values are dictionaries

by Cyril Ferlicot

On sam. 27 mai 2017 at 20:29, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: > Hi > > I'm looking for an implementation of dictionary whose value is also a > dictionary. > > Stef > Hi! I don't remember the exact name but I know that an algo at synectique use such a collection. The name is something like Ktree or something like this but I don't have a computer to check this week end. â >From my phone Cyril -- Cheers Cyril Ferlicot

May 27, 2017