February 2018 - Pharo-users

Re: [Pharo-users] Web scrapping with Pharo Chrome

by Alistair Grant

Hi Offray, On 14 February 2018 at 20:29, Offray Vladimir Luna CÃ¡rdenas <offray.luna(a)mutabit.com> wrote: > Yes. Me too. Alistair, any starting points with this example? I will take > from there and we could get visibility in the upcoming Open Data Day. I'm not sure that I understand what you're after, but maybe the following will help. This simply returns a collection of all the h4 headings that are in cells: | rootNode divs cells cellTitleNodes cellTitles | rootNode := GoogleChrome get: 'http://mutabit.com/grafoscopio/index.en.html'. divs := rootNode findAllTags: 'div'. cells := divs select: [ :each | (' ' split: (each attributeAt: 'class')) includes: 'mdl-cell' ]. cellTitleNodes := cells flatCollect: [ :each | each findAllTags: 'h4' ]. cellTitles := cellTitleNodes collect: [ :each | (each findAllStrings: true) first nodeValue ]. { rootNode. divs. cells. cellTitleNodes. cellTitles } > Of course, we're going to document everything and share back (I have already > proposed some improvements in documentation via PR on the Git repo). Thanks very much for improving the readme. I've merged the PR. Cheers, Alistair > Cheers, > > Offray > > > On 14/02/18 14:13, Stephane Ducasse wrote: > > I would love to have a little how to and that we can turn it into a > document. > > Stef > > On Wed, Feb 14, 2018 at 5:25 PM, Offray Vladimir Luna CÃ¡rdenas > <offray.luna(a)mutabit.com> wrote: >> >> Hi, >> >> I have been finally able to install and use properly Pharo Chrome. The >> issues I reported in other thread, were caused by conflicts between >> OSProcess and OSSubProcess on Linux. Now I'm able to launch Chrome, point it >> to particular addresses and get some info from there. >> >> I would like to continue the conversation related with web scraping using >> Pharo Chrome, which comes handy now that there is a lot of React and other >> technologies making the web more and more opaque for digital citizenship and >> data activism endeavors. So, as an starting example, I would like to scrap >> the Grafoscopio's own page [1]. It makes use of Material Design Light [2]. >> >> [1] http://mutabit.com/grafoscopio/index.en.html >> [2] http://getmdl.io/ >> >> An starting example would be fine to kickstart myself (from the ones I'm >> reading, there is still something I don't get). Let's say I want to get all >> cards in [1] , as shown in the screenshot below. I know the div class of >> each one, and the div class where they are located. Which could be a minimal >> example of a scraper, to start with? >> >> Thanks, >> >> Offray >> >> > >

Feb. 15, 2018

Re: [Pharo-users] Pillar questions

by Arturo Zambrano

I found the baseline menu entry.. sorry [image: Inline image 1] installations fails when I choose Install baseline of Pillar (default) [image: Inline image 2] On Thu, Feb 15, 2018 at 12:50 PM, Arturo Zambrano <arturo.zambrano(a)gmail.com > wrote: > Thanks Stef & Guillermo. > > Sorry, I'm a bit confused with Icerberg and GIT support. > I'm using Iceberg and works fine for my small project without handling > dependencies in github. > > I don't understand how dependencies are managed in this case. > > What I understand is that Pillar7 lives in Github, I cloned that repo, > moved to the indicated branch > when I try to install the packages (Icerber ->load package) it, as > expected, > asks for its dependencies. That's why I tried ConfigurationOfPillar loadXYZ > > @Stef, I'm trying to load Pillar packages into my image (pharo 6.1) for > development. I tried the readme of that branch, I will report the errors I > got in a separate thread. > > @Guillermo do you mean to manually clone each dependency github repo > with iceberg? I think I'm missing something, some documentation maybe, any > pointer is welcome. > > Thanks > > > On Thu, Feb 15, 2018 at 6:01 AM, Guillermo Polito < > guillermopolito(a)gmail.com> wrote: > >> And you should load baselines with iceberg (and git). >> >> On Thu, Feb 15, 2018 at 8:25 AM, Stephane Ducasse < >> stepharo.self(a)gmail.com> wrote: >> >>> No you should use iceberg. The newpipeline is managed on github. >>> Have a look at the readme of the that branch >>> Before executing the build scripts check what it is doing. >>> >>> Stef >>> >>> >>> On Wed, Feb 14, 2018 at 10:00 PM, Arturo Zambrano < >>> arturo.zambrano(a)gmail.com> wrote: >>> >>>> Hi Guillermo >>>> >>>> I'd recommend to use the latest pillar version in this branch: >>>>> >>>>> https://github.com/pillar-markup/pillar/tree/newpipeline >>>>> >>>>> >>>> >>>> I cloned that repo (using Iceberg on 6.1 image), moved to that branch >>>> and then tried >>>> ConfigurationOfPillar load{stable|bleedingEdge| etc} >>>> >>>> To get all the dependencies. >>>> I got an error (see below) regarding magritte. >>>> >>>> Any advice? >>>> >>>> TIA >>>> >>>> [image: Inline image 1] >>>> >>>> >>>> >>>> >>>> >>>> >>>>> >>>>>> Thanks! >>>>>> Arturo >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> >>>>> >>>>> Guille Polito >>>>> >>>>> Research Engineer >>>>> >>>>> Centre de Recherche en Informatique, Signal et Automatique de Lille >>>>> >>>>> CRIStAL - UMR 9189 >>>>> >>>>> French National Center for Scientific Research - *http://www.cnrs.fr >>>>> <http://www.cnrs.fr>* >>>>> >>>>> >>>>> *Web:* *http://guillep.github.io* <http://guillep.github.io> >>>>> >>>>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >>>>> >>>> >>>> >>> >> >> >> -- >> >> >> >> Guille Polito >> >> Research Engineer >> >> Centre de Recherche en Informatique, Signal et Automatique de Lille >> >> CRIStAL - UMR 9189 >> >> French National Center for Scientific Research - *http://www.cnrs.fr >> <http://www.cnrs.fr>* >> >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io> >> >> *Phone: *+33 06 52 70 66 13 >> > >

Feb. 15, 2018

Re: [Pharo-users] global exception handler mechanism

by Denis Kudriashov

Hi 2018-02-05 11:00 GMT+01:00 Igor Stasenko <siguctua(a)gmail.com>: > > > On 5 February 2018 at 10:17, Marcus Denker <marcus.denker(a)inria.fr> wrote: > >> Hello, >> >> The Sunit-Debugger does a stack search on startup to find the test >> related exception. >> >> It might be good to do it better, especially as the debug process starts >> from the exception >> I might be a good idea to actually keep a reference to it. >> >> Would there be any downside of Exception>>debug passing a reference to >> the Debugger? >> >> Marcus >> >> >> > On 2 Feb 2018, at 17:19, Peter UhnÃ¡k <i.uhnak(a)gmail.com> wrote: >> > >> > Hi, >> > >> > is there a way to install a global handler for exceptions? >> > >> > Right now if I want to log all exceptions, I use approach from >> ShoreLine and create a PreDebugAction which is activated when any >> "unhandled" exception occurs. >> > >> > That would be good enough, however the Debugger has zero knowledge of >> the exception that is actually occuring, as Exception>>debug passes on only >> the title. So if I want to get back to the original exception and maybe log >> it with beacon, I need to do a lot of stack and context shenanigans: >> > >> > MyPreDebugAction>>logException >> > MyLogger >> > runDuring: [ (debugger session interruptedProcess >> suspendedContext stack >> > detect: [ :context | context receiver >> isKindOf: Exception ]) receiver emit ] >> > >> > >> > Is there a better way to approach this? >> > > if you own a process (you creating it) , then it is just a piece of cake, > just make a topmost block > with #on:do: > if you don't own a process (suppose you wanna watch already existing one) > , it can be done by injecting own context at the bottom of stack , > well, unless there are exception handler(s) upper, that catch any > exception(s) before you can see them. > I added it in Pharo 6: aProcess on: Error do: [:err | err logCr] It works in both cases: when process is not started and when it is already running. I used it to intercept errors of forked processes during test run. It would be cool to add #ensure: in addition. > > I think a much better way would be to watch exceptions even before they > being handled, > for this, i would override Exception>>#signal > and add some kind of logging or notification .. well, anything you see fit. > That's, of course, a kind of dangerous, be careful. Do not attempt to > throw new exceptions while processing just signaled one, else > you'll get infinite recursion :) > > > > >> > Thanks, >> > Peter >> >> >> > > > -- > Best regards, > Igor Stasenko. >

Feb. 15, 2018

Re: [Pharo-users] Pillar questions

by Arturo Zambrano

Thanks Stef & Guillermo. Sorry, I'm a bit confused with Icerberg and GIT support. I'm using Iceberg and works fine for my small project without handling dependencies in github. I don't understand how dependencies are managed in this case. What I understand is that Pillar7 lives in Github, I cloned that repo, moved to the indicated branch when I try to install the packages (Icerber ->load package) it, as expected, asks for its dependencies. That's why I tried ConfigurationOfPillar loadXYZ @Stef, I'm trying to load Pillar packages into my image (pharo 6.1) for development. I tried the readme of that branch, I will report the errors I got in a separate thread. @Guillermo do you mean to manually clone each dependency github repo with iceberg? I think I'm missing something, some documentation maybe, any pointer is welcome. Thanks On Thu, Feb 15, 2018 at 6:01 AM, Guillermo Polito <guillermopolito(a)gmail.com > wrote: > And you should load baselines with iceberg (and git). > > On Thu, Feb 15, 2018 at 8:25 AM, Stephane Ducasse <stepharo.self(a)gmail.com > > wrote: > >> No you should use iceberg. The newpipeline is managed on github. >> Have a look at the readme of the that branch >> Before executing the build scripts check what it is doing. >> >> Stef >> >> >> On Wed, Feb 14, 2018 at 10:00 PM, Arturo Zambrano < >> arturo.zambrano(a)gmail.com> wrote: >> >>> Hi Guillermo >>> >>> I'd recommend to use the latest pillar version in this branch: >>>> >>>> https://github.com/pillar-markup/pillar/tree/newpipeline >>>> >>>> >>> >>> I cloned that repo (using Iceberg on 6.1 image), moved to that branch >>> and then tried >>> ConfigurationOfPillar load{stable|bleedingEdge| etc} >>> >>> To get all the dependencies. >>> I got an error (see below) regarding magritte. >>> >>> Any advice? >>> >>> TIA >>> >>> [image: Inline image 1] >>> >>> >>> >>> >>> >>> >>>> >>>>> Thanks! >>>>> Arturo >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> >>>> >>>> Guille Polito >>>> >>>> Research Engineer >>>> >>>> Centre de Recherche en Informatique, Signal et Automatique de Lille >>>> >>>> CRIStAL - UMR 9189 >>>> >>>> French National Center for Scientific Research - *http://www.cnrs.fr >>>> <http://www.cnrs.fr>* >>>> >>>> >>>> *Web:* *http://guillep.github.io* <http://guillep.github.io> >>>> >>>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >>>> >>> >>> >> > > > -- > > > > Guille Polito > > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > > CRIStAL - UMR 9189 > > French National Center for Scientific Research - *http://www.cnrs.fr > <http://www.cnrs.fr>* > > > *Web:* *http://guillep.github.io* <http://guillep.github.io> > > *Phone: *+33 06 52 70 66 13 >

Feb. 15, 2018

Techtalk Today: Link

by Marcus Denker

Hi, TechTalk today at 17:00 Local European time (in 45 minutes) https://association.pharo.org/event-2797065 <https://association.pharo.org/event-2797065> https://www.youtube.com/c/MarcusDenker/live <https://www.youtube.com/c/MarcusDenker/live> Marcus

Feb. 15, 2018

Re: [Pharo-users] Pillar questions

by Guillermo Polito

And you should load baselines with iceberg (and git). On Thu, Feb 15, 2018 at 8:25 AM, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: > No you should use iceberg. The newpipeline is managed on github. > Have a look at the readme of the that branch > Before executing the build scripts check what it is doing. > > Stef > > > On Wed, Feb 14, 2018 at 10:00 PM, Arturo Zambrano < > arturo.zambrano(a)gmail.com> wrote: > >> Hi Guillermo >> >> I'd recommend to use the latest pillar version in this branch: >>> >>> https://github.com/pillar-markup/pillar/tree/newpipeline >>> >>> >> >> I cloned that repo (using Iceberg on 6.1 image), moved to that branch and >> then tried >> ConfigurationOfPillar load{stable|bleedingEdge| etc} >> >> To get all the dependencies. >> I got an error (see below) regarding magritte. >> >> Any advice? >> >> TIA >> >> [image: Inline image 1] >> >> >> >> >> >> >>> >>>> Thanks! >>>> Arturo >>>> >>> >>> >>> >>> -- >>> >>> >>> >>> Guille Polito >>> >>> Research Engineer >>> >>> Centre de Recherche en Informatique, Signal et Automatique de Lille >>> >>> CRIStAL - UMR 9189 >>> >>> French National Center for Scientific Research - *http://www.cnrs.fr >>> <http://www.cnrs.fr>* >>> >>> >>> *Web:* *http://guillep.github.io* <http://guillep.github.io> >>> >>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >>> >> >> > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Feb. 15, 2018

Re: [Pharo-users] Web scrapping with Pharo Chrome

by Stephane Ducasse

super. Stef On Wed, Feb 14, 2018 at 8:29 PM, Offray Vladimir Luna CÃ¡rdenas < offray.luna(a)mutabit.com> wrote: > Yes. Me too. Alistair, any starting points with this example? I will take > from there and we could get visibility in the upcoming Open Data Day. > > Of course, we're going to document everything and share back (I have > already proposed some improvements in documentation via PR on the Git repo). > > Cheers, > > Offray > > On 14/02/18 14:13, Stephane Ducasse wrote: > > I would love to have a little how to and that we can turn it into a > document. > > Stef > > On Wed, Feb 14, 2018 at 5:25 PM, Offray Vladimir Luna CÃ¡rdenas < > offray.luna(a)mutabit.com> wrote: > >> Hi, >> >> I have been finally able to install and use properly Pharo Chrome. The >> issues I reported in other thread, were caused by conflicts between >> OSProcess and OSSubProcess on Linux. Now I'm able to launch Chrome, point >> it to particular addresses and get some info from there. >> >> I would like to continue the conversation related with web scraping using >> Pharo Chrome, which comes handy now that there is a lot of React and other >> technologies making the web more and more opaque for digital citizenship >> and data activism endeavors. So, as an starting example, I would like to >> scrap the Grafoscopio's own page [1]. It makes use of Material Design Light >> [2]. >> >> [1] http://mutabit.com/grafoscopio/index.en.html >> [2] http://getmdl.io/ >> >> An starting example would be fine to kickstart myself (from the ones I'm >> reading, there is still something I don't get). Let's say I want to get all >> cards in [1] , as shown in the screenshot below. I know the div class of >> each one, and the div class where they are located. Which could be a >> minimal example of a scraper, to start with? >> >> Thanks, >> >> Offray >> >> > >

Feb. 15, 2018

Re: [Pharo-users] Pillar questions

by Stephane Ducasse

No you should use iceberg. The newpipeline is managed on github. Have a look at the readme of the that branch Before executing the build scripts check what it is doing. Stef On Wed, Feb 14, 2018 at 10:00 PM, Arturo Zambrano <arturo.zambrano(a)gmail.com > wrote: > Hi Guillermo > > I'd recommend to use the latest pillar version in this branch: >> >> https://github.com/pillar-markup/pillar/tree/newpipeline >> >> > > I cloned that repo (using Iceberg on 6.1 image), moved to that branch and > then tried > ConfigurationOfPillar load{stable|bleedingEdge| etc} > > To get all the dependencies. > I got an error (see below) regarding magritte. > > Any advice? > > TIA > > [image: Inline image 1] > > > > > > >> >>> Thanks! >>> Arturo >>> >> >> >> >> -- >> >> >> >> Guille Polito >> >> Research Engineer >> >> Centre de Recherche en Informatique, Signal et Automatique de Lille >> >> CRIStAL - UMR 9189 >> >> French National Center for Scientific Research - *http://www.cnrs.fr >> <http://www.cnrs.fr>* >> >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io> >> >> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >> > >

Feb. 15, 2018

Re: [Pharo-users] Pillar questions

by Arturo Zambrano

Hi Guillermo I'd recommend to use the latest pillar version in this branch: > > https://github.com/pillar-markup/pillar/tree/newpipeline > > I cloned that repo (using Iceberg on 6.1 image), moved to that branch and then tried ConfigurationOfPillar load{stable|bleedingEdge| etc} To get all the dependencies. I got an error (see below) regarding magritte. Any advice? TIA [image: Inline image 1] > >> Thanks! >> Arturo >> > > > > -- > > > > Guille Polito > > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > > CRIStAL - UMR 9189 > > French National Center for Scientific Research - *http://www.cnrs.fr > <http://www.cnrs.fr>* > > > *Web:* *http://guillep.github.io* <http://guillep.github.io> > > *Phone: *+33 06 52 70 66 13 >

Feb. 14, 2018

Re: [Pharo-users] Web scrapping with Pharo Chrome

by Offray Vladimir Luna CÃ¡rdenas

Yes. Me too. Alistair, any starting points with this example? I will take from there and we could get visibility in the upcoming Open Data Day. Of course, we're going to document everything and share back (I have already proposed some improvements in documentation via PR on the Git repo). Cheers, Offray On 14/02/18 14:13, Stephane Ducasse wrote: > I would love to have a little how to and that we can turn it into a > document.Â > > Stef > > On Wed, Feb 14, 2018 at 5:25 PM, Offray Vladimir Luna CÃ¡rdenas > <offray.luna(a)mutabit.com <mailto:offray.luna@mutabit.com>> wrote: > > Hi, > > I have been finally able to install and use properly Pharo Chrome. > The issues I reported in other thread, were caused by conflicts > between OSProcess and OSSubProcess on Linux. Now I'm able to > launch Chrome, point it to particular addresses and get some info > from there. > > I would like to continue the conversation related with web > scraping using Pharo Chrome, which comes handy now that there is a > lot of React and other technologies making the web more and more > opaque for digital citizenship and data activism endeavors. So, as > an starting example, I would like to scrap the Grafoscopio's own > page [1]. It makes use of Material Design Light [2]. > > [1] http://mutabit.com/grafoscopio/index.en.html > <http://mutabit.com/grafoscopio/index.en.html> > [2] http://getmdl.io/ > > An starting example would be fine to kickstart myself (from the > ones I'm reading, there is still something I don't get). Let's say > I want to get all cards in [1] , as shown in the screenshot below. > I know the div class of each one, and the div class where they are > located. Which could be a minimal example of a scraper, to start with? > > Thanks, > > Offray > > >

Feb. 14, 2018