- Pharo-users - lists.pharo.org

Re: [Pharo-users] Coding XPath as Smalltalk

by stepharo

Le 3/9/16 Ã 11:30, PBKResearch a Ã©crit : > Hi Monty > > Many thanks. I have picked up a project that I had not worked on for a while, which explains why I am using an old image. I shall try the latest Moose image, as you suggest. My only anxiety is that I need to be able to use a rather ancient package called TextLint, and I do not know whether it will load OK in a new Pharo. If not, I shall try to update my existing image. Let me know because I wanted to resurrect TextLint :) > > With the latest XPath, will it be clear how to use the binary syntax to carry out node tests like the example of '//div[@id=''catlinks'']//' that I cited below? The case I am interested in is where the actual identifier ('catlinks' in this case) is a variable rather than a constant. It would be possible to do it in standard XPath by assembling the XPath string with a variable component, but it might be more convenient in the binary syntax. > > Many thanks for your help. > > Peter Kenny > > -----Original Message----- > From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of monty > Sent: 03 September 2016 06:54 > To: pharo-users(a)lists.pharo.org > Subject: Re: [Pharo-users] Coding XPath as Smalltalk > > Peter, you're using an ancient version with bugs that were fixed last fall. The newest version has more tests and correct behavior (checked against a reference implementation). Just download a new Moose image and you'll get it, along with an up to date XMLParser. (But if you insist on upgrading in your old image, run "XPath initialize" after) > > The binary syntax (there are keyword equivalents now) officially only supports XPath axis selectors like #/ and #// that take node test arguments where the node tests can be name tests like 'name,' '*', 'prefix:*' or type tests like 'text()', 'comment()', 'element(name)'. > > Filters aren't officially supported with that syntax, but you can always use select: on the result. ?? was removed, but I might add it back as shorthand. Filters are implemented differently now. > >> From: PBKResearch <peter(a)pbkresearch.co.uk> >> To: pharo-users(a)lists.pharo.org >> Subject: [Pharo-users] Coding XPath as Smalltalk >> >> Hello >> >> I am using XPath as a way of dissecting web pages, especially from Wiktionary. Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: >> >> document xPath: '//div[@id=''catlinks'']//li//text()' >> >> and found that an equivalent is: >> >> document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. >> (I had to put two dummy arguments in the three-argument block to get it to work.) >> >> Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? >> >> Many thanks for any help. >> >> Peter Kenny > >

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by monty

> Sent: Saturday, September 03, 2016 at 2:56 AM > From: stepharo <stepharo(a)free.fr> > To: pharo-users(a)lists.pharo.org > Subject: Re: [Pharo-users] Coding XPath as Smalltalk > > > > Le 3/9/16 Ã 08:41, monty a Ã©crit : > >> Sent: Saturday, September 03, 2016 at 2:02 AM > >> From: stepharo <stepharo(a)free.fr> > >> To: "Any question about pharo is welcome" <pharo-users(a)lists.pharo.org> > >> Subject: Re: [Pharo-users] Coding XPath as Smalltalk > >> > >> Hi monty > >> > >> In which repository this maintained version is? > > PharoExtras/XPath (you gave me the write access). > > Excellent. > I like the idea that we all share an improve common identifiable places. Yes, I also moved XMLParserHTML and XMLParserStAX from PharoExtras/XMLParser to separate PharoExtras repos. PharoExtras could be the place for community-maintained standard libs. > > > > > >> PharoExtras? > >> > >> Is it the entry in the catalog? > > It has a catalog entry at http://catalog.pharo.org and a CI job at https://ci.inria.fr/pharo-contribution/job/XPath/ > > super cool! > I'm happy > > > > >> Stef > >> > >> > >> > >> Le 3/9/16 Ã 07:54, monty a Ã©crit : > >>> Peter, you're using an ancient version with bugs that were fixed last fall. The newest version has more tests and correct behavior (checked against a reference implementation). Just download a new Moose image and you'll get it, along with an up to date XMLParser. (But if you insist on upgrading in your old image, run "XPath initialize" after) > >>> > >>> The binary syntax (there are keyword equivalents now) officially only supports XPath axis selectors like #/ and #// that take node test arguments where the node tests can be name tests like 'name,' '*', 'prefix:*' or type tests like 'text()', 'comment()', 'element(name)'. > >>> > >>> Filters aren't officially supported with that syntax, but you can always use select: on the result. ?? was removed, but I might add it back as shorthand. Filters are implemented differently now. > >>> > >>>> From: PBKResearch <peter(a)pbkresearch.co.uk> > >>>> To: pharo-users(a)lists.pharo.org > >>>> Subject: [Pharo-users] Coding XPath as Smalltalk > >>>> > >>>> Hello > >>>> > >>>> I am using XPath as a way of dissecting web pages, especially from Wiktionary. Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: > >>>> > >>>> document xPath: '//div[@id=''catlinks'']//li//text()' > >>>> > >>>> and found that an equivalent is: > >>>> > >>>> document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. > >>>> (I had to put two dummy arguments in the three-argument block to get it to work.) > >>>> > >>>> Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? > >>>> > >>>> Many thanks for any help. > >>>> > >>>> Peter Kenny > >> > >> > > > > >

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by monty

Thanks! > Sent: Saturday, September 03, 2016 at 4:31 AM > From: "Sven Van Caekenberghe" <sven(a)stfx.eu> > To: "Any question about pharo is welcome" <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] Coding XPath as Smalltalk > > > > On 03 Sep 2016, at 08:17, Tudor Girba <tudor(a)tudorgirba.com> wrote: > > > > Hi, > > > > Indeed, Monty is doing a great job at maintaining and evolving the XML support. > > Yes indeed ! > > > Cheers, > > Doru > > > > > >> On Sep 3, 2016, at 8:06 AM, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: > >> > >> Thank you Monty for the clarification. I should say the original XPath package was written by Phil Hargett and I just added a couple of methods. Glad you rewrote the lib! > >> Cheers, > >> > >> HernÃ¡n > >> > >> > >> 2016-09-03 3:01 GMT-03:00 monty <monty2(a)programmer.net>: > >> > >> Hernan, the PharoExtras/XPath repo has a major rewrite of your package to support all of XPath 1.0 + XPath 2.0 extensions like the element() and attribute() type tests and namespace literals in name tests like '{namespaceURI}localName'. A rewrite was needed because the old lib only implemented a small subset of the spec and would infinite loop on some inputs. > >> > >> Sent: Thursday, September 01, 2016 at 3:56 PM > >> From: "HernÃ¡n Morales Durand" <hernan.morales(a)gmail.com> > >> > >> To: "Any question about pharo is welcome" <pharo-users(a)lists.pharo.org> > >> Subject: Re: [Pharo-users] Coding XPath as Smalltalk > >> > >> > >> 2016-09-01 16:51 GMT-03:00 PBKResearch <peter(a)pbkresearch.co.uk>: > >> Hi Hernan > >> > >> > >> I donât understand your first question â I canât see a connection between SPARQL and what I am doing. > >> > >> > >> > >> You could get the Wikitionary data by querying a SPARQL endpoint http://wiktionary.dbpedia.org/sparql instead of scrapping web pages (which seems more difficult) > >> > >> > >> I downloaded XPath from http://smalltalkhub.com/mc/PharoExtras/XPath/. However, I am probably using a somewhat out of date version; I downloaded it about a year ago. > >> > >> > >> > >> I don't know about that version. I copied an old version from SqueakSource (with permission) and updated from time to time, but there is no much. There is also a XPath2 repository which you may try. > >> > >> HernÃ¡n > >> > >> > >> Peter > >> > >> > >> From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of HernÃ¡n Morales Durand > >> Sent: 01 September 2016 18:54 > >> To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > >> Subject: Re: [Pharo-users] Coding XPath as Smalltalk > >> > >> > >> Hi Peter, > >> > >> > >> 2016-09-01 10:26 GMT-03:00 PBKResearch <peter(a)pbkresearch.co.uk>: > >> > >> Hello > >> > >> > >> I am using XPath as a way of dissecting web pages, especially from Wiktionary. > >> > >> > >> Any specific reason to not use the SPARQL endpoint? > >> > >> > >> > >> > >> Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. > >> > >> > >> Which XPath version are you using? How did you installed it? > >> > >> > >> > >> > >> I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: > >> > >> > >> document xPath: '//div[@id=''catlinks'']//li//text()' > >> > >> > >> and found that an equivalent is: > >> > >> > >> document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. > >> > >> (I had to put two dummy arguments in the three-argument block to get it to work.) > >> > >> > >> Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? > >> > >> > >> Many thanks for any help. > >> > >> > >> Peter Kenny > >> > >> > >> > >> > > > > -- > > www.tudorgirba.com > > www.feenk.com > > > > âLive like you mean it." > > > > > > >

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by monty

> Sent: Saturday, September 03, 2016 at 5:30 AM > From: PBKResearch <peter(a)pbkresearch.co.uk> > To: "'Any question about pharo is welcome'" <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] Coding XPath as Smalltalk > > Hi Monty > > Many thanks. I have picked up a project that I had not worked on for a while, which explains why I am using an old image. I shall try the latest Moose image, as you suggest. My only anxiety is that I need to be able to use a rather ancient package called TextLint, and I do not know whether it will load OK in a new Pharo. If not, I shall try to update my existing image. If you'd looked at CI job, you'd see that XPath builds on Pharo 5 through 3 (but should work back to 1.4). You can always start fresh with a clean, old image from http://files.pharo.org/image/ or the Moose website if TextLint doesn't work anymore. > With the latest XPath, will it be clear how to use the binary syntax to carry out node tests like the example of '//div[@id=''catlinks'']//' that I cited below? The case I am interested in is where the actual identifier ('catlinks' in this case) is a variable rather than a constant. It would be possible to do it in standard XPath by assembling the XPath string with a variable component, but it might be more convenient in the binary syntax. > You could do this: ((doc // 'div') select: [:each | (each attributeAt: 'id') = catlinks]) // 'li' // 'text()' where "catlinks" is a var. Or you could use xPath:context: with an XPath var that you dynamically bind using custom contexts: doc xPath: '//div[@id=$catlinks]//li//text()' context: (XPathContext variables: {'catlinks' -> catlinks}) The advantage over this: doc xPath: '//div[@id=''', catlinks, ''']//li//text()' is that the xPath: expression string is the same each time, so it's only compiled once, the first time, and cached for later uses (inspect 'XPath compiledXPathCache') instead of being compiled each time the xPath: expression string arg changes. > Many thanks for your help. > > Peter Kenny > > -----Original Message----- > From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of monty > Sent: 03 September 2016 06:54 > To: pharo-users(a)lists.pharo.org > Subject: Re: [Pharo-users] Coding XPath as Smalltalk > > Peter, you're using an ancient version with bugs that were fixed last fall. The newest version has more tests and correct behavior (checked against a reference implementation). Just download a new Moose image and you'll get it, along with an up to date XMLParser. (But if you insist on upgrading in your old image, run "XPath initialize" after) > > The binary syntax (there are keyword equivalents now) officially only supports XPath axis selectors like #/ and #// that take node test arguments where the node tests can be name tests like 'name,' '*', 'prefix:*' or type tests like 'text()', 'comment()', 'element(name)'. > > Filters aren't officially supported with that syntax, but you can always use select: on the result. ?? was removed, but I might add it back as shorthand. Filters are implemented differently now. > > > From: PBKResearch <peter(a)pbkresearch.co.uk> > > To: pharo-users(a)lists.pharo.org > > Subject: [Pharo-users] Coding XPath as Smalltalk > > > > Hello > > > > I am using XPath as a way of dissecting web pages, especially from Wiktionary. Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: > > > > document xPath: '//div[@id=''catlinks'']//li//text()' > > > > and found that an equivalent is: > > > > document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. > > (I had to put two dummy arguments in the three-argument block to get it to work.) > > > > Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? > > > > Many thanks for any help. > > > > Peter Kenny > > >

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by PBKResearch

Hi Monty Many thanks. I have picked up a project that I had not worked on for a while, which explains why I am using an old image. I shall try the latest Moose image, as you suggest. My only anxiety is that I need to be able to use a rather ancient package called TextLint, and I do not know whether it will load OK in a new Pharo. If not, I shall try to update my existing image. With the latest XPath, will it be clear how to use the binary syntax to carry out node tests like the example of '//div[@id=''catlinks'']//' that I cited below? The case I am interested in is where the actual identifier ('catlinks' in this case) is a variable rather than a constant. It would be possible to do it in standard XPath by assembling the XPath string with a variable component, but it might be more convenient in the binary syntax. Many thanks for your help. Peter Kenny -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of monty Sent: 03 September 2016 06:54 To: pharo-users(a)lists.pharo.org Subject: Re: [Pharo-users] Coding XPath as Smalltalk Peter, you're using an ancient version with bugs that were fixed last fall. The newest version has more tests and correct behavior (checked against a reference implementation). Just download a new Moose image and you'll get it, along with an up to date XMLParser. (But if you insist on upgrading in your old image, run "XPath initialize" after) The binary syntax (there are keyword equivalents now) officially only supports XPath axis selectors like #/ and #// that take node test arguments where the node tests can be name tests like 'name,' '*', 'prefix:*' or type tests like 'text()', 'comment()', 'element(name)'. Filters aren't officially supported with that syntax, but you can always use select: on the result. ?? was removed, but I might add it back as shorthand. Filters are implemented differently now. > From: PBKResearch <peter(a)pbkresearch.co.uk> > To: pharo-users(a)lists.pharo.org > Subject: [Pharo-users] Coding XPath as Smalltalk > > Hello > > I am using XPath as a way of dissecting web pages, especially from Wiktionary. Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: > > document xPath: '//div[@id=''catlinks'']//li//text()' > > and found that an equivalent is: > > document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. > (I had to put two dummy arguments in the three-argument block to get it to work.) > > Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? > > Many thanks for any help. > > Peter Kenny

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by Sven Van Caekenberghe

> On 03 Sep 2016, at 08:17, Tudor Girba <tudor(a)tudorgirba.com> wrote: > > Hi, > > Indeed, Monty is doing a great job at maintaining and evolving the XML support. Yes indeed ! > Cheers, > Doru > > >> On Sep 3, 2016, at 8:06 AM, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >> >> Thank you Monty for the clarification. I should say the original XPath package was written by Phil Hargett and I just added a couple of methods. Glad you rewrote the lib! >> Cheers, >> >> HernÃ¡n >> >> >> 2016-09-03 3:01 GMT-03:00 monty <monty2(a)programmer.net>: >> >> Hernan, the PharoExtras/XPath repo has a major rewrite of your package to support all of XPath 1.0 + XPath 2.0 extensions like the element() and attribute() type tests and namespace literals in name tests like '{namespaceURI}localName'. A rewrite was needed because the old lib only implemented a small subset of the spec and would infinite loop on some inputs. >> >> Sent: Thursday, September 01, 2016 at 3:56 PM >> From: "HernÃ¡n Morales Durand" <hernan.morales(a)gmail.com> >> >> To: "Any question about pharo is welcome" <pharo-users(a)lists.pharo.org> >> Subject: Re: [Pharo-users] Coding XPath as Smalltalk >> >> >> 2016-09-01 16:51 GMT-03:00 PBKResearch <peter(a)pbkresearch.co.uk>: >> Hi Hernan >> >> >> I donât understand your first question â I canât see a connection between SPARQL and what I am doing. >> >> >> >> You could get the Wikitionary data by querying a SPARQL endpoint http://wiktionary.dbpedia.org/sparql instead of scrapping web pages (which seems more difficult) >> >> >> I downloaded XPath from http://smalltalkhub.com/mc/PharoExtras/XPath/. However, I am probably using a somewhat out of date version; I downloaded it about a year ago. >> >> >> >> I don't know about that version. I copied an old version from SqueakSource (with permission) and updated from time to time, but there is no much. There is also a XPath2 repository which you may try. >> >> HernÃ¡n >> >> >> Peter >> >> >> From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of HernÃ¡n Morales Durand >> Sent: 01 September 2016 18:54 >> To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> >> Subject: Re: [Pharo-users] Coding XPath as Smalltalk >> >> >> Hi Peter, >> >> >> 2016-09-01 10:26 GMT-03:00 PBKResearch <peter(a)pbkresearch.co.uk>: >> >> Hello >> >> >> I am using XPath as a way of dissecting web pages, especially from Wiktionary. >> >> >> Any specific reason to not use the SPARQL endpoint? >> >> >> >> >> Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. >> >> >> Which XPath version are you using? How did you installed it? >> >> >> >> >> I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: >> >> >> document xPath: '//div[@id=''catlinks'']//li//text()' >> >> >> and found that an equivalent is: >> >> >> document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. >> >> (I had to put two dummy arguments in the three-argument block to get it to work.) >> >> >> Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? >> >> >> Many thanks for any help. >> >> >> Peter Kenny >> >> >> >> > > -- > www.tudorgirba.com > www.feenk.com > > âLive like you mean it." > >

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by stepharo

Le 3/9/16 Ã 08:41, monty a Ã©crit : >> Sent: Saturday, September 03, 2016 at 2:02 AM >> From: stepharo <stepharo(a)free.fr> >> To: "Any question about pharo is welcome" <pharo-users(a)lists.pharo.org> >> Subject: Re: [Pharo-users] Coding XPath as Smalltalk >> >> Hi monty >> >> In which repository this maintained version is? > PharoExtras/XPath (you gave me the write access). Excellent. I like the idea that we all share an improve common identifiable places. > > >> PharoExtras? >> >> Is it the entry in the catalog? > It has a catalog entry at http://catalog.pharo.org and a CI job at https://ci.inria.fr/pharo-contribution/job/XPath/ super cool! I'm happy > >> Stef >> >> >> >> Le 3/9/16 Ã 07:54, monty a Ã©crit : >>> Peter, you're using an ancient version with bugs that were fixed last fall. The newest version has more tests and correct behavior (checked against a reference implementation). Just download a new Moose image and you'll get it, along with an up to date XMLParser. (But if you insist on upgrading in your old image, run "XPath initialize" after) >>> >>> The binary syntax (there are keyword equivalents now) officially only supports XPath axis selectors like #/ and #// that take node test arguments where the node tests can be name tests like 'name,' '*', 'prefix:*' or type tests like 'text()', 'comment()', 'element(name)'. >>> >>> Filters aren't officially supported with that syntax, but you can always use select: on the result. ?? was removed, but I might add it back as shorthand. Filters are implemented differently now. >>> >>>> From: PBKResearch <peter(a)pbkresearch.co.uk> >>>> To: pharo-users(a)lists.pharo.org >>>> Subject: [Pharo-users] Coding XPath as Smalltalk >>>> >>>> Hello >>>> >>>> I am using XPath as a way of dissecting web pages, especially from Wiktionary. Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: >>>> >>>> document xPath: '//div[@id=''catlinks'']//li//text()' >>>> >>>> and found that an equivalent is: >>>> >>>> document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. >>>> (I had to put two dummy arguments in the three-argument block to get it to work.) >>>> >>>> Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? >>>> >>>> Many thanks for any help. >>>> >>>> Peter Kenny >> >> >

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by stepharo

+ 1000000 Le 3/9/16 Ã 08:17, Tudor Girba a Ã©crit : > Hi, > > Indeed, Monty is doing a great job at maintaining and evolving the XML support. > > Cheers, > Doru > > >> On Sep 3, 2016, at 8:06 AM, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >> >> Thank you Monty for the clarification. I should say the original XPath package was written by Phil Hargett and I just added a couple of methods. Glad you rewrote the lib! >> Cheers, >> >> HernÃ¡n >> >> >> 2016-09-03 3:01 GMT-03:00 monty <monty2(a)programmer.net>: >> >> Hernan, the PharoExtras/XPath repo has a major rewrite of your package to support all of XPath 1.0 + XPath 2.0 extensions like the element() and attribute() type tests and namespace literals in name tests like '{namespaceURI}localName'. A rewrite was needed because the old lib only implemented a small subset of the spec and would infinite loop on some inputs. >> >> Sent: Thursday, September 01, 2016 at 3:56 PM >> From: "HernÃ¡n Morales Durand" <hernan.morales(a)gmail.com> >> >> To: "Any question about pharo is welcome" <pharo-users(a)lists.pharo.org> >> Subject: Re: [Pharo-users] Coding XPath as Smalltalk >> >> >> 2016-09-01 16:51 GMT-03:00 PBKResearch <peter(a)pbkresearch.co.uk>: >> Hi Hernan >> >> >> I donât understand your first question â I canât see a connection between SPARQL and what I am doing. >> >> >> >> You could get the Wikitionary data by querying a SPARQL endpoint http://wiktionary.dbpedia.org/sparql instead of scrapping web pages (which seems more difficult) >> >> >> I downloaded XPath from http://smalltalkhub.com/mc/PharoExtras/XPath/. However, I am probably using a somewhat out of date version; I downloaded it about a year ago. >> >> >> >> I don't know about that version. I copied an old version from SqueakSource (with permission) and updated from time to time, but there is no much. There is also a XPath2 repository which you may try. >> >> HernÃ¡n >> >> >> Peter >> >> >> From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of HernÃ¡n Morales Durand >> Sent: 01 September 2016 18:54 >> To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> >> Subject: Re: [Pharo-users] Coding XPath as Smalltalk >> >> >> Hi Peter, >> >> >> 2016-09-01 10:26 GMT-03:00 PBKResearch <peter(a)pbkresearch.co.uk>: >> >> Hello >> >> >> I am using XPath as a way of dissecting web pages, especially from Wiktionary. >> >> >> Any specific reason to not use the SPARQL endpoint? >> >> >> >> >> Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. >> >> >> Which XPath version are you using? How did you installed it? >> >> >> >> >> I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: >> >> >> document xPath: '//div[@id=''catlinks'']//li//text()' >> >> >> and found that an equivalent is: >> >> >> document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. >> >> (I had to put two dummy arguments in the three-argument block to get it to work.) >> >> >> Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? >> >> >> Many thanks for any help. >> >> >> Peter Kenny >> >> >> >> > -- > www.tudorgirba.com > www.feenk.com > > âLive like you mean it." > > >

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by monty

> Sent: Saturday, September 03, 2016 at 2:02 AM > From: stepharo <stepharo(a)free.fr> > To: "Any question about pharo is welcome" <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] Coding XPath as Smalltalk > > Hi monty > > In which repository this maintained version is? PharoExtras/XPath (you gave me the write access). > PharoExtras? > > Is it the entry in the catalog? It has a catalog entry at http://catalog.pharo.org and a CI job at https://ci.inria.fr/pharo-contribution/job/XPath/ > > Stef > > > > Le 3/9/16 Ã 07:54, monty a Ã©crit : > > Peter, you're using an ancient version with bugs that were fixed last fall. The newest version has more tests and correct behavior (checked against a reference implementation). Just download a new Moose image and you'll get it, along with an up to date XMLParser. (But if you insist on upgrading in your old image, run "XPath initialize" after) > > > > The binary syntax (there are keyword equivalents now) officially only supports XPath axis selectors like #/ and #// that take node test arguments where the node tests can be name tests like 'name,' '*', 'prefix:*' or type tests like 'text()', 'comment()', 'element(name)'. > > > > Filters aren't officially supported with that syntax, but you can always use select: on the result. ?? was removed, but I might add it back as shorthand. Filters are implemented differently now. > > > >> From: PBKResearch <peter(a)pbkresearch.co.uk> > >> To: pharo-users(a)lists.pharo.org > >> Subject: [Pharo-users] Coding XPath as Smalltalk > >> > >> Hello > >> > >> I am using XPath as a way of dissecting web pages, especially from Wiktionary. Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: > >> > >> document xPath: '//div[@id=''catlinks'']//li//text()' > >> > >> and found that an equivalent is: > >> > >> document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. > >> (I had to put two dummy arguments in the three-argument block to get it to work.) > >> > >> Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? > >> > >> Many thanks for any help. > >> > >> Peter Kenny > > > > >

Sept. 3, 2016

Re: [Pharo-users] Coding XPath as Smalltalk

by Tudor Girba

Hi, Indeed, Monty is doing a great job at maintaining and evolving the XML support. Cheers, Doru > On Sep 3, 2016, at 8:06 AM, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: > > Thank you Monty for the clarification. I should say the original XPath package was written by Phil Hargett and I just added a couple of methods. Glad you rewrote the lib! > Cheers, > > HernÃ¡n > > > 2016-09-03 3:01 GMT-03:00 monty <monty2(a)programmer.net>: > > Hernan, the PharoExtras/XPath repo has a major rewrite of your package to support all of XPath 1.0 + XPath 2.0 extensions like the element() and attribute() type tests and namespace literals in name tests like '{namespaceURI}localName'. A rewrite was needed because the old lib only implemented a small subset of the spec and would infinite loop on some inputs. > > Sent: Thursday, September 01, 2016 at 3:56 PM > From: "HernÃ¡n Morales Durand" <hernan.morales(a)gmail.com> > > To: "Any question about pharo is welcome" <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] Coding XPath as Smalltalk > > > 2016-09-01 16:51 GMT-03:00 PBKResearch <peter(a)pbkresearch.co.uk>: > Hi Hernan > > > I donât understand your first question â I canât see a connection between SPARQL and what I am doing. > > > > You could get the Wikitionary data by querying a SPARQL endpoint http://wiktionary.dbpedia.org/sparql instead of scrapping web pages (which seems more difficult) > > > I downloaded XPath from http://smalltalkhub.com/mc/PharoExtras/XPath/. However, I am probably using a somewhat out of date version; I downloaded it about a year ago. > > > > I don't know about that version. I copied an old version from SqueakSource (with permission) and updated from time to time, but there is no much. There is also a XPath2 repository which you may try. > > HernÃ¡n > > > Peter > > > From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of HernÃ¡n Morales Durand > Sent: 01 September 2016 18:54 > To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > Subject: Re: [Pharo-users] Coding XPath as Smalltalk > > > Hi Peter, > > > 2016-09-01 10:26 GMT-03:00 PBKResearch <peter(a)pbkresearch.co.uk>: > > Hello > > > I am using XPath as a way of dissecting web pages, especially from Wiktionary. > > > Any specific reason to not use the SPARQL endpoint? > > > > > Generally I get good results, but I could get useful extra flexibility by using the binary Smalltalk operators to represent XPath, as mentioned at the end of the class comment for XPath. However, the description there is very terse, and I am having difficulty seeing how to include more complex expressions, especially attribute tests. > > > Which XPath version are you using? How did you installed it? > > > > > I have put some of my XPath expressions through the XPath compiler and looked at the output, and out of that I have found expressions which work but look very clumsy. As an example, I have used the fragment: > > > document xPath: '//div[@id=''catlinks'']//li//text()' > > > and found that an equivalent is: > > > document //'div' ?? [:node :x :y|(node attributeAt: 'id') = 'catlinks']//'li'//[:n| n isStringNode]]. > > (I had to put two dummy arguments in the three-argument block to get it to work.) > > > Is there a more extensive explanation of the use of these binary operators? If not, could some kind person show me the most concise translation of the sample XPath above, to give me a start in working out more complex cases? > > > Many thanks for any help. > > > Peter Kenny > > > > -- www.tudorgirba.com www.feenk.com âLive like you mean it."

Sept. 3, 2016