Just a quick try to scrap a query from google search  (using Chrome plugin + XMLHTMLParser + Path) :


Really cool��� and in few lines you get your own �� google �� powered search engine ;-).  

Nevertheless this is a bit slow (5 seconds for one page, around 20s for the 5 from the script (I think this might be doable to speed things a bit).

HTH,


C��drick


PS: code below in text

searches := OrderedCollection new.
query := 'black friday' urlEncoded.

"chrome plugin"
browser := GoogleChrome new.
browser headless: true.
browser open.
page := browser firstTab.
0 to: 50 by: 10 do: [:paging | 
page get: 'https://www.google.com/search?q=',query,'&start=', paging asString. 
result := page html.
dom := result parseHTML. "uses XMLHTMLParser"
searches addAll: (dom xpath: '//div[@class="g"]/div/div/div/a/@href'). 
].   

searches size
 "64".
searches first.
 "href=""https://www.fnac.com/black-friday"""

"xpath cheat sheet: https://gist.github.com/LeCoupa/8c305ec8c713aad07b14"