July 2016 - Pharo-dev - lists.pharo.org

Download link for latest Pharo6 image on download page

by Marcus Denker

Hi, I have added a link to the download page for Pharo6: http://pharo.org/download This way it is easier to findâ¦ it is of course in active development, but quite usable most of the time. Marcus

July 14, 2016

Re: [Pharo-dev] tricks for XML parsing.

by stepharo

thanks for the analysis :) I just skimmed over the tabl of contents and this caugth my eyes. Le 14/7/16 Ã 01:58, monty a Ã©crit : > Thanks for the link. > > In-place parsing is a non-starter because it means storing the entire input as a string in memory, so you could only parse files that fit in Pharo's address space. The multi-gigabyte OpenStreetMap docs the article mentions would be unparsable with SAX in a 32-bit VM. > > Linked lists for storing child nodes is common (LibXML2 and Xerces do it) and provides constant time insertion and sibling access, but arrays/vectors (Arrays/OrderedCollections) are more cache friendly and faster for sequential access and in Pharo are almost always the correct choice. > > There is always the option of an FFI-based parser, but it shouldn't be a hybrid like Python's minidom (FFI Expat with a Python DOM implementation), because something like that already exists in Smalltalk/X (FFI Expat with a Smalltalk DOM) and it was slower than a St/X port of XMLParser in my tests (I assume due to the FFI overhead), so it's probably not worth it. But a non-hybrid parser with everything (including the DOM) done in C should definitely be faster. > >> Sent: Wednesday, July 13, 2016 at 10:27 AM >> From: stepharo <stepharo(a)free.fr> >> To: "Pharo Development List" <pharo-dev(a)lists.pharo.org> >> Subject: [Pharo-dev] tricks for XML parsing. >> >> Hi guys >> >> these free books may be interesting for you >> >> http://aosabook.org/ >> >> http://aosabook.org/en/posa/parsing-xml-at-the-speed-of-light.html >> >> >> stef >> >> >> >

July 14, 2016

Re: [Pharo-dev] tricks for XML parsing.

by Jan Vrany

On Thu, 2016-07-14 at 01:58 +0200, monty wrote: > Thanks for the link. > > In-place parsing is a non-starter because it means storing the entire > input as a string in memory, so you could only parse files that fit > in Pharo's address space. The multi-gigabyte OpenStreetMap docs the > article mentions would be unparsable with SAX in a 32-bit VM. I do not understand. I only know expat which does - AFAIK - in-place parsing and surelt does not need the whole input in memory.Â > There is always the option of an FFI-based parser, but it shouldn't > be a hybrid like Python's minidom (FFI Expat with a Python DOM > implementation), > because something like that already exists in Smalltalk/X (FFI Expat > with a Smalltalk DOM) I guess you refer to the implementation I did ages ago.Â > and it was slower than a St/X port of XMLParser in my tests (I assume > due to the FFI overhead), so it's probably not worth it. Very, very interesting. Where can I find the benchmarks?Â I just run a very simple benchmark on 112MB document (http://www.xml-be nchmark.org/downloads.html) and results are quite the opposite:Â Benchmark resut: Generated at :14-07-2016 07:32:25 AM Â Â Â Â Â Â Â Â Â Â Â BenchmarkÂ Â Â Â Â Â Execution Time [ms]Â Â Â Â Â Â # of M&S GCs [1]Â Â Â Â Â Â # of newspace GCs [1]Â Â Â Parameters BenchmarkXML Â Â Â Â Â Â Â Â Â Â Â Â SAX - VWÂ Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 93418Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 0Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 2060Â Â Â Â Â Â Â Â Â SAX - XMLSuiteÂ Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 9921Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 0Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 410 Â Â As you can see, the latter is roughly 10 times faster.Â I agree my implementation which uses Expat is clearly suboptimalÂ and need to be improved (for example it does not use a ILC-basedÂ send to driver so you have a lot of cache misses and does a lotÂ of unnecessary memcpy()s, but this can be easily improved) Jan Â Â > But a non-hybrid parser with everything (including the DOM) done in C > should definitely be faster. > > > Sent: Wednesday, July 13, 2016 at 10:27 AM > > From: stepharo <stepharo(a)free.fr> > > To: "Pharo Development List" <pharo-dev(a)lists.pharo.org> > > Subject: [Pharo-dev] tricks for XML parsing. > > > > Hi guys > > > > these free books may be interesting for you > > > > Â Â Â Â Â http://aosabook.org/ > > > > http://aosabook.org/en/posa/parsing-xml-at-the-speed-of-light.html > > > > > > stef > > > > > > >

July 14, 2016

Re: [Pharo-dev] tricks for XML parsing.

by monty

Thanks for the link. In-place parsing is a non-starter because it means storing the entire input as a string in memory, so you could only parse files that fit in Pharo's address space. The multi-gigabyte OpenStreetMap docs the article mentions would be unparsable with SAX in a 32-bit VM. Linked lists for storing child nodes is common (LibXML2 and Xerces do it) and provides constant time insertion and sibling access, but arrays/vectors (Arrays/OrderedCollections) are more cache friendly and faster for sequential access and in Pharo are almost always the correct choice. There is always the option of an FFI-based parser, but it shouldn't be a hybrid like Python's minidom (FFI Expat with a Python DOM implementation), because something like that already exists in Smalltalk/X (FFI Expat with a Smalltalk DOM) and it was slower than a St/X port of XMLParser in my tests (I assume due to the FFI overhead), so it's probably not worth it. But a non-hybrid parser with everything (including the DOM) done in C should definitely be faster. > Sent: Wednesday, July 13, 2016 at 10:27 AM > From: stepharo <stepharo(a)free.fr> > To: "Pharo Development List" <pharo-dev(a)lists.pharo.org> > Subject: [Pharo-dev] tricks for XML parsing. > > Hi guys > > these free books may be interesting for you > > http://aosabook.org/ > > http://aosabook.org/en/posa/parsing-xml-at-the-speed-of-light.html > > > stef > > >

July 13, 2016

Re: [Pharo-dev] tricks for XML parsing.

by Serge Stinckwich

There is also a chapter very similar to the one ObjVLisp here: http://aosabook.org/en/500L/a-simple-object-model.html On Wed, Jul 13, 2016 at 3:27 PM, stepharo <stepharo(a)free.fr> wrote: > Hi guys > > these free books may be interesting for you > > http://aosabook.org/ > > http://aosabook.org/en/posa/parsing-xml-at-the-speed-of-light.html > > > stef > > -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/

July 13, 2016

tricks for XML parsing.

by stepharo

Hi guys these free books may be interesting for you http://aosabook.org/ http://aosabook.org/en/posa/parsing-xml-at-the-speed-of-light.html stef

July 13, 2016

Re: [Pharo-dev] What is the correct way to create new integration configuration?

by Tudor Girba

Thanks a lot, Juraj! Doru > On Jul 13, 2016, at 3:45 PM, Juraj Kubelka <juraj.kubelka(a)icloud.com> wrote: > >> >> 13. 7. 2016 v 9:32, Marcus Denker <marcus.denker(a)inria.fr>: >> >> >>> On 13 Jul 2016, at 14:46, Juraj Kubelka <juraj.kubelka(a)icloud.com> wrote: >>> >>> Hi, >>> >>> I am working on a new configuration for GTools. >>> >>> What is the correct way to integrate cases with a slice? >>> For example this one: https://pharo.fogbugz.com/f/cases/18471/GTTools-relies-on-hasSpecialSelecto… >>> It has slice in Pharo60Inbox. Should I copy commits to GToolkit repository? >>> >> This one is not yet integrated, I submitted a slice to not have to wait for GT, but with my pre-holiday stress I did not menage to actually >> get it in yet. >> >> So instead you can ignore the slice and just get these changes in. > > OK. I am preparing new configuration here: https://pharo.fogbugz.com/f/cases/18773/ > I am almost done. > I will take care of GTool integrations. So next time there is something to integrate, just ping me. Anyway, i will regularly check FogBugz. > >> >>> And how it happens that GToolkit repository has commits done by TheIntegrator? >> >> The auto-rewriting deprecations happen to rewrite code. This then is saved in the Pharo repo and needs to be merged. >> (or if it is the latest it can just be copied). >> >> The same happens if changes have to be done fast or if they cross-cut many packages. >> >> it is a bit of a messâ¦ for the future we need a unified way to handle thingsâ¦ the current approach with the configurations and >> external repositories is not good.. far too slow and far too much manual work. > > Thatâs true. It is a lot of manual work and it is also hard to know if I integrate it well, with all changes, etc. > > Cheers, > Juraj > >> >> Marcus -- www.tudorgirba.com www.feenk.com "When people care, great things can happen."

July 13, 2016

Re: [Pharo-dev] What is the correct way to create new integration configuration?

by Juraj Kubelka

> 13. 7. 2016 v 9:32, Marcus Denker <marcus.denker(a)inria.fr>: > > >> On 13 Jul 2016, at 14:46, Juraj Kubelka <juraj.kubelka(a)icloud.com <mailto:juraj.kubelka@icloud.com>> wrote: >> >> Hi, >> >> I am working on a new configuration for GTools. >> >> What is the correct way to integrate cases with a slice? >> For example this one: https://pharo.fogbugz.com/f/cases/18471/GTTools-relies-on-hasSpecialSelecto… <https://pharo.fogbugz.com/f/cases/18471/GTTools-relies-on-hasSpecialSelecto…> >> It has slice in Pharo60Inbox. Should I copy commits to GToolkit repository? >> > This one is not yet integrated, I submitted a slice to not have to wait for GT, but with my pre-holiday stress I did not menage to actually > get it in yet. > > So instead you can ignore the slice and just get these changes in. OK. I am preparing new configuration here: https://pharo.fogbugz.com/f/cases/18773/ <https://pharo.fogbugz.com/f/cases/18773/> I am almost done. I will take care of GTool integrations. So next time there is something to integrate, just ping me. Anyway, i will regularly check FogBugz. > >> And how it happens that GToolkit repository has commits done by TheIntegrator? > > The auto-rewriting deprecations happen to rewrite code. This then is saved in the Pharo repo and needs to be merged. > (or if it is the latest it can just be copied). > > The same happens if changes have to be done fast or if they cross-cut many packages. > > it is a bit of a messâ¦ for the future we need a unified way to handle thingsâ¦ the current approach with the configurations and > external repositories is not good.. far too slow and far too much manual work. Thatâs true. It is a lot of manual work and it is also hard to know if I integrate it well, with all changes, etc. Cheers, Juraj > > Marcus >

July 13, 2016

Re: [Pharo-dev] What is the correct way to create new integration configuration?

by Marcus Denker

> On 13 Jul 2016, at 14:46, Juraj Kubelka <juraj.kubelka(a)icloud.com> wrote: > > Hi, > > I am working on a new configuration for GTools. > > What is the correct way to integrate cases with a slice? > For example this one: https://pharo.fogbugz.com/f/cases/18471/GTTools-relies-on-hasSpecialSelecto… <https://pharo.fogbugz.com/f/cases/18471/GTTools-relies-on-hasSpecialSelecto…> > It has slice in Pharo60Inbox. Should I copy commits to GToolkit repository? > This one is not yet integrated, I submitted a slice to not have to wait for GT, but with my pre-holiday stress I did not menage to actually get it in yet. So instead you can ignore the slice and just get these changes in. > And how it happens that GToolkit repository has commits done by TheIntegrator? The auto-rewriting deprecations happen to rewrite code. This then is saved in the Pharo repo and needs to be merged. (or if it is the latest it can just be copied). The same happens if changes have to be done fast or if they cross-cut many packages. it is a bit of a messâ¦ for the future we need a unified way to handle thingsâ¦ the current approach with the configurations and external repositories is not good.. far too slow and far too much manual work. Marcus

July 13, 2016

What is the correct way to create new integration configuration?

by Juraj Kubelka

Hi, I am working on a new configuration for GTools. What is the correct way to integrate cases with a slice? For example this one: https://pharo.fogbugz.com/f/cases/18471/GTTools-relies-on-hasSpecialSelecto… <https://pharo.fogbugz.com/f/cases/18471/GTTools-relies-on-hasSpecialSelecto…> It has slice in Pharo60Inbox. Should I copy commits to GToolkit repository? And how it happens that GToolkit repository has commits done by TheIntegrator? Thanks, Juraj

July 13, 2016