October 2017 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Problem with input to XML Parser - 'Invalid UTF8 encoding'

by Henrik Sperre Johansen

XML expects a prolog in the document itself defining the encoding, if absent, the standard specifies utf-8. So when you use an XML parser to parse an HTML page, it will disregard any HTTP encodings, interpret the contents as an XML document with missing prolog, and try to parse as utf8. When you use ZnUrl getContents however, it respects the HTTP charset header field, which correctly identifies the contents as 8859-1, and lets you correctly read it into an internal string. Subsequently parsing said internal string, the XML parser won't try to do any conversion, and therefore works. Cheers, Henry -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Oct. 9, 2017

Re: [Pharo-users] Problem with input to XML Parser - 'Invalid UTF8 encoding'

by Peter Kenny

Note: This was sent on Sunday at 19.45 but seems to have disappeared on its way to pharo users. Re-sent just to complete the story. _____________________________________________________ Paul Good to have found the charset discrepancy - that may have something to do with it. But I don't think it has to do with the CâÃ¨ in the body of the page. I have just parsed another page published today, with the same error, and again it fails in parsing the <head> node, so it has not even reached the body. The <head> contains a meta which describes the article - a sort of paraphrase of the article headline - and it fails in the middle of decoding that. The character at which it fails is again $Â«, so that is definitely the cause. Maybe the wrong charset is the explanation of why it messes up that - but I don't know enough about the different charsets to know. Does ISO-8859-1 even contain $Â«? Peter Addendum: I have looked a bit further, and the charset problem lies behind it all. The ISO-8859-1 charset *does* include $Â«, at decimal 171 or hex AB. At the point where it fails, the parser is reading the string 'Â«B', which is hex AB 42 in ISO-8859-1, and the debugger shows that the parser is trying to decode hex AB 42 as a multibyte UTF8 character. So there are two questions remaining: (a) why does the parser try to decode it as UTF8? (b) why does reading the string in before calling the parser get round the problem? -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Oct. 9, 2017

Re: [Pharo-users] [Ann] PharoLauncher v1.0.1 released!

by Esteban Lorenzano

> On 9 Oct 2017, at 10:10, Christophe Demarey <Christophe.Demarey(a)inria.fr> wrote: > > Hi Peter, > >> Le 6 oct. 2017 Ã 21:13, Peter UhnÃ¡k <i.uhnak(a)gmail.com <mailto:i.uhnak@gmail.com>> a Ã©crit : >> >> +1.0.1 :-) >> >> > Pharo Launcher files (images, VMs) are now considered as user Documents (easier to find). Consequently, they are now stored in the standard documents folder, i.e. >> > $HOME/Pharo >> >> This seems like a step backwards. The expected location is $XDG_DATA_HOME/Pharo . Is it possible to change or configure this? why a step backwards? Pharo images are not *data* in the sense is intended for XDG_DATA_HOME. In practice, an image is more like a document Pharo can readâ¦ data files are configurations, etc. an app needs to work properly in the local environment. Pharo images instead, are modifiable/editable/whatever euphemism you wantâ¦ thatâs why we considered more like VMWare considers their Virtual Machines, then they belong to some place in the home directory that is of easy access. > > We decide to consider Pharo Launcher files (virtual machines, images) as user documents. Thatâs why we changed the location of these files on all platforms. > $XDG_DATA_HOME/Pharo is not a document location on linux and there is no standard Documents location. Thatâs why we ended to put Pharo Launcher files in a Pharo folder in the HOME directory. yeah, this. > Maybe it is not the best choice. I like that choice :) > > If we can find an agreement on the best Pharo Launcher documents location on Linux, it can be changed. > > You can already configure where you want to store Pharo Launcher images and vms (2 different settings). yeahâ¦ you can always customise your own working set. Esteban > > Christophe

Oct. 9, 2017

Re: [Pharo-users] [Ann] PharoLauncher v1.0.1 released!

by Christophe Demarey

Hi Peter, > Le 6 oct. 2017 Ã 21:13, Peter UhnÃ¡k <i.uhnak(a)gmail.com> a Ã©crit : > > +1.0.1 :-) > > > Pharo Launcher files (images, VMs) are now considered as user Documents (easier to find). Consequently, they are now stored in the standard documents folder, i.e. > > $HOME/Pharo > > This seems like a step backwards. The expected location is $XDG_DATA_HOME/Pharo . Is it possible to change or configure this? We decide to consider Pharo Launcher files (virtual machines, images) as user documents. Thatâs why we changed the location of these files on all platforms. $XDG_DATA_HOME/Pharo is not a document location on linux and there is no standard Documents location. Thatâs why we ended to put Pharo Launcher files in a Pharo folder in the HOME directory. Maybe it is not the best choice. If we can find an agreement on the best Pharo Launcher documents location on Linux, it can be changed. You can already configure where you want to store Pharo Launcher images and vms (2 different settings). Christophe

Oct. 9, 2017

Re: [Pharo-users] [Ann] PharoLauncher v1.0.1 released!

by Christophe Demarey

> Le 7 oct. 2017 Ã 14:26, Ben Coman <btc(a)openinworld.com> a Ã©crit : > > On Sat, Oct 7, 2017 at 3:38 PM, Nicolai Hess <nicolaihess(a)gmail.com <mailto:nicolaihess@gmail.com>> wrote: > Just starting pharolauncher on windows still does not work (if you put it into the default "program" folder). > On start, it tries to create a local direcitory "pharo-local" and does not have the permission to do so. > You need to start it with admin-rights. > > Why are we making it harder for Windows users in corporate environments by requiring them to have admin-rights they often won't have, while in Linux PharoLancher is installed into user folders? Â« Program Â» folder is the default location where to install apps on Windows. If a user does not have admin rights, he can simply install it under its home directory. > Also there is the UAC Virtualization fiasco that complicates installing upgrades. > http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2014-January/091… <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2014-January/091…> > > Could we *please* install Windows PharoLauncher into user folders? Users can choose to install Pharo Launcher where they wish. We could choose another default location but not sure it is best option. To me, the real problem is: why does Pharo apps (and Pharo Launcher) needs to write data in the application directory? By example, for Pharo Launcher, we do not need to load any code nor to backup sessions.

Oct. 9, 2017

Job offers

by Stephane Ducasse

Two jobs in Europe: In Le Mans France, CGI is looking for an OO developer, ideally Smalltalk. CGI is a French IT services company and works in lots of different domains. One clearly needs to be able to work in France, and, most likely in French as well. https://cgi.njoyn.com/cgi/xweb/xweb.asp?CLID=21001&page=jobdetails&JobID=J0… The second is in Kaiserslautern Germany with Bison. Bison is a Swiss and German IT services company similar in many ways to CGI above. http://jobview.monster.com/Software-Developer-m-w-Job-Kaiserslautern-Rheinl… Good Luck!

Oct. 8, 2017

Re: [Pharo-users] Problem with input to XML Parser - 'Invalid UTF8 encoding'

by PBKResearch

Paul Good to have found the charset discrepancy - that may have something to do with it. But I don't think it has to do with the CâÃ¨ in the body of the page. I have just parsed another page published today, with the same error, and again it fails in parsing the <head> node, so it has not even reached the body. The <head> contains a meta which describes the article - a sort of paraphrase of the article headline - and it fails in the middle of decoding that. The character at which it fails is again $Â«, so that is definitely the cause. Maybe the wrong charset is the explanation of why it messes up that - but I don't know enough about the different charsets to know. Does ISO-8859-1 even contain $Â«? Peter -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of Paul DeBruicker Sent: 08 October 2017 18:41 To: pharo-users(a)lists.pharo.org Subject: Re: [Pharo-users] Problem with input to XML Parser - 'Invalid UTF8 encoding' in the HEAD tag of that page with the article they declare it is ISO-8859-1 and not UTF-8. In the page they have a CâÃ¨ The little back-tick next to the C is UTF8 8217 (http://www.codetable.net/decimal/8217) So their encoding is messed up, and maybe the XMLHTMLParser should throw a warning or something if there is a mismatch. Glad you found a work around. In another thread (on SVG Icons) Sven referred to ways of getting input from a URL for XMLDOMParser. I have recently had some problems doing this. I have found a workaround, so it is not urgent, but I thought I should put it on record in case anyone else is bitten by it, and so maybe Monty can look at it. I am using the subclass XMLHTMLParser, and my usual way of invoking it was: 1. XMLHTMLParser parseURL: <urlstring>. This works in most cases, but with one particular site - http://www.corriere.it/....., which is an Italian newspaper - I had frequent failures, with the error message 'Invalid UTF8 encoding'. The parser has the option of parsing a string, which is obtained by other means, so I tried reading it in with Zinc: 2. XMLHTMLParser parse: <urlstring> asZnUrl retrieveContents. And this worked, so clearly the encoding on the site is OK. I realised that the XML-Parser package has its own methods, which reproduce a lot of the functionality of Zinc, so I tried the equivalent: 3. XMLHTMLParser parse: <urlstring> asXMLURI get. To my surprise, this worked equally well. I had expected problems, because presumably forms (1) and (3) use the same UTF8 decoding. For now, I am using the form (3) for all my work, and have not had any problems since. So the message to anyone who is using the form (1) and getting problems is to try (2) or (3) instead. I am using Moose 6.1 (Pharo 6.0 Latest update: #60486) on Windows 10. I think most articles on the Corriere web site will generate the error, but one which has always failed for me is: http://www.corriere.it/esteri/17_ottobre_03/felipe-spagna-catalogna-discorso -8f7ac0d6-a86d-11e7-a090-96160224e787.shtml I tried to trace through the error using the debugger, but it got too confusing. However, I did establish that the failure occurred early in decoding the HTML <head>, in the line beginning <meta name="description".. The only unusual thing at this point is the French-style open-quote: '<'. Whether this could explain the problem I don't know. Any suggestions gratefully received. Peter Kenny </quote> -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Oct. 8, 2017

Re: [Pharo-users] Problem with input to XML Parser - 'Invalid UTF8 encoding'

by Paul DeBruicker

in the HEAD tag of that page with the article they declare it is ISO-8859-1 and not UTF-8. In the page they have a CâÃ¨ The little back-tick next to the C is UTF8 8217 (http://www.codetable.net/decimal/8217) So their encoding is messed up, and maybe the XMLHTMLParser should throw a warning or something if there is a mismatch. Glad you found a work around. In another thread (on SVG Icons) Sven referred to ways of getting input from a URL for XMLDOMParser. I have recently had some problems doing this. I have found a workaround, so it is not urgent, but I thought I should put it on record in case anyone else is bitten by it, and so maybe Monty can look at it. I am using the subclass XMLHTMLParser, and my usual way of invoking it was: 1. XMLHTMLParser parseURL: <urlstring>. This works in most cases, but with one particular site - http://www.corriere.it/....., which is an Italian newspaper - I had frequent failures, with the error message 'Invalid UTF8 encoding'. The parser has the option of parsing a string, which is obtained by other means, so I tried reading it in with Zinc: 2. XMLHTMLParser parse: <urlstring> asZnUrl retrieveContents. And this worked, so clearly the encoding on the site is OK. I realised that the XML-Parser package has its own methods, which reproduce a lot of the functionality of Zinc, so I tried the equivalent: 3. XMLHTMLParser parse: <urlstring> asXMLURI get. To my surprise, this worked equally well. I had expected problems, because presumably forms (1) and (3) use the same UTF8 decoding. For now, I am using the form (3) for all my work, and have not had any problems since. So the message to anyone who is using the form (1) and getting problems is to try (2) or (3) instead. I am using Moose 6.1 (Pharo 6.0 Latest update: #60486) on Windows 10. I think most articles on the Corriere web site will generate the error, but one which has always failed for me is: http://www.corriere.it/esteri/17_ottobre_03/felipe-spagna-catalogna-discorso -8f7ac0d6-a86d-11e7-a090-96160224e787.shtml I tried to trace through the error using the debugger, but it got too confusing. However, I did establish that the failure occurred early in decoding the HTML <head>, in the line beginning <meta name="description".. The only unusual thing at this point is the French-style open-quote: '<'. Whether this could explain the problem I don't know. Any suggestions gratefully received. Peter Kenny </quote> -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Oct. 8, 2017

Re: [Pharo-users] FileReference#entries caching?

by Alistair Grant

On Sun, Oct 08, 2017 at 07:08:41PM +0200, Guillermo Polito wrote: > > > On Sun, Oct 8, 2017 at 7:01 PM, Alistair Grant <akgrant0710(a)gmail.com> wrote: > > On Sun, Oct 08, 2017 at 06:33:08PM +0200, Guillermo Polito wrote: > > Hi, > > > > I don't think that FileSystem is caching entries... > > I haven't seen any code in the Pharo image or FilePlugin that caches > data. > > > > However, we found somehow > > related bug the other day with Mariano and Pablo while running > OSSupprocess > > tests on travis/linux. The bug is as follows. The test was doing: > > > > '/tmp' asFileReference entries do: [...] > > > > And then it was calling 'lsof' over pharo's pid to know files that pharo > has > > open. > > > > funny thing is that lsof was telling us that '/tmp' was open! When we > already > > retrieved the entries and the file reference was collected... > > > > The same in osx or local linux mate running in a virtual box did not > happen :/. > > I was able to reproduce the issue on Ubuntu 16.04 using > Pharo7.0-32bit-d13eed2.image. > > I haven't tracked the cause down, but the problem doesn't exist with the > proposed fix I have for symbolic links (fogbugz 18279 > https://pharo.fogbugz.com/f/cases/18279/isSymlink-seems- > to-be-broken-on-Linux). > This patch also updates the code responsible for iterating over > directories, which is why it can resolve this issue as well. > > > > Nice! How much time until we can integrate such thing ? :D I submitted the code in August. Eliot kindly provided some great feedback with lots of details. I submitted updated code on 6 September. Since then I haven't seen anything from Eliot, so I hope everything is OK. Cheers, Alistair > Cheers, > Alistair > > > > On Sat, Oct 7, 2017 at 5:44 PM, Steven R. Baker <steven(a)stevenrbaker.com> > > wrote: > > > > > > > > > > On 07/10/17 14:22, Peter Uhn?k wrote: > > > > On Sat, Oct 7, 2017 at 1:02 PM, Steven R. Baker < > > steven(a)stevenrbaker.com> wrote: > > > > Is it cached or something? How do I guarantee that I'm > *definitely* > > getting the current state of the directory on disk? > > > > > > This looks extremely strange, I've never seen that inspecting > files/ > > directories would be somehow "out of sync" with the disk, because > it is > > always asking the disk. > > > > > > > > > > Should I not even be using FileReference for this? > > > > > > If you are loading png icons into Pharo, you can consider using > https:/ > > /github.com/peteruhnak/IconFactory/ instead (shameless self > promotion > > :) ) > > > > > > Will definitely switch to this; I wasn't aware of the pattern, but > > obviously I want to follow them where possible. :) > > > > > > Peter > > > > > > > > > > > > > > -- > > [presentation] Guille Polito > > [CNRS-filaire] > > Research Engineer > > > > Centre de Recherche en Informatique, Signal et > > Automatique de Lille > > > > CRIStAL - UMR 9189 > > > > French National Center for Scientific Research > - > > http://www.cnrs.fr > > > > > > Web: http://guillep.github.io > > > > Phone: +33 06 52 70 66 13 > > > > > > > > -- > [presentation] Guille Polito > [CNRS-filaire] > Research Engineer > > Centre de Recherche en Informatique, Signal et > Automatique de Lille > > CRIStAL - UMR 9189 > > French National Center for Scientific Research - > http://www.cnrs.fr > > > Web: http://guillep.github.io > > Phone: +33 06 52 70 66 13 >

Oct. 8, 2017

Re: [Pharo-users] FileReference#entries caching?

by Guillermo Polito

On Sun, Oct 8, 2017 at 7:01 PM, Alistair Grant <akgrant0710(a)gmail.com> wrote: > On Sun, Oct 08, 2017 at 06:33:08PM +0200, Guillermo Polito wrote: > > Hi, > > > > I don't think that FileSystem is caching entries... > > I haven't seen any code in the Pharo image or FilePlugin that caches > data. > > > > However, we found somehow > > related bug the other day with Mariano and Pablo while running > OSSupprocess > > tests on travis/linux. The bug is as follows. The test was doing: > > > > '/tmp' asFileReference entries do: [...] > > > > And then it was calling 'lsof' over pharo's pid to know files that pharo > has > > open. > > > > funny thing is that lsof was telling us that '/tmp' was open! When we > already > > retrieved the entries and the file reference was collected... > > > > The same in osx or local linux mate running in a virtual box did not > happen :/. > > I was able to reproduce the issue on Ubuntu 16.04 using > Pharo7.0-32bit-d13eed2.image. > > I haven't tracked the cause down, but the problem doesn't exist with the > proposed fix I have for symbolic links (fogbugz 18279 > https://pharo.fogbugz.com/f/cases/18279/isSymlink-seems- > to-be-broken-on-Linux). > This patch also updates the code responsible for iterating over > directories, which is why it can resolve this issue as well. > > Nice! How much time until we can integrate such thing ? :D > > Cheers, > Alistair > > > > On Sat, Oct 7, 2017 at 5:44 PM, Steven R. Baker <steven(a)stevenrbaker.com > > > > wrote: > > > > > > > > > > On 07/10/17 14:22, Peter Uhn?k wrote: > > > > On Sat, Oct 7, 2017 at 1:02 PM, Steven R. Baker < > > steven(a)stevenrbaker.com> wrote: > > > > Is it cached or something? How do I guarantee that I'm > *definitely* > > getting the current state of the directory on disk? > > > > > > This looks extremely strange, I've never seen that inspecting > files/ > > directories would be somehow "out of sync" with the disk, > because it is > > always asking the disk. > > > > > > > > > > Should I not even be using FileReference for this? > > > > > > If you are loading png icons into Pharo, you can consider using > https:/ > > /github.com/peteruhnak/IconFactory/ instead (shameless self > promotion > > :) ) > > > > > > Will definitely switch to this; I wasn't aware of the pattern, but > > obviously I want to follow them where possible. :) > > > > > > Peter > > > > > > > > > > > > > > -- > > [presentation] Guille Polito > > [CNRS-filaire] > > Research Engineer > > > > Centre de Recherche en Informatique, Signal et > > Automatique de Lille > > > > CRIStAL - UMR 9189 > > > > French National Center for Scientific Research > - > > http://www.cnrs.fr > > > > > > Web: http://guillep.github.io > > > > Phone: +33 06 52 70 66 13 > > > > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Oct. 8, 2017