On Wed, Mar 15, 2017 at 05:28:37PM +0000, Rein, Patrick wrote:
Thanks for looking into this :)
@Dave: Can you explain what you would have expected to happen here? I see the point that squeaksource could also encode the response as UTF-8. However, currently the page is correctly encoded and delivered in iso-8859-1. From the error message I read that Zinc is nevertheless trying to decode it as UTF-8 which fails when it encounters a character with a code point > 127.
I did not have anything specific in mind, I just wanted to mention it in case it was relevant to the problem. Based on the follow up discussion, I do not think that it is relevant. For what it's worth, the specific issue in the older squeaksource image is that when someone registers an account with a user name that requires a WideString rather than a ByteString, it ends up scrambling the squeaksource user interface. I am fairly sure that this is fixed in newer versions of Squeak and/or Seaside, so for now it is just an annoyance for the person who keeps squeaksource.com running. Whenever that person gets sufficiently annoyed, I'm sure that the image will get updated ;-) Dave
Bests Patrick ________________________________________ From: Pharo-dev <pharo-dev-bounces@lists.pharo.org> on behalf of David T. Lewis <lewis@mail.msen.com> Sent: Wednesday, March 15, 2017 17:42 To: Pharo Development List Subject: Re: [Pharo-dev] ZnInvalidUTF8 on response from squeaksource
squeaksource.com is still running on a quite old image, and I know that it has problems with multibyte characters. If you are seeing problems related to this, it's not the fault of Zinc.
If you can confirm that this is what is happening, then I guess it is time to update that trusty old squeaksource.com image :-)
Dave
On Wed, Mar 15, 2017 at 8:19 PM, Patrick R. <patrick.rein@hpi.de> wrote:
Hi everyone,
I have been working on bringing http://squeaksource.com/ical/ up to speed for Squeak and wanted to make sure that it also works for Pharo.
Therefore,
I have created a travis build job for Squeak and Pharo (https://travis-ci.org/codeZeilen/ical-smalltalk/jobs/211298950) which pulls the source from squeaksource.com.
Now the issue is that loading the package in Pharo fails with a GoferException wrapping a ZnInvalidUTF8 Exception. We figured that this might be the result of the squeaksource page delivering the page as iso-8859-1 as it contains special characters. Any ideas on how to get this to work? I do not have access to the ical repository description and I would like to avoid mirroring the whole repository on GitHub.
In a fresh 60437 image, in Playground evaluating...
Metacello new configuration: 'ICal'; repository: 'github://codeZeilen/ical-smalltalk:master/repository'; onConflict: [:ex | ex allow]; load. ==> Could not resolve: ICal-Core [ICal-Core-PaulDeBruicker.5] in /home/ben/.local/share/Pharo/images/60437-01/pharo-local/package-cache http://squeaksource.com/ical ERROR: 'GoferRepositoryError: Could not access http://squeaksource.com/ical: ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding'
In a new fresh 60437 Image (i.e. empty package-cache) World menu > Monticello > +Repository > squeaksource.com... MCSqueaksourceRepository location: 'http://squeaksource.com/ical' user: '' password: '' ==> open repository then errors "MCRepositoryError: Could not access http://squeaksource.com/ical: ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding"
In Chrome, opening http://www.squeaksource.com/ical then clicking <Versions> and the browser's View Page Source, I see... <?xml version="1.0" encoding="iso-8859-1"?>
Googling: zinc iso-8859-1 finds... http://forum.world.st/Problem-using-Zinc-in-Pharo-4-Moose-5-1-td4825329.html but "ZnByteEncoder iso88591" errors with "KeyNotFound: key 'iso88591' not found in Dictionary" and inspecting "ZnByteEncoder byteTextConverters keys sorted" confirms this key is missing (@Sven, I'm curious why was this removed? )
Now https://en.wikipedia.org/wiki/ISO/IEC_8859-1 indicates IBM819 is an alias and " ZnByteEncoder newForEncoding: 'ibm819' " works okay
So in MCHttpRepository>>#loadAllFileNames changing... queryAt: 'C' put: 'M;O=D' ; get. to... queryAt: 'C' put: 'M;O=D' . ZnDefaultCharacterEncoder value: (ZnByteEncoder newForEncoding: 'ibm819') during: [client get].
Then from Monticello opening the previously defined http://squeaksource.com/ical works!!
Now I was hoping that reverting #loadAllFileNames and in Playground doing... converters := ZnByteEncoder byteTextConverters. converters at: 'iso-8859-1' put: (converters at: 'ibm819'). might alleviate the problem, but no luck.
Anyone know a better way to deal with this that hardcoding the encoding into #loadAllFileNames?
cheers -ben