Hi, We have been hit in DrGeo by another regression in Pharo8 (it is not present in DrGeo versions based on previous Pharo). When loading a Sketch with accentuated Latin characters (same occurs with Chinese but I have so far not enough information to assert it is the same error) there is a ZnInvalidUTF8 error. Attached Pharo8 See details https://bugs.launchpad.net/drgeo/+bug/1837745 For the record the stream passed to the XML parser (an old version but I don't think it is the problem) is from readStream message send to FileReference. Do you want a bug ticket? Hilaire -- Dr. Geo http://drgeo.eu
Can you include the file that you are trying to load ? Also, the stack trace is very short. Are you sure XML parser is ready for Pharo 8 (i.e. are its tests green) ?
On 24 Jul 2019, at 16:59, Hilaire <hilaire@drgeo.eu> wrote:
Hi,
We have been hit in DrGeo by another regression in Pharo8 (it is not present in DrGeo versions based on previous Pharo).
When loading a Sketch with accentuated Latin characters (same occurs with Chinese but I have so far not enough information to assert it is the same error) there is a ZnInvalidUTF8 error.
Attached Pharo8
See details https://bugs.launchpad.net/drgeo/+bug/1837745
For the record the stream passed to the XML parser (an old version but I don't think it is the problem) is from readStream message send to FileReference.
Do you want a bug ticket?
Hilaire
-- Dr. Geo http://drgeo.eu
<PharoDebug.log>
Hi Sven, Le 24/07/2019 à 17:05, Sven Van Caekenberghe a écrit :
Can you include the file that you are trying to load ?
Attached. It looks like issue come when saving the file. The UTF8 characters are not saved properly. The unix file command was indicating it is an utf8 but Emacs shows it was different.
Also, the stack trace is very short.
Yes, I saw that... I don't know why. I did not cut it. I got a longer one this time. But it is likely not the cause.
Are you sure XML parser is ready for Pharo 8 (i.e. are its tests green) ?
My bad, I am using P7. But I don't know either about the full compatibility of the XMLParser version I am using, the one before compatibility break. It is now clear the is issue when saving. To write, I use WriteStream on String. Should it be different? Hilaire -- Dr. Geo http://drgeo.eu
On 24 Jul 2019, at 17:32, Hilaire <hilaire@drgeo.eu> wrote:
Hi Sven,
Le 24/07/2019 à 17:05, Sven Van Caekenberghe a écrit :
Can you include the file that you are trying to load ?
Attached.
It looks like issue come when saving the file. The UTF8 characters are not saved properly. The unix file command was indicating it is an utf8 but Emacs shows it was different.
Also, the stack trace is very short.
Yes, I saw that... I don't know why. I did not cut it. I got a longer one this time. But it is likely not the cause.
Are you sure XML parser is ready for Pharo 8 (i.e. are its tests green) ?
My bad, I am using P7. But I don't know either about the full compatibility of the XMLParser version I am using, the one before compatibility break.
It is now clear the is issue when saving. To write, I use WriteStream on String. Should it be different?
But how do you write it to a file then ? FileLocator desktop / 'hilaire.xml' writeStreamDo: [ :out | out << 'Les élèves Françaises ont 10 â¬' ]. (FileLocator desktop / 'hilaire.xml') contents. ">> 'Les élèves Françaises ont 10 â¬'"
Hilaire
-- Dr. Geo http://drgeo.eu
<test.fgeo><PharoDebug.log>
[...] stream := WriteStream on: (String new: 4000). DrGeoXml new   app: self app;   saveOn: stream. [...] DrGeoXml>>saveOn: stream    | doc writer root |    doc := XMLDocument new version: '1.0'.    writer := XMLWriter on: stream.    root := XMLElement named: #drgenius.    self writeFigureAsXmlTo: root.    doc addElement: root.    doc printXMLOn: writer. The stream is passed to the XMLWriter, then it goes down to each items of the sketch to write its own XML node. For example, to write its basic information as its name, where I have issue: writeAsXmlTo: aNode    "return the newly created element"    | node |    self rehash.    node := XMLElement named: self basicType attributes: Dictionary new.    node attributeAt: #type put: self nodeType;       attributeAt: #name put: (name ifNil: ['']);       attributeAt: #id put: self hash asString.    self writeParentsAsXmlTo: node.    aNode addElement: node.    ^node This code is as is since about 2005-2007 I will dig in it but it is strange. Le 24/07/2019 à 18:45, Sven Van Caekenberghe a écrit :
But how do you write it to a file then ?
FileLocator desktop / 'hilaire.xml' writeStreamDo: [ :out | out << 'Les élèves Françaises ont 10 â¬' ].
(FileLocator desktop / 'hilaire.xml') contents.
">> 'Les élèves Françaises ont 10 â¬'"
-- Dr. Geo http://drgeo.eu
Hi Hilaire I downloaded latest Dr Geo and took a quick look at how the files are saved. It looks like the xml contents are not encoded (in UTF-8) but simply written to a binary write stream, which means that when a WideString is written you should consistently get a badly encoded file. You could test this out by changing this code DrGDirectoryLocal>>put: stream into: filename (location asFileReference / filename) ensureDelete binaryWriteStreamDo: [ :fileStream | fileStream nextPutAll: stream contents] to rather be: DrGDirectoryLocal>>put: stream into: filename (location asFileReference / filename) ensureDelete writeStreamEncoded: 'utf-8' do: [ :fileStream | fileStream nextPutAll: stream contents ] When I made this modification it worked for me i.e. the file was readable and correctly encoded. Regards Carlo On 24 Jul 2019, at 21:07, Hilaire <hilaire@drgeo.eu> wrote: [...] stream := WriteStream on: (String new: 4000). DrGeoXml new app: self app; saveOn: stream. [...] DrGeoXml>>saveOn: stream | doc writer root | doc := XMLDocument new version: '1.0'. writer := XMLWriter on: stream. root := XMLElement named: #drgenius. self writeFigureAsXmlTo: root. doc addElement: root. doc printXMLOn: writer. The stream is passed to the XMLWriter, then it goes down to each items of the sketch to write its own XML node. For example, to write its basic information as its name, where I have issue: writeAsXmlTo: aNode "return the newly created element" | node | self rehash. node := XMLElement named: self basicType attributes: Dictionary new. node attributeAt: #type put: self nodeType; attributeAt: #name put: (name ifNil: ['']); attributeAt: #id put: self hash asString. self writeParentsAsXmlTo: node. aNode addElement: node. ^node This code is as is since about 2005-2007 I will dig in it but it is strange. Le 24/07/2019 à 18:45, Sven Van Caekenberghe a écrit :
But how do you write it to a file then ?
FileLocator desktop / 'hilaire.xml' writeStreamDo: [ :out | out << 'Les élèves Françaises ont 10 â¬' ].
(FileLocator desktop / 'hilaire.xml') contents.
">> 'Les élèves Françaises ont 10 â¬'"
-- Dr. Geo http://drgeo.eu
participants (3)
-
Hilaire -
sk -
Sven Van Caekenberghe