Hi, I have discovered a problem with my DrGeo command line handler. Its purpose, under linux only for now, is to let the user open a DrGeo sketch file from the desktop file manager by double-clicking on the file. Sadly, it seems it does not work when non ascii characters are present in the path or the filename. The command line handler parameters are correctly set form the bash script, I got this output: drgeo --sketch=/home/hilaire/Axe-Symétrie/axes de symétrie.fgeo But once in Pharo, the string representing the path to the file is wrongly interpreted, I have this error from Pharo: Error: /home/hilaire/Axe-Symétrie/axes de symétrie.fgeo does not exist!
From the DrGeo.sh startup script, Pharo is fired with this instruction:
exec "$VM/pharo" \ --plugins "$VM" \ --encoding utf-8 \ -vm-display-X11 \ -title "Dr.Geo" \ $image \ $DRGEO_OPT"$filename" Opening this same file directly form DrGeo works correctly. Any idea? Thanks Hilaire -- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu
Responding to myself for archiving purpose. I forgot again Pharo internal string are not utf-8 but ByteString, so the string need to be converted back to utf8. For example in one of the handler method, it can be modified as follow: DrGeoCommandLineHandler>>loadSketch: aString | utfString | utfString := aString convertFromWithConverter: UTF8TextConverter new. self checkForFile: utfString. DrGeo fileFullscreen: utfString Et voilà . Hilaire Le 20/01/2015 15:20, Hilaire a écrit :
But once in Pharo, the string representing the path to the file is wrongly interpreted, I have this error from Pharo:
Error: /home/hilaire/Axe-Symétrie/axes de symétrie.fgeo does not exist!
-- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu
On 20 Jan 2015, at 16:21, Hilaire <hilaire@drgeo.eu> wrote:
I forgot again Pharo internal string are not utf-8 but ByteString, so the string need to be converted back to utf8.
No they are not - Strings and Characters in Pharo are using plain Unicode encoding internally. What you see in your particular case is that an external string comes in, originally encoded in UTF-8, yet the VM or maybe some code at the image level, just took the incoming string or bytes and just converted them directly to a Pharo string, not using the proper encoding. Your fix corrects that fault by re-interpreting the wrongly decoded characters as bytes using the correct encoder. http://stfx.eu/EnterprisePharo/Zinc-Encoding-Meta/ The general problem is probably that the VM or image level does not know what encoder to use (just guessing). Sven
Le 20/01/2015 16:34, Sven Van Caekenberghe a écrit :
No they are not - Strings and Characters in Pharo are using plain Unicode encoding internally.
Thanks for the update, and the reference link. Hilaire -- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu
Command line arguments enter the image level via VirtualMachine>>#getSystemAttribute: At that point they are already Strings. In your case, they must already be wrong at that point.
On 20 Jan 2015, at 16:51, Hilaire <hilaire@drgeo.eu> wrote:
Le 20/01/2015 16:34, Sven Van Caekenberghe a écrit :
No they are not - Strings and Characters in Pharo are using plain Unicode encoding internally.
Thanks for the update, and the reference link.
Hilaire
-- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu
On Tue, Jan 20, 2015 at 8:00 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Command line arguments enter the image level via VirtualMachine>>#getSystemAttribute:
At that point they are already Strings.
ByteString, according to the primitive. So if the shell supplies e.g. UTF-8 strings for command-line parameters, which the VM sees as bytes, then the ByteString instances answered by getSystemAttribute: would need decoding, right?
In your case, they must already be wrong at that point.
Not necessarily. The getSystemAttribute: primitive doesn't do decoding. Perhaps it should.
On 20 Jan 2015, at 16:51, Hilaire <hilaire@drgeo.eu> wrote:
Le 20/01/2015 16:34, Sven Van Caekenberghe a écrit :
No they are not - Strings and Characters in Pharo are using plain Unicode encoding internally.
Thanks for the update, and the reference link.
Hilaire
-- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu
-- best, Eliot
Hi Eliot,
On 20 Jan 2015, at 20:38, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Tue, Jan 20, 2015 at 8:00 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote: Command line arguments enter the image level via VirtualMachine>>#getSystemAttribute:
At that point they are already Strings.
ByteString, according to the primitive. So if the shell supplies e.g. UTF-8 strings for command-line parameters, which the VM sees as bytes, then the ByteString instances answered by getSystemAttribute: would need decoding, right?
In your case, they must already be wrong at that point.
Not necessarily. The getSystemAttribute: primitive doesn't do decoding. Perhaps it should.
Yes, probably. I just tried on Mac OS X, Pharo 4: $ export FOO=élève-Français $ echo $FOO élève-Français $ ./pharo Pharo.image eval 'OSPlatform current environment at: #FOO' 'élève-Français' $ ./pharo Pharo.image eval '(OSPlatform current environment at: #FOO) asByteArray utf8Decoded' 'élève-Français' The question is, is this true for all platforms ? Windows ?
On 20 Jan 2015, at 16:51, Hilaire <hilaire@drgeo.eu> wrote:
Le 20/01/2015 16:34, Sven Van Caekenberghe a écrit :
No they are not - Strings and Characters in Pharo are using plain Unicode encoding internally.
Thanks for the update, and the reference link.
Hilaire
-- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu
-- best, Eliot
On Tue, Jan 20, 2015 at 12:35 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi Eliot,
On 20 Jan 2015, at 20:38, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Tue, Jan 20, 2015 at 8:00 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote: Command line arguments enter the image level via VirtualMachine>>#getSystemAttribute:
At that point they are already Strings.
ByteString, according to the primitive. So if the shell supplies e.g. UTF-8 strings for command-line parameters, which the VM sees as bytes, then the ByteString instances answered by getSystemAttribute: would need decoding, right?
In your case, they must already be wrong at that point.
Not necessarily. The getSystemAttribute: primitive doesn't do decoding. Perhaps it should.
Yes, probably. I just tried on Mac OS X, Pharo 4:
$ export FOO=élève-Français
$ echo $FOO élève-Français
$ ./pharo Pharo.image eval 'OSPlatform current environment at: #FOO' 'élève-Français'
$ ./pharo Pharo.image eval '(OSPlatform current environment at: #FOO) asByteArray utf8Decoded' 'élève-Français'
The question is, is this true for all platforms ? Windows ?
I'm trying to test this in Pharo 3. I get KeyNotFound: key #FOO not found in PlatformIndependentEnvironment PlatformIndependentEnvironment(OSEnvironment)>>at: in Block: [ KeyNotFound signalFor: aKey ] UndefinedObject>>ifNil: PlatformIndependentEnvironment(OSEnvironment)>>at:ifAbsent: PlatformIndependentEnvironment(OSEnvironment)>>at: UndefinedObject>>DoIt OpalCompiler>>evaluate OpalCompiler(AbstractCompiler)>>evaluate: SmalltalkImage>>evaluate: EvaluateCommandLineHandler>>evaluate: in Block: [ ... BlockClosure>>on:do: Does the environment access depend on NativeBoost?
On 20 Jan 2015, at 16:51, Hilaire <hilaire@drgeo.eu> wrote:
Le 20/01/2015 16:34, Sven Van Caekenberghe a écrit :
No they are not - Strings and Characters in Pharo are using plain Unicode encoding internally.
Thanks for the update, and the reference link.
Hilaire
-- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu
-- best, Eliot
-- best, Eliot
On 21 Jan 2015, at 23:25, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Tue, Jan 20, 2015 at 12:35 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote: Hi Eliot,
On 20 Jan 2015, at 20:38, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Tue, Jan 20, 2015 at 8:00 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote: Command line arguments enter the image level via VirtualMachine>>#getSystemAttribute:
At that point they are already Strings.
ByteString, according to the primitive. So if the shell supplies e.g. UTF-8 strings for command-line parameters, which the VM sees as bytes, then the ByteString instances answered by getSystemAttribute: would need decoding, right?
In your case, they must already be wrong at that point.
Not necessarily. The getSystemAttribute: primitive doesn't do decoding. Perhaps it should.
Yes, probably. I just tried on Mac OS X, Pharo 4:
$ export FOO=élève-Français
$ echo $FOO élève-Français
$ ./pharo Pharo.image eval 'OSPlatform current environment at: #FOO' 'élève-Français'
$ ./pharo Pharo.image eval '(OSPlatform current environment at: #FOO) asByteArray utf8Decoded' 'élève-Français'
The question is, is this true for all platforms ? Windows ?
I'm trying to test this in Pharo 3. I get
KeyNotFound: key #FOO not found in PlatformIndependentEnvironment PlatformIndependentEnvironment(OSEnvironment)>>at: in Block: [ KeyNotFound signalFor: aKey ] UndefinedObject>>ifNil: PlatformIndependentEnvironment(OSEnvironment)>>at:ifAbsent: PlatformIndependentEnvironment(OSEnvironment)>>at: UndefinedObject>>DoIt OpalCompiler>>evaluate OpalCompiler(AbstractCompiler)>>evaluate: SmalltalkImage>>evaluate: EvaluateCommandLineHandler>>evaluate: in Block: [ ... BlockClosure>>on:do:
Does the environment access depend on NativeBoost?
Yes, I believe so.
On 20 Jan 2015, at 16:51, Hilaire <hilaire@drgeo.eu> wrote:
Le 20/01/2015 16:34, Sven Van Caekenberghe a écrit :
No they are not - Strings and Characters in Pharo are using plain Unicode encoding internally.
Thanks for the update, and the reference link.
Hilaire
-- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu
-- best, Eliot
-- best, Eliot
participants (3)
-
Eliot Miranda -
Hilaire -
Sven Van Caekenberghe