- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] pharo-contribution CI machines out of disk space

by Christophe Demarey

> Le 17 avr. 2018 Ã 06:02, monty <monty2(a)programmer.net> a Ã©crit : > > https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/1820/console > Youâre right. I did some cleaning and put slaves back on line. Thanks for the report.

April 17, 2018

[Pharo 7.0-dev] Build #773: 21523-package-ReflectionMirrors-Primitives-needs-to-be-moved-to-the-BaselineOfPharoBootstrap

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #773 was: SUCCESS. The Pull Request #1051 was integrated: "21523-package-ReflectionMirrors-Primitives-needs-to-be-moved-to-the-BaselineOfPharoBootstrap" Pull request url: https://github.com/pharo-project/pharo/pull/1051 Issue Url: https://pharo.fogbugz.com/f/cases/21523 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

April 17, 2018

Re: [Pharo-dev] [Pharo-Launcher] call for tests on Windows

by Christophe Demarey

Windows 10 on top of virtual box > Le 17 avr. 2018 Ã 10:29, phil(a)highoctane.be a Ã©crit : > > Hi Christophe, > > What Windows version do you have? > > I have Windows 10 and a 4K screen. > > Phil > > On Tue, Apr 17, 2018 at 9:43 AM, Christophe Demarey <christophe.demarey(a)inria.fr <mailto:christophe.demarey@inria.fr>> wrote: > Hi Phil, Eliot, > >> Le 16 avr. 2018 Ã 23:58, Eliot Miranda <eliot.miranda(a)gmail.com <mailto:eliot.miranda@gmail.com>> a Ã©crit : >> >> Hi Phil, >> >> On Mon, Apr 16, 2018 at 12:54 PM, phil(a)highoctane.be <mailto:phil@highoctane.be> <phil(a)highoctane.be <mailto:phil@highoctane.be>> wrote: >> I have downloaded the https://github.com/pharo-project/pharo-vm/blob/master/opensmalltalk-vm/buil… <https://github.com/pharo-project/pharo-vm/blob/master/opensmalltalk-vm/buil…> and put it in the folder that contains the PharoLauncher Pharo.exe and then the pointers are correct again (they have their masking done correctly. >> >> Good! >> >> So, there is a packaging issue somewhere for Pharo Windows VMs as downloaded by the Laucher. >> >> Indeed, we must include the manifest files on Windows. > > I did not get this problem. Is this manifest file new? Is there only one manifest file to include or many? > > Thanks for reporting this issue. > Christophe > >

April 17, 2018

Re: [Pharo-dev] Environment variables encoding ?

by Henrik Sperre Johansen

Damien Pollet wrote > It seems macOS normalizes UTF-8 differently from everyone else in file > names (I think base character + composing instead of precomposed > codepoint). That might affect PWD. > For environment variables, even if most sensible platforms should have > adopted UTF-8 by now, I wouldn't be surprised if there's no official > encoding whatsoever (i.e. they're just bytes with a 0 at the endâ¦) > > On 17 April 2018 at 09:36, Sven Van Caekenberghe < > sven@ > > wrote: > >> Hi, >> >> The dictionary >> >> OSPlatform current environment >> >> contains a copy of the OS's environment variables (more correctly of the >> VM process), as key/value pairs. >> >> These are obtained via the following system calls: >> >> on macOS & *nix >> >> LIBC environ >> >> on Windows >> >> KERNEL32 GetEnvironmentStrings >> >> It is however a bit unclear how these are encoded. On macOS & *nix that >> seems to be UTF8, on Windows there are some reports that it appears to be >> Latin1 - but both might be locale specific, I don't know either way. >> >> Does anyone know for sure ? >> >> I furthermore think that OSEnvironment and its subclasses, who do this >> call, should be responsible for decoding the C strings into proper Pharo >> strings, and not leave that responsibility to its users. >> >> Fundamentally, in the following, the decoding is still not done correctly >> and that is wrong/confusing IMHO. >> >> $ FOO=benoÃ®t ./pharo Pharo.image eval 'OSEnvironment current >> associations' >> {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. >> 'SHELL'->'/bin/bash'. 'TMPDIR'->'/var/folders/sy/ >> sndrtj9j1tq06j0lfnshmrl80000gn/T/'. 'FOO'->'benoÃÂ®t'. >> 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'. >> 'TERM_PROGRAM_VERSION'->'404'. >> 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. >> 'USER'->'sven'. >> 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. >> 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'. >> 'PWD'->'/tmp/benoiÃât'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. >> 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. >> 'LC_CTYPE'->'UTF-8'. 'DISPLAY'->'/private/tmp/com. >> apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'. >> 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoiÃât'. >> '_'->'/tmp/benoiÃât/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. >> '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'} >> >> Of course, if we change this, we will need to fix callers. >> >> Opinions ? >> >> Sven >> >> PS: Furthermore, I note that there is a subtle difference in how $FOO and >> $PWD in the above are UTF-8 encoded. In the former, normalisation was >> done, >> in the latter not. Maybe that could lead to problems (when >> comparing/composing them). This is a difficult/complex subject ( >> https://medium.com/concerning-pharo/an-implementation-of-unicode- >> normalization-7c6719068f43). >> >> >> >> > > > -- > Damien Pollet > type less, do more [ | ] http://people.untyped.org/damien.pollet If by different, you mean that it actually normalizes the file names, then yes. All Mac filenames are in a well defined form; NFD. On linux, they're just arrays of bytes, and anything goes. That the bytes mostly happen to be valid utf8 strings in NFC, is just a by-product of the fact that's the format most programs use when calling the file primitives. Cheers, Henry -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

April 17, 2018

Re: [Pharo-dev] Environment variables encoding ?

by Sven Van Caekenberghe

Yes, I also saw the *W variants, my initial reaction was that those would make more sense, but it is hard to see all consequences. > On 17 Apr 2018, at 10:49, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > > primitiveGetenv returns values in the current locale's code page on Windows; > a value bound to â¬ returns a stings with single char 128 on MS1252 (western > european) at least. > > On windows, there are three versions of each api call with string > parameters/returns; > xxx (depending on UNICODE being defined, either resolves to *A or *W) > xxxA (Ascii, or, more accurately, current code page)* > xxxW (UTF-16) > > IIRC, the intention is that primitives receiving/sending char* to the image > will expect/return utf8, so a conversion macro before passing it on to the > syscall would be necessary on Windows; I believe one exist already and is > used in at least the file plugin primitives. > > Cheers, > Henry > > * The windows FFI fallback used if primitive fails calls the *A version > directly, and can be changed to call *W correctly, but there's a fair bit of > wrapping fluff involved; > https://youtu.be/Um41DPPs5ZA?list=PL843D1D545F9F52B6&t=1591 > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html >

April 17, 2018

Re: [Pharo-dev] Environment variables encoding ?

by Sven Van Caekenberghe

> On 17 Apr 2018, at 10:40, Nicolai Hess <nicolaihess(a)gmail.com> wrote: > > > > 2018-04-17 10:05 GMT+02:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > > > > On 17 Apr 2018, at 09:57, Damien Pollet <damien.pollet(a)gmail.com> wrote: > > > > It seems macOS normalizes UTF-8 differently from everyone else in file names (I think base character + composing instead of precomposed codepoint). That might affect PWD. > > For environment variables, even if most sensible platforms should have adopted UTF-8 by now, I wouldn't be surprised if there's no official encoding whatsoever (i.e. they're just bytes with a 0 at the endâ¦) > > ;-) > > We can decode everything, we have all the tools, but of course, we first have to know what encoding is being used. Hence my question. > > > On 17 April 2018 at 09:36, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Hi, > > > > The dictionary > > > > OSPlatform current environment > > > > contains a copy of the OS's environment variables (more correctly of the VM process), as key/value pairs. > > > > These are obtained via the following system calls: > > > > on macOS & *nix > > > > LIBC environ > > > > on Windows > > > > KERNEL32 GetEnvironmentStrings > > > Interestingly, this is only for the dictionary operations (asDictionary, keysAndValuesDo...) > If you just access the variable with getEnv, it works: > > OSPlatform current environment setEnv:'FOO' value:'benoÃ®t'. > OSPlatform current environment getEnv:'FOO'. "'benoÃ®t'" > OSPlatform current environment asDictionary at: 'FOO'. "'benoÅt'" Hmm, not for me (on macOS): $ FOO=benoÃ®t ./pharo Pharo.image eval "OSPlatform current environment getEnv:'FOO'" 'benoÃÂ®t' If you put it in yourself, are you not cheating then ? > > > > It is however a bit unclear how these are encoded. On macOS & *nix that seems to be UTF8, on Windows there are some reports that it appears to be Latin1 - but both might be locale specific, I don't know either way. > > > > Does anyone know for sure ? > > > > I furthermore think that OSEnvironment and its subclasses, who do this call, should be responsible for decoding the C strings into proper Pharo strings, and not leave that responsibility to its users. > > > > Fundamentally, in the following, the decoding is still not done correctly and that is wrong/confusing IMHO. > > > > $ FOO=benoÃ®t ./pharo Pharo.image eval 'OSEnvironment current associations' > > {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. 'SHELL'->'/bin/bash'. 'TMPDIR'->'/var/folders/sy/sndrtj9j1tq06j0lfnshmrl80000gn/T/'. 'FOO'->'benoÃÂ®t'. 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'. 'TERM_PROGRAM_VERSION'->'404'. 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. 'USER'->'sven'. 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'. 'PWD'->'/tmp/benoiÃât'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. 'LC_CTYPE'->'UTF-8'. 'DISPLAY'->'/private/tmp/com.apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'. 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoiÃât'. '_'->'/tmp/benoiÃât/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'} > > > > Of course, if we change this, we will need to fix callers. > > > > Opinions ? > > > > Sven > > > > PS: Furthermore, I note that there is a subtle difference in how $FOO and $PWD in the above are UTF-8 encoded. In the former, normalisation was done, in the latter not. Maybe that could lead to problems (when comparing/composing them). This is a difficult/complex subject (https://medium.com/concerning-pharo/an-implementation-of-unicode-normalizat…) > > > > > > > > > > > > > > -- > > Damien Pollet > > type less, do more [ | ] http://people.untyped.org/damien.pollet

April 17, 2018

Re: [Pharo-dev] Environment variables encoding ?

by Henrik Sperre Johansen

primitiveGetenv returns values in the current locale's code page on Windows; a value bound to â¬ returns a stings with single char 128 on MS1252 (western european) at least. On windows, there are three versions of each api call with string parameters/returns; xxx (depending on UNICODE being defined, either resolves to *A or *W) xxxA (Ascii, or, more accurately, current code page)* xxxW (UTF-16) IIRC, the intention is that primitives receiving/sending char* to the image will expect/return utf8, so a conversion macro before passing it on to the syscall would be necessary on Windows; I believe one exist already and is used in at least the file plugin primitives. Cheers, Henry * The windows FFI fallback used if primitive fails calls the *A version directly, and can be changed to call *W correctly, but there's a fair bit of wrapping fluff involved; https://youtu.be/Um41DPPs5ZA?list=PL843D1D545F9F52B6&t=1591 -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

April 17, 2018

Re: [Pharo-dev] Environment variables encoding ?

by Nicolai Hess

2018-04-17 10:05 GMT+02:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > > > > On 17 Apr 2018, at 09:57, Damien Pollet <damien.pollet(a)gmail.com> wrote: > > > > It seems macOS normalizes UTF-8 differently from everyone else in file > names (I think base character + composing instead of precomposed > codepoint). That might affect PWD. > > For environment variables, even if most sensible platforms should have > adopted UTF-8 by now, I wouldn't be surprised if there's no official > encoding whatsoever (i.e. they're just bytes with a 0 at the endâ¦) > > ;-) > > We can decode everything, we have all the tools, but of course, we first > have to know what encoding is being used. Hence my question. > > > On 17 April 2018 at 09:36, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Hi, > > > > The dictionary > > > > OSPlatform current environment > > > > contains a copy of the OS's environment variables (more correctly of the > VM process), as key/value pairs. > > > > These are obtained via the following system calls: > > > > on macOS & *nix > > > > LIBC environ > > > > on Windows > > > > KERNEL32 GetEnvironmentStrings > Interestingly, this is only for the dictionary operations (asDictionary, keysAndValuesDo...) If you just access the variable with getEnv, it works: OSPlatform current environment setEnv:'FOO' value:'benoÃ®t'. OSPlatform current environment getEnv:'FOO'. "'benoÃ®t'" OSPlatform current environment asDictionary at: 'FOO'. "'benoÅt'" > > > > It is however a bit unclear how these are encoded. On macOS & *nix that > seems to be UTF8, on Windows there are some reports that it appears to be > Latin1 - but both might be locale specific, I don't know either way. > > > > Does anyone know for sure ? > > > > I furthermore think that OSEnvironment and its subclasses, who do this > call, should be responsible for decoding the C strings into proper Pharo > strings, and not leave that responsibility to its users. > > > > Fundamentally, in the following, the decoding is still not done > correctly and that is wrong/confusing IMHO. > > > > $ FOO=benoÃ®t ./pharo Pharo.image eval 'OSEnvironment current > associations' > > {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. > 'SHELL'->'/bin/bash'. 'TMPDIR'->'/var/folders/sy/ > sndrtj9j1tq06j0lfnshmrl80000gn/T/'. 'FOO'->'benoÃÂ®t'. > 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'. > 'TERM_PROGRAM_VERSION'->'404'. 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. > 'USER'->'sven'. 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. > 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'. > 'PWD'->'/tmp/benoiÃât'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. > 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. > 'LC_CTYPE'->'UTF-8'. 'DISPLAY'->'/private/tmp/com. > apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'. > 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoiÃât'. > '_'->'/tmp/benoiÃât/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. > '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'} > > > > Of course, if we change this, we will need to fix callers. > > > > Opinions ? > > > > Sven > > > > PS: Furthermore, I note that there is a subtle difference in how $FOO > and $PWD in the above are UTF-8 encoded. In the former, normalisation was > done, in the latter not. Maybe that could lead to problems (when > comparing/composing them). This is a difficult/complex subject ( > https://medium.com/concerning-pharo/an-implementation-of-unicode- > normalization-7c6719068f43). > > > > > > > > > > > > > > -- > > Damien Pollet > > type less, do more [ | ] http://people.untyped.org/damien.pollet > > >

April 17, 2018

Re: [Pharo-dev] [Pharo-Launcher] call for tests on Windows

by phil＠highoctane.be

Hi Christophe, What Windows version do you have? I have Windows 10 and a 4K screen. Phil On Tue, Apr 17, 2018 at 9:43 AM, Christophe Demarey < christophe.demarey(a)inria.fr> wrote: > Hi Phil, Eliot, > > Le 16 avr. 2018 Ã 23:58, Eliot Miranda <eliot.miranda(a)gmail.com> a Ã©crit : > > Hi Phil, > > On Mon, Apr 16, 2018 at 12:54 PM, phil(a)highoctane.be <phil(a)highoctane.be> > wrote: > >> I have downloaded the https://github.com/pharo-p >> roject/pharo-vm/blob/master/opensmalltalk-vm/build.win32x86/ >> pharo.cog.spur.lowcode/Pharo.exe.manifest and put it in the folder that >> contains the PharoLauncher Pharo.exe and then the pointers are correct >> again (they have their masking done correctly. >> > > Good! > > >> So, there is a packaging issue somewhere for Pharo Windows VMs as >> downloaded by the Laucher. >> > > Indeed, we must include the manifest files on Windows. > > > I did not get this problem. Is this manifest file new? Is there only one > manifest file to include or many? > > Thanks for reporting this issue. > Christophe > >

April 17, 2018

Re: [Pharo-dev] Environment variables encoding ?

by Sven Van Caekenberghe

> On 17 Apr 2018, at 09:57, Damien Pollet <damien.pollet(a)gmail.com> wrote: > > It seems macOS normalizes UTF-8 differently from everyone else in file names (I think base character + composing instead of precomposed codepoint). That might affect PWD. > For environment variables, even if most sensible platforms should have adopted UTF-8 by now, I wouldn't be surprised if there's no official encoding whatsoever (i.e. they're just bytes with a 0 at the endâ¦) ;-) We can decode everything, we have all the tools, but of course, we first have to know what encoding is being used. Hence my question. > On 17 April 2018 at 09:36, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > Hi, > > The dictionary > > OSPlatform current environment > > contains a copy of the OS's environment variables (more correctly of the VM process), as key/value pairs. > > These are obtained via the following system calls: > > on macOS & *nix > > LIBC environ > > on Windows > > KERNEL32 GetEnvironmentStrings > > It is however a bit unclear how these are encoded. On macOS & *nix that seems to be UTF8, on Windows there are some reports that it appears to be Latin1 - but both might be locale specific, I don't know either way. > > Does anyone know for sure ? > > I furthermore think that OSEnvironment and its subclasses, who do this call, should be responsible for decoding the C strings into proper Pharo strings, and not leave that responsibility to its users. > > Fundamentally, in the following, the decoding is still not done correctly and that is wrong/confusing IMHO. > > $ FOO=benoÃ®t ./pharo Pharo.image eval 'OSEnvironment current associations' > {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. 'SHELL'->'/bin/bash'. 'TMPDIR'->'/var/folders/sy/sndrtj9j1tq06j0lfnshmrl80000gn/T/'. 'FOO'->'benoÃÂ®t'. 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'. 'TERM_PROGRAM_VERSION'->'404'. 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. 'USER'->'sven'. 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'. 'PWD'->'/tmp/benoiÃât'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. 'LC_CTYPE'->'UTF-8'. 'DISPLAY'->'/private/tmp/com.apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'. 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoiÃât'. '_'->'/tmp/benoiÃât/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'} > > Of course, if we change this, we will need to fix callers. > > Opinions ? > > Sven > > PS: Furthermore, I note that there is a subtle difference in how $FOO and $PWD in the above are UTF-8 encoded. In the former, normalisation was done, in the latter not. Maybe that could lead to problems (when comparing/composing them). This is a difficult/complex subject (https://medium.com/concerning-pharo/an-implementation-of-unicode-normalizat…) > > > > > > > -- > Damien Pollet > type less, do more [ | ] http://people.untyped.org/damien.pollet

April 17, 2018