January 2019 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [ANN] New stable VM version.

by Alistair Grant

Hi Esteban, On Fri, 18 Jan 2019 at 14:49, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > Hi, > > Yes, I needed to promote a new stable version because last weekâs was having a bug (which is not fixed). > This one is based on the build: > > 201901172323-5a38b34 On Linux (Ubuntu 16.04) it's still the old VM: $ curl get.pharo.org/64/vm70 | bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5320 100 5320 0 0 37021 0 --:--:-- --:--:-- --:--:-- 37202 Downloading the latest pharoVM: http://files.pharo.org/get-files/70/pharo64-linux-stable.zip pharo-vm/pharo Creating starter scripts pharo and pharo-ui $ ./pharo --version 5.0-201901051900 Sat Jan 5 19:12:50 UTC 2019 gcc 4.8 [Production Spur 64-bit VM] CoInterpreter VMMaker.oscog-eem.2504 uuid: a00b0fad-c04c-47a6-8a11-5dbff110ac11 Jan 5 2019 StackToRegisterMappingCogit VMMaker.oscog-eem.2504 uuid: a00b0fad-c04c-47a6-8a11-5dbff110ac11 Jan 5 2019 VM: 201901051900 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Sat Jan 5 20:00:11 2019 CommitHash: 7a3c6b64 Plugins: 201901051900 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Linux travis-job-f22c8934-2412-48ed-8180-7a42b62c7389 4.4.0-101-generic #124~14.04.1-Ubuntu SMP Fri Nov 10 19:05:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux plugin path: /dev/shm/vm/pharo-vm/lib/pharo/5.0-201901051900 [default: /dev/shm/vm/pharo-vm/lib/pharo/5.0-201901051900/] Cheers, Alistair > Enjoy (and please report problems) > > Esteban

Jan. 18, 2019

Re: [Pharo-dev] DebugSession>>activePC:

by ducasse

>> >> On my TODO is to make it stand-alone and provide is as a âcompatibility transformâ, too. > > I have to dig because I remember that I went over all the deprecation in Pharo 60 and started to look at the ones that I > could âtransformifiedâ so that we get a nice package that rewrite more :) > > >> So we can add it to methods that we want to keep for compatibility, but they will nevertheless transform the code automatically. >> (this then might be disabled in production to not transform) > > Yes I like that I will look for my code. Apparently I published under MCSmalltalkhubRepository owner: 'PharoExtras' project: âMigrator' Several packages MigratorPharo60 contains what I did for Pharo60 probably Migrator contains some of the Pharo70 Now I would like to understand how to organise it. I have the impression that we should keep the one that can be transformed. I do not think that we should go to Pharo50 because it is too old. Stef

Jan. 18, 2019

Re: [Pharo-dev] Better management of encoding of environment variables

by ducasse

>>> I think that will just overcomplicate things. Right now, all Strings in Pharo are unicode strings. >> >> Cool. I didn't realise that. But to be pedantic, which unicode encoding? >> Should I presume from Sven's "UTF-8 encoding step" comment below >> and the WideString class comment "This class represents the array of 32 bit wide characters" >> that the WideString encoding is UTF-32? So should its comment be updated to advise that? > > Not really, Pharo Strings are a collection of Characters, each of which is a Unicode code point (yes a 32 bit one). > > An encoding projects this rather abstract notion onto a sequence of bytes, > > UTF-32 (ZnUTF32Encoder, https://en.wikipedia.org/wiki/UTF-32 <https://en.wikipedia.org/wiki/UTF-32>) is for example endian dependent. > > Read the first part of > > https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf… <https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf…> I love that book :) This is too cool to have cool doc

Jan. 18, 2019

Re: [Pharo-dev] DebugSession>>activePC:

by ducasse

Hi marcus >> I simply love the dynamic rewriting this is just too cool. We should systematically use it. >> I will continue to use it in any deprecation. >> > > On my TODO is to make it stand-alone and provide is as a âcompatibility transformâ, too. I have to dig because I remember that I went over all the deprecation in Pharo 60 and started to look at the ones that I could âtransformifiedâ so that we get a nice package that rewrite more :) > So we can add it to methods that we want to keep for compatibility, but they will nevertheless transform the code automatically. > (this then might be disabled in production to not transform) Yes I like that I will look for my code. > >> Now I have a simple question (You can explain it to me over lunch one of these days). >> >> I do not get why RBAST would not be a good representation for the compiler? >> I would like to know what is the difference. >> > I think it is a good one. I have not yet seen a reason why not. But remember, Roel left Squeak because his visitor pattern for the compiler was rejected as a dumb ideaâ¦ so there are definitely different views on core questions. > > E.g. the RB AST is annotated and the whole things for sure uses a bit more memory than the compiler designed for a machine from 1978. > >> You mean that before going from BC to AST was difficult? > > You need to do the mapping somehow, the compiler needs to remember the BC offset in the code generation phase and the AST (somehow) needs to store that information (either in every node or some table). > >> How opal performs it? It does not use the source of the method to recreate the AST but he can do it from the BC? >> > > It uses the IR (which I still am not 100% sure about, it came from the old âClosureCompilerâ Design and it turned out to be quite useful, for example for the mapping: every IR node retains the offset of the BC it creates, then the IR Nodes > retain the AST node that created them. > > -> so we just do a query: âIRMethod, give me the IRInstruction that created BC offset X. then âIR, which AST node did create you? then the AST Node: what is your highlight interval in the source? > > The devil is in the detail as one IR can produce multiple byte code offsets (and byte codes) and one byte code might be created by two IR nodes, but it does seem to work with some tricks. ok I see. And all in all it is really nice. > Which I want to remove by improving the mapping and even the IR moreâ¦ there is even the question: do we need the IR? could we not do it simpler? > > The IR was quite nice back when we tried to do things with byte code manipulation (Bytesurgeon), now it feels a bit of an overkill. But it simplifies e.g. the bc mapping. We should document this because this is cool. Stef

Jan. 18, 2019

Re: [Pharo-dev] Better management of encoding of environment variables

by Sven Van Caekenberghe

> On 18 Jan 2019, at 14:45, Ben Coman <btc(a)openinworld.com> wrote: > > > > On Fri, 18 Jan 2019 at 21:39, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > > On 18 Jan 2019, at 14:23, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > > > > > I think that will just overcomplicate things. Right now, all Strings in Pharo are unicode strings. > > Cool. I didn't realise that. But to be pedantic, which unicode encoding? > Should I presume from Sven's "UTF-8 encoding step" comment below > and the WideString class comment "This class represents the array of 32 bit wide characters" > that the WideString encoding is UTF-32? So should its comment be updated to advise that? Not really, Pharo Strings are a collection of Characters, each of which is a Unicode code point (yes a 32 bit one). An encoding projects this rather abstract notion onto a sequence of bytes, UTF-32 (ZnUTF32Encoder, https://en.wikipedia.org/wiki/UTF-32) is for example endian dependent. Read the first part of https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf… > cheers -ben > > Characters are represented with their corresponding unicode codepoint. > > If all characters in a string have codepoints < 256 then they are just stored in a bytestring. Otherwise they are WideStrings. > > > > I think assuming a single representation for strings, and then encode when interacting with external apps/APIs is MUCH simpler. > > Absolutely ! > > (and yes I know that for outgoing FFI calls that might mean a UTF-8 encoding step, so be it).

Jan. 18, 2019

[Pharo 7.0] Build #131: 22887-TonelWritercreateDefaultOrganizationFrom-can-return-string

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #131 was: SUCCESS. The Pull Request #2267 was integrated: "22887-TonelWritercreateDefaultOrganizationFrom-can-return-string" Pull request url: https://github.com/pharo-project/pharo/pull/2267 Issue Url: https://pharo.fogbugz.com/f/cases/22887 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Jan. 18, 2019

Re: [Pharo-dev] Better management of encoding of environment variables

by Guillermo Polito

On Fri, Jan 18, 2019 at 2:46 PM Ben Coman <btc(a)openinworld.com> wrote: > > > On Fri, 18 Jan 2019 at 21:39, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> >> >> > On 18 Jan 2019, at 14:23, Guillermo Polito <guillermopolito(a)gmail.com> >> wrote: >> > >> > >> > I think that will just overcomplicate things. Right now, all Strings in >> Pharo are unicode strings. > > > Cool. I didn't realise that. But to be pedantic, which unicode encoding? > Should I presume from Sven's "UTF-8 encoding step" comment below > and the WideString class comment "This class represents the array of 32 > bit wide characters" > that the WideString encoding is UTF-32? So should its comment be updated > to advise that? > None :D That's the funny thing, they are not encoded. Actually, you should see Strings as collections of Characters, and Characters defined in terms of their abstract code points. ByteStrings are an optimized (just more compact) version that stores codepoints that fit in a byte. > cheers -ben > > Characters are represented with their corresponding unicode codepoint. >> > If all characters in a string have codepoints < 256 then they are just >> stored in a bytestring. Otherwise they are WideStrings. >> > >> > I think assuming a single representation for strings, and then encode >> when interacting with external apps/APIs is MUCH simpler. >> >> Absolutely ! >> >> (and yes I know that for outgoing FFI calls that might mean a UTF-8 >> encoding step, so be it). >> > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Jan. 18, 2019

Re: [Pharo-dev] DebugSession>>activePC:

by Nicolas Cellier

Le ven. 18 janv. 2019 Ã 14:42, Marcus Denker <marcus.denker(a)inria.fr> a Ã©crit : > > > > On 18 Jan 2019, at 14:26, ducasse <stepharo(a)netcourrier.com> wrote: > > > > I simply love the dynamic rewriting this is just too cool. We should > systematically use it. > > I will continue to use it in any deprecation. > > > > On my TODO is to make it stand-alone and provide is as a âcompatibility > transformâ, too. > > So we can add it to methods that we want to keep for compatibility, but > they will nevertheless transform the code automatically. > (this then might be disabled in production to not transform) > > > Now I have a simple question (You can explain it to me over lunch one of > these days). > > > > I do not get why RBAST would not be a good representation for the > compiler? > > I would like to know what is the difference. > > > I think it is a good one. I have not yet seen a reason why not. But > remember, Roel left Squeak because his visitor pattern for the compiler was > rejected as a dumb ideaâ¦ so there are definitely different views on core > questions. > > E.g. the RB AST is annotated and the whole things for sure uses a bit more > memory than the compiler designed for a machine from 1978. > > > You mean that before going from BC to AST was difficult? > > You need to do the mapping somehow, the compiler needs to remember the BC > offset in the code generation phase and the AST (somehow) needs to store > that information (either in every node or some table). > > > How opal performs it? It does not use the source of the method to > recreate the AST but he can do it from the BC? > > > > It uses the IR (which I still am not 100% sure about, it came from the old > âClosureCompilerâ Design and it turned out to be quite useful, for example > for the mapping: every IR node retains the offset of the BC it creates, > then the IR Nodes > retain the AST node that created them. > > -> so we just do a query: âIRMethod, give me the IRInstruction that > created BC offset X. then âIR, which AST node did create you? then the AST > Node: what is your highlight interval in the source? > > The devil is in the detail as one IR can produce multiple byte code > offsets (and byte codes) and one byte code might be created by two IR > nodes, but it does seem to work with some tricks. > Which I want to remove by improving the mapping and even the IR moreâ¦ > there is even the question: do we need the IR? could we not do it simpler? > > The IR was quite nice back when we tried to do things with byte code > manipulation (Bytesurgeon), now it feels a bit of an overkill. But it > simplifies e.g. the bc mapping. > > Marcus > > > agree! IR is super when we want to manipuate the byte codes (decompiling, instrumenting, etc...). But if it's just for generating the CompiledMethod byte codes (compiling), it's a bit too much and is one contributor of compilation slowdown. Maybe we could have another polymorphic kind of IRBuilder that would be a DirectByteCodeGenerator

Jan. 18, 2019

Re: [Pharo-dev] [ANN] New stable VM version.

by Guillermo Polito

On Fri, Jan 18, 2019 at 2:49 PM Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > Hi, > > Yes, I needed to promote a new stable version because last weekâs was > having a bug (which is not fixed). > not? now? :) This version includes the following: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/355 https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/352 https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/354 > This one is based on the build: > > 201901172323-5a38b34 > > Enjoy (and please report problems) > > Esteban > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Jan. 18, 2019

Re: [Pharo-dev] Better management of encoding of environment variables

by David T. Lewis

On Fri, Jan 18, 2019 at 01:40:26PM +0100, Sven Van Caekenberghe wrote: > Dave, > > > On 18 Jan 2019, at 01:54, David T. Lewis via Pharo-dev <pharo-dev(a)lists.pharo.org> wrote: > > > > > > From: "David T. Lewis" <lewis(a)mail.msen.com> > > Subject: Re: [Pharo-dev] Better management of encoding of environment variables > > Date: 18 January 2019 at 01:54:34 GMT+1 > > To: Pharo Development List <pharo-dev(a)lists.pharo.org> > > > > > > On Thu, Jan 17, 2019 at 04:57:18PM +0100, Sven Van Caekenberghe wrote: > >> > >>> On 16 Jan 2019, at 23:23, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > >>> > >>> On Wed, Jan 16, 2019 at 2:37 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >>> > >>> The image side is perfectly capable of dealing with platform differences > >>> in a clean/clear way, and at least we can then use the full power of our > >>> language and our tools. > >>> > >> Agreed. At the same time I think it is very important that we don't reply > >> on the FFI for environment variable access. This is a basic cross-platform > >> facility. So I would like to see the environment accessed through primitives, > >> but have the image place interpretation on the result of the primitive(s), > >> and have the primitive(s) answer a raw result, just a sequence of uninterpreted > >> bytes. > >> > >> OK, I can understand that ENV VAR access is more fundamental than FFI > >> (although FFI is already essential for Pharo, also during startup). > >> > >>> VisualWorks takes this approach and provides a class UninterpretedBytes > >>> that the VM is aware of. That's always seemed like an ugly name and > >>> overkill to me. I would just use ByteArray and provide image level > >>> conversion from ByteArray to String, which is what I believe we have anyway. > >> > >> Right, bytes are always uninterpreted, else they would be something else. > >> We got ByteArray>>#decodedWith: and ByteArray>>#utf8Decoded and our ByteArray > >> inspector decodes automatically if it can. > >> > > > > Hi Sven, > > > > I am the author of the getenv primitives, and I am also sadly uninformed > > about matters of character sets and strings in a multilingual environment. > > > > The primitives answer environment variable variable values as ByteString > > rather than ByteArray. This made sense to me at the time that I wrote it, > > because ByteString is easy to display in an inspector, and because it is > > easily converted to ByteArray. > > > > For an American English speaker this seems like a good choice, but I > > wonder now if it is a bad decision. After all, it is also trivially easy > > to convert a ByteArray to ByteString for display in the image. > > > > Would it be helpful to have getenv primitives that answer ByteArray > > instead, and to let all conversion (including in OSProcess) be done in > > the image? > > > > Thanks, > > Dave > > Normally, the correct way to represent uninterpreted bytes is with a ByteArray. Decoding these bytes as characters is the specific task of a character encoder/decoder, with a deliberate choice as to which to use. > > Since the getenv() system call uses simple C strings, it is understandable that this was carried over. It is probably not worth or too risky to change that - as long as the receiver understands that it is a raw OS string that needs more work. > > Like with file path encoding/decoding, environment variable encoding/decoding is plain messy and complex. IMHO it is better to manage that at the image level where we are more agile and can better handle that complexity. > Thanks Sven, that makes perfect sense to me. > > BTW: using funny Unicode chars, like ???? [https://www.fileformat.info/info/unicode/char/1f388/index.htm] is something even English speakers do. > You are right. I wrote those getenv primitives 20 years ago and back then we were still doing our emoticons like this: ;-) Thanks, Dave

Jan. 18, 2019