January 2019 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Better management of encoding of environment variables
by Guillermo Polito Jan. 19, 2019

Jan. 19, 2019

On Fri, Jan 18, 2019 at 1:58 AM David T. Lewis via Pharo-dev < pharo-dev(a)lists.pharo.org> wrote: > On Thu, Jan 17, 2019 at 04:57:18PM +0100, Sven Van Caekenberghe wrote: > > > > > On 16 Jan 2019, at 23:23, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > > > > > > On Wed, Jan 16, 2019 at 2:37 AM Sven Van Caekenberghe <sven(a)stfx.eu> > wrote: > > > > > > The image side is perfectly capable of dealing with platform > differences > > > in a clean/clear way, and at least we can then use the full power of > our > > > language and our tools. > > > > > Agreed. +1 At the same time I think it is very important that we don't reply > > on the FFI for environment variable access. This is a basic > cross-platform > > facility. So I would like to see the environment accessed through > primitives, > > but have the image place interpretation on the result of the > primitive(s), > > and have the primitive(s) answer a raw result, just a sequence of > uninterpreted > > bytes. > Having looked at it not so long ago, I'll add my 2cts. Environment access is a very particular scenario. We have in Pharo many startup actions that directly or indirectly (FileLocator home?) require environment variable access, and thus we have to be really careful and picky to make sure that they all work, dependencies are installed in the right order and so on... In Pharo6 this was specially difficult because FFI was dynamically compiling methods, => which required access to argument names, => which required access to the sources files, => which required access to the env vars (because in Pharo the source/changes files are looked up in other directories than the image/vm ones) => which loops :) In Pharo7 argument names in FFI calls are embedded in the method meta-data so all that is avoided. Still I'd agree that moving this support to a primitive would make it less fragile. I'd apply the same to getting/setting the working directory. > > > OK, I can understand that ENV VAR access is more fundamental than FFI > > (although FFI is already essential for Pharo, also during startup). > > > > > VisualWorks takes this approach and provides a class UninterpretedBytes > > > that the VM is aware of. That's always seemed like an ugly name and > > > overkill to me. I would just use ByteArray and provide image level > > > conversion from ByteArray to String, which is what I believe we have > anyway. > > > > Right, bytes are always uninterpreted, else they would be something else. > > We got ByteArray>>#decodedWith: and ByteArray>>#utf8Decoded and our > ByteArray > > inspector decodes automatically if it can. > > > > Hi Sven, > > I am the author of the getenv primitives, and I am also sadly uninformed > about matters of character sets and strings in a multilingual environment. > > The primitives answer environment variable variable values as ByteString > rather than ByteArray. This made sense to me at the time that I wrote it, > because ByteString is easy to display in an inspector, and because it is > easily converted to ByteArray. > > For an American English speaker this seems like a good choice, but I > wonder now if it is a bad decision. Well, as soon as you want to manage some internationalisation, indeed it is. But also it is a source of bugs, because assuming ascii is not right for english either. Most platforms will assume utf8 by default, and it's not quite the same for many symbols :). For example, Character allByteCharacters size. => 256 Character allByteCharacters utf8Encoded size. 384 Character allByteCharacters select: [ :c | c asString utf8Encoded size > 1 ]. 'â¬ âÆââ¦â â¡Ëâ°Å â¹Å Å½ âââââ¢ââËâ¢Å¡âºÅ Å¾Å¸ Â¡Â¢Â£Â¤Â¥Â¦Â§Â¨Â©ÂªÂ«Â¬ÂÂ®Â¯Â°Â±Â²Â³Â´ÂµÂ¶Â·Â¸Â¹ÂºÂ»Â¼Â½Â¾Â¿ÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃ Ã¡Ã¢Ã£Ã¤Ã¥Ã¦Ã§Ã¨Ã©ÃªÃ«Ã¬ÃÃ®Ã¯Ã°Ã±Ã²Ã³Ã´ÃµÃ¶Ã·Ã¸Ã¹ÃºÃ»Ã¼Ã½Ã¾Ã¿' Of course, many of those characters may not be used in the day-to-day of many people, but as soon as we find one of those (I'm thinking about the not super strange case of a database storing names :)). Also think about the poor windows users (like myself since 2 weeks ago), that have to think about utf16! BTW, I hope I'm not breaking anybody's mail client by pasting strange characters here :D (and if so, you may want suggest them to review how they manage encoding :)) After all, it is also trivially easy > to convert a ByteArray to ByteString for display in the image. > Yes, but it's sometimes difficult to find such places, as there are many primitives spread in a lot of places doing the wrong thing, which is a source of bugs... I'd like to fix it from the root, the question is how to do it without breaking ^^. In Pharo we are doing at many places, self primitiveXXX asByteArray utf8Decoded So making the primitives return ByteArray instances instead of ByteString should be safe enough :). But this is in my opinion clearly a hack instead of fixing the real problem, and we have to be careful to guard such patterns with comments everywhere explaining why the bytearray conversion is really needed there... > Would it be helpful to have getenv primitives that answer ByteArray > instead, and to let all conversion (including in OSProcess) be done in > the image? > Well, personally I would like that getenv/setenv and getcwd setcwd support are not in a plugin but as a basic service provided by the vm. Cheers, Guille > > Thanks, > Dave > > > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

3 2

literate programming
by Nicolai Hess Jan. 18, 2019

Jan. 18, 2019

Physically Based Rendering an interesting book, using literate programming, as free book: http://www.pbr-book.org/

1 0

Re: [Pharo-dev] [ANN] New stable VM version.
by Esteban Lorenzano Jan. 18, 2019

Jan. 18, 2019

> On 18 Jan 2019, at 15:57, Sean P. DeNigris via Pharo-dev <pharo-dev(a)lists.pharo.org> wrote: > > > From: "Sean P. DeNigris" <sean(a)clipperadams.com> > Subject: Re: [ANN] New stable VM version. > Date: 18 January 2019 at 15:57:12 CET > To: pharo-dev(a)lists.pharo.org > > > EstebanLM wrote >> Enjoy > > Thanks! Which image versions does this affect? I want to clear my Launcher > cachesâ¦ Pharo 7.0 and Pharo 8.0 images. > > > > ----- > Cheers, > Sean > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html > > >

1 0

[ Save The Date ] Pharo Days 2019: Thursday April 4 & Friday April 5 @ Lille, FR
by Sven Van Caekenberghe Jan. 18, 2019

Jan. 18, 2019

[ Save The Date ] Pharo Days 2019: Thursday April 4 & Friday April 5 @ Lille, FR Dear members of the Pharo community, We are happy to announce that we will be organising Pharo Days this year, in Lille, France. This will be a two day event: Thursday April 4 & Friday April 5. The main venue will be the Amphitheatre of INRIA Lille - Nord Europe. Each day will consist of a number of short 20 to 30 minutes tech talks in the morning, with a more free format in the afternoon: general hacking space, pair programming, demos, side meetings, tutorials, coding sprints, Q&A's, shows us your projects - real life human interaction. Of course, there will be social events as well. Please join us to make this another successful edition, after Annecy (FR) 2011, Lille 2012, Bern (CH) & Lille 2013, Lille 2015, Namur (BE) 2016 and Lille 2017. Mark your calendars, we will provide more details when they become available. The Pharo Board, Association & Consortium http://pharo.org http://association.pharo.org http://consortium.pharo.org

1 0

[ANN] New stable VM version.
by Esteban Lorenzano Jan. 18, 2019

Jan. 18, 2019

Hi, Yes, I needed to promote a new stable version because last weekâs was having a bug (which is not fixed). This one is based on the build: 201901172323-5a38b34 Enjoy (and please report problems) Esteban

4 3

Re: [Pharo-dev] DebugSession>>activePC:
by ducasse Jan. 18, 2019

Jan. 18, 2019

>> >> On my TODO is to make it stand-alone and provide is as a âcompatibility transformâ, too. > > I have to dig because I remember that I went over all the deprecation in Pharo 60 and started to look at the ones that I > could âtransformifiedâ so that we get a nice package that rewrite more :) > > >> So we can add it to methods that we want to keep for compatibility, but they will nevertheless transform the code automatically. >> (this then might be disabled in production to not transform) > > Yes I like that I will look for my code. Apparently I published under MCSmalltalkhubRepository owner: 'PharoExtras' project: âMigrator' Several packages MigratorPharo60 contains what I did for Pharo60 probably Migrator contains some of the Pharo70 Now I would like to understand how to organise it. I have the impression that we should keep the one that can be transformed. I do not think that we should go to Pharo50 because it is too old. Stef

1 0

[Pharo 7.0] Build #131: 22887-TonelWritercreateDefaultOrganizationFrom-can-return-string
by ci-pharo-ci-jenkins2＠inria.fr Jan. 18, 2019

Jan. 18, 2019

There is a new Pharo build available! The status of the build #131 was: SUCCESS. The Pull Request #2267 was integrated: "22887-TonelWritercreateDefaultOrganizationFrom-can-return-string" Pull request url: https://github.com/pharo-project/pharo/pull/2267 Issue Url: https://pharo.fogbugz.com/f/cases/22887 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

1 0

Re: [Pharo-dev] DebugSession>>activePC:
by Eliot Miranda Jan. 18, 2019

Jan. 18, 2019

Hi Thomas, > On Jan 10, 2019, at 2:24 AM, Thomas Dupriez via Pharo-dev <pharo-dev(a)lists.pharo.org> wrote: > > <mime-attachment> in a stack of contexts the active pc is different for the top context. For other than the top context, a contextâs pc will be pointing after the send that created the context above it, so to find the pc of the send one finds the previous pc. For the top context its pc is the active pc. Typically the debugger is invoked in two different modes, interruption or exception. When interrupted, a process is stopped at the next suspension point (method entry or backward branch) and the top context in the process is the context to be displayed in the debugger. When an exception occurs the exception search machinery will find the signaling context, the context that raised the exception, which will be below the search machinery and the debugger invocation above that. The active pc of the signaling context will be the of for the send of digbsl et al. So the distinction is important and the utility method is probably useful. Do you want to remove the method simply because there are no senders in the image? If so, this is indicative of a serious problem with the Pharo development process. In the summer I ported VMMaker.oscog to Pharo 6. Now as feenk try and build a VMMaker.oscog image on Pharo 7, the system is broken, in part because of depreciations and in part because useful methods (isOptimisedBlock (isOptimizedBlock?) in the Opal compiler) have been removed. Just because a method is not in the image does not imply it is not in use. It simply means that it is not in use in the base image. As the system gets modularised this issue will only increase. There are lots of collection methods that exist as a library that are not used in the base image and removing them would clearly damage the library for users. This is the case for lots of so-called system code. There are users out there, like those of us in the vm team, who rely on such system code, and it is extremely unsettling and frustrating to have that system code change all the time. If Pharo is to be a useful platform to the vm team it has to be more stable.

5 9

[Pharo 7.0] Build #130: 22896-Creating-methods-in-a-subclass-with-a-class-using-a-trait
by ci-pharo-ci-jenkins2＠inria.fr Jan. 18, 2019

Jan. 18, 2019

There is a new Pharo build available! The status of the build #130 was: FAILURE. The Pull Request #2266 was integrated: "22896-Creating-methods-in-a-subclass-with-a-class-using-a-trait" Pull request url: https://github.com/pharo-project/pharo/pull/2266 Issue Url: https://pharo.fogbugz.com/f/cases/22896 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

1 0

Some clarifications (was DebugSession>>activePC:)
by ducasse Jan. 17, 2019

Jan. 17, 2019

Hi Eliot I would like to make some clarifications. Preamble: -------------- I was thinking that I should not reply but I think that I have the right to clarify a bit because I do not like several points. I reply as Stef "the guy that had the vision of Pharo and that spent 10 years pushing and building Pharo.â So I think that it makes me credible enough. My points: ------------- - With you this is always the same: Iâm too emotional and it ends everything. Stephane is emotional so we cannot communicate with him. You often place yourself as a professional and that Iâm emotional. Could you stop this little game because to me it starts to be a bit boring? Or I will just put a filter and trash systematically your emails. - Last week you told us that time bombing our process was a bad idea in an answer about settings keeping references instead of releasing them. First this had nothing to do with the problem. You see P7 was delayed because we considered that the system was not ready but P7 should be released. May I make the remark that the world is using time-based delivery? - About CMake, you may be right that makefile is better than CMake but a part of the world is using CMake and the net result is that we lost our effort and infrastructure just to follow you. Ronie uses CMake. Igor which I consider as a talented developer used CMake because he thought it was the best tool he should use. - About infrastructure and process. I wanted to check the REplugin this week end because we should use it since the world is using Perl reg-expression and I could not find the code of the plugin on github. I saw that some plugin code is not even versioned and only available on a strange ftp and it was surprising to me. Iâm still surprised that after 3 years (when esteban requested that all plugin code are grouped together in a single place, this is still not the case). - Did it not look strange to you that Esteban left the VM list in the past? Esteban looks like a reasonable and stable person to me. - You are systematically telling to me that Iâm offended when people criticized us. Iâm not offended by critics. We are not offended by critics. We have internal emails that are much more violent about our own process. We are not blind and we reflect on our own process and we invite people to REPORT problems. Now we ask people to be credible when they do it because we are busy - report situation - step by step reproducibility - proportionality of the voice versus the problem - and not been offensive We did not see any official reports about problems that we may have introduced. - You are vocal about âinstability". And you report that one method changed. Seriously? Are you not exagerating a bit? Let us talk about instability: - When we talk about instability we talk about the fact that we cannot use for real ephemerons because they corrupt the memory and that we get random crash. - We had instability when we have memory leaks that randomly slows down Pharo. - We had instability which delays our release with a bug in FFI that made cairo crash on windows 64 bits. If you want to interact with us (and not only me, because Iâm the tip of the iceberg and many people are frustrated by your attitude), let us start with positive communication and attitude: - You express your problems and we see how we (together) can fix them. - You can contribute by writing tests and by entering bug entries in the bug tracker. About Pharo: ââââââââ We understand that some people do not like it but we do not force anybody use and make business with it. Now I want to make something super clear: Pharo will not sacrifice agility and improvements on the altar of compatibility with Squeak. Pharo will change because Pharo is agile and because many things should be improved. We pay real attention about backwards compatibility. Much more than people think. Because we have many external projects and libraries that we support. Now if Pharo is used to build the VM then we will have to find a way to pay attention. And we will reinvest in building a process that checks this. Our problem is that integration time cannot take hours and right now validating Pharo is a bit too long. We will work on it. We strongly advocate to invest in tests. Tests are a good mechanisms to support evolution. If you have tests we can automatically rewrite deprecated methods and we will use this more often. We finally start to have: - a better compiler and not an ancient one which reported error as Morph. - a better architecture, more modular - a real bootstrap (we should improve it and build tools to support it - we are working on it) - strong libraries (File, Stream, HTTP) more documented and tested - better tools (Iceberg and Calypso are definitively steps in the right direction) And we will continue to improve. We will iterate on all these to make Pharo even better. We are writing more tests to support those changes. I repeat it: I think that we are doing a pretty good job about the quality that we deliver. Stef > On 11 Jan 2019, at 18:54, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Hi Stef, > >> On Jan 10, 2019, at 7:59 AM, ducasse <stepharo(a)netcourrier.com> wrote: >> >> Eliot I would like also two points to this. >> >> - First we asked thomas to write tests about the debugger model and you see if they would be tests about methods we could understand >> that they are used and control what they do. So we should all thank thomas for his energy in this not so easy task. >> >> - Second it would be nice if you could refrain to be systematically negative about what we are doing. I think that our development process >> is much better than many others :) It is not perfect because this does not exist. >> I think that we are doing a great job make Smalltalk cool. And yes it may happen that one untested, undocumented method >> get lost. I think that we are doing pretty good given the resources we have. > > Even more serious an issue for the Pharo community than a development process which fails to support the Nedâs of users is a defensive attitude that does not want to discuss serious issues maturely. I bring up the stability and backward-portability issue because it is *important*; it has affected ClÃ©mentâs ability to deliver Sista and my and feenkâs efforts to support VM development on Pharo. If your response to my trying to discuss seriously and objectively a problem that needs discussion is always to say âplease donât be negativeâ I have even less confidence that Pharo can be a realistic platform for my work and the work of others. > > >> Stef >> >>> On 10 Jan 2019, at 15:11, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >>> >>> Hi Thomas, >>> >>>> On Jan 10, 2019, at 2:24 AM, Thomas Dupriez via Pharo-dev <pharo-dev(a)lists.pharo.org> wrote: >>>> >>>> <mime-attachment> >>> >>> in a stack of contexts the active pc is different for the top context. For other than the top context, a contextâs pc will be pointing after the send that created the context above it, so to find the pc of the send one finds the previous pc. For the top context its pc is the active pc. >>> >>> Typically the debugger is invoked in two different modes, interruption or exception. When interrupted, a process is stopped at the next suspension point (method entry or backward branch) and the top context in the process is the context to be displayed in the debugger. When an exception occurs the exception search machinery will find the signaling context, the context that raised the exception, which will be below the search machinery and the debugger invocation above that. The active pc of the signaling context will be the of for the send of digbsl et al. >>> >>> So the distinction is important and the utility method is probably useful. >>> >>> Do you want to remove the method simply because there are no senders in the image? >>> >>> If so, this is indicative of a serious problem with the Pharo development process. In the summer I ported VMMaker.oscog to Pharo 6. Now as feenk try and build a VMMaker.oscog image on Pharo 7, the system is broken, in part because of depreciations and in part because useful methods (isOptimisedBlock (isOptimizedBlock?) in the Opal compiler) have been removed. >>> >>> Just because a method is not in the image does not imply it is not in use. It simply means that it is not in use in the base image. As the system gets modularised this issue will only increase. There are lots of collection methods that exist as a library that are not used in the base image and removing them would clearly damage the library for users. This is the case for lots of so-called system code. There are users out there, like those of us in the vm team, who rely on such system code, and it is extremely unsettling and frustrating to have that system code change all the time. If Pharo is to be a useful platform to the vm team it has to be more stable. >> >> >>

3 3