September 2014 - Pharo-users

Update of AgileVisualization.com

by Alexandre Bergel

Dear All, AgileVisualization.com , the book about Roassal, has been updated with a new chapter. The HTML versions of the chapter are also online. Agile Visualization is written using Pillar and Skeleton. Thanks Damien Cassou and Yuriy Tymchuk (Uko) for these wonderful frameworks. I am also looking for contributors. I know that some of you guys have done wonderful things with Roassal. Sharing your knowledge with the rest of us would be fantastic. You may even get paid to write a chapter :-) Get in touch with me for more info. Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Sept. 24, 2014

Re: [Pharo-users] Ridiculous we are

by Sven Van Caekenberghe

On 24 Sep 2014, at 19:09, Benjamin Pollack <benjamin(a)bitquabit.com> wrote: > On Wed, 24 Sep 2014 13:03:57 -0400, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> >> Did you read the actual conversation in the issue ? >> >> https://pharo.fogbugz.com/f/cases/14054/Issue-with-path-with-accented-chara… >> >> It has been renamed and there is a fix (as a change set, not as a slice, yet). Basically, there was a primitive call into a plugin that failed to do encoding. >> > > No, I apologize; I missed the bug link. Thanks for reposting it. > >> Now regarding the issues you raised. Pharo does not do Unicode canonicalisation or any of that other fancy stuff (like categorisation, proper ordering and so on). This is another orthogonal and way more general issue. >> >> Regarding the pathnames encoding: if the OS itself does not know it, how can we ? > > That's actually the argument *against* using UTF-8 as the standard Pharo way to represent filenames--at least on Unix systems. If Pharo used ByteArrays to represent paths, with convenience methods for working with UTF-8 (since I do agree that's the most likely thing for a user/dev to want), then you'd be able to work with all files no matter what, *and* have a convenient way of doing so for the common case. > > This is an old discussion, and I do see both sides of it. In terms of SCMs, Mercurial and Git both just say "it's a collection of bytes", whereas Subversion says "it's Unicode code points." This has some uncomfortable implications for both systems when working on multiple platforms. Benjamin, I think I understand the concern / situation that you describe. But I fail to see how not-interpreting it and interpreting it in different encodings can work in practice, especially since your point seems to be that there is no meta information that gives a definitive answer. I would guess that other languages, say Java or Python, have some approach to handle this problem ? Also, since we are living with the current approach without much problems, I think the issue is not terribly pressing. Sven

Sept. 24, 2014

Re: [Pharo-users] Ring package, when to use it?

by Marcus Denker

On 24 Sep 2014, at 18:35, Juraj Kubelka <juraj.kubelka(a)gmail.com> wrote: > Thank you Marcus for the explanation. > > So now I understand that if I want to analyse existing packages/class/methods/etc in the image, Ring is not a kind of interest. > > But as I think about it, if someone uses Ring as a base to analyse environment, then it could be useful to use the same analysis tool for any source, e.g. not loaded packages. Am I right or are the some limitations? > > That is the ideaâ¦ Ring Definition and normal classes/methods share a common API and can be used interchangeably. Marcus

Sept. 24, 2014

Re: [Pharo-users] Ridiculous we are

by Benjamin Pollack

On Wed, 24 Sep 2014 13:03:57 -0400, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Did you read the actual conversation in the issue ? > > https://pharo.fogbugz.com/f/cases/14054/Issue-with-path-with-accented-chara… > > It has been renamed and there is a fix (as a change set, not as a slice, > yet). Basically, there was a primitive call into a plugin that failed to > do encoding. > No, I apologize; I missed the bug link. Thanks for reposting it. > Now regarding the issues you raised. Pharo does not do Unicode > canonicalisation or any of that other fancy stuff (like categorisation, > proper ordering and so on). This is another orthogonal and way more > general issue. > > Regarding the pathnames encoding: if the OS itself does not know it, how > can we ? That's actually the argument *against* using UTF-8 as the standard Pharo way to represent filenames--at least on Unix systems. If Pharo used ByteArrays to represent paths, with convenience methods for working with UTF-8 (since I do agree that's the most likely thing for a user/dev to want), then you'd be able to work with all files no matter what, *and* have a convenient way of doing so for the common case. This is an old discussion, and I do see both sides of it. In terms of SCMs, Mercurial and Git both just say "it's a collection of bytes", whereas Subversion says "it's Unicode code points." This has some uncomfortable implications for both systems when working on multiple platforms. --Benjamin

Sept. 24, 2014

Re: [Pharo-users] Ridiculous we are

by Sven Van Caekenberghe

On 24 Sep 2014, at 18:48, Benjamin Pollack <benjamin(a)bitquabit.com> wrote: > On Tue, 23 Sep 2014 08:51:54 -0400, Hilaire <hilaire(a)drgeo.eu> wrote: > >> Le 23/09/2014 14:09, Damien Cassou a Ã©crit : >>> I recently read documents about utf-8 encoding. In all of them, the >>> author says that pathnames should be kept as is because you never know >>> which encoding the filesystem uses. So, a filename should probably be >>> a bytearray. >> >> >> yes, but a #Ã© should be encoded in two bytes. > > As noted in my previous message, "Ã©" could be represented as either one or two Unicode code points, and these in turn could validly be either two or three bytes in UTF-8. My gut says that $Ã© should be U+00E9, because otherwise you should have to use two Characters ($e and $Â´), but you could legitimately argue otherwise as well, and at any rate, #Ã© could definitely be either. This is likely the core of the issue you're hitting. Did you read the actual conversation in the issue ? https://pharo.fogbugz.com/f/cases/14054/Issue-with-path-with-accented-chara… It has been renamed and there is a fix (as a change set, not as a slice, yet). Basically, there was a primitive call into a plugin that failed to do encoding. Now regarding the issues you raised. Pharo does not do Unicode canonicalisation or any of that other fancy stuff (like categorisation, proper ordering and so on). This is another orthogonal and way more general issue. Regarding the pathnames encoding: if the OS itself does not know it, how can we ? I think that the current approach (assuming UTF-8) makes (the most) sense for a system that runs on multiple platforms. Sven

Sept. 24, 2014

Re: [Pharo-users] [ANN] I stop maintaining the Ubuntu packages

by volkert＠nivoba.de

Sad news. I switched from OSX to Ubuntu and i also plan to switch from Win8 to Ubuntu. I really hope "Pharo" will find a new maintainer for the most popular Linux Distribution. Thank you for your work ... BW, Volkert Am 24.09.2014 um 09:41 schrieb Damien Cassou: > Dear all, > > I've recently switched my Linux distribution from Ubuntu to NixOS. > This means I can no longer maintain the Ubuntu packages and their PPA > (https://launchpad.net/~pharo/) > > These packages need a maintainer! > > The good side of it is that it is not a lot of work as I did all the > automating infrastructure: > https://github.com/pharo-project/pharo-ubuntu. Basically, the > maintainer will have to launch a few shell scripts regularly (once a > month for example). I will obviously be available to help the new > maintainer. > > The community needs *you*. > > Good news is: I packaged the Pharo VM for the Nix package manager > which can be installed on many Unix (e.g., Linux, MacOS X, FreeBSD). > And NixOS will have the Pharo VM package in its next release in > October! > -- www.nivoba.de "The more complex an object, the larger the investment in learning to use it, and the greater the resistance to abandon it.", NW

Sept. 24, 2014

Re: [Pharo-users] Packaging Pharo for many distributions at once

by Damien Cassou

On Wed, Sep 24, 2014 at 12:47 PM, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > All non-native package managers have that issue. nix included -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill

Sept. 24, 2014

Re: [Pharo-users] Mac VM - developer unidentified

by Damien Cassou

On Wed, Sep 24, 2014 at 11:13 AM, Usman Bhatti <usman.bhatti(a)gmail.com> wrote: > for less knowledgable it can be a negative message. yes I agree. -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill

Sept. 24, 2014

Re: [Pharo-users] Ridiculous we are

by Benjamin Pollack

On Tue, 23 Sep 2014 08:51:54 -0400, Hilaire <hilaire(a)drgeo.eu> wrote: > Le 23/09/2014 14:09, Damien Cassou a Ã©crit : >> I recently read documents about utf-8 encoding. In all of them, the >> author says that pathnames should be kept as is because you never know >> which encoding the filesystem uses. So, a filename should probably be >> a bytearray. > > > yes, but a #Ã© should be encoded in two bytes. As noted in my previous message, "Ã©" could be represented as either one or two Unicode code points, and these in turn could validly be either two or three bytes in UTF-8. My gut says that $Ã© should be U+00E9, because otherwise you should have to use two Characters ($e and $Â´), but you could legitimately argue otherwise as well, and at any rate, #Ã© could definitely be either. This is likely the core of the issue you're hitting.

Sept. 24, 2014

Re: [Pharo-users] Ridiculous we are

by Benjamin Pollack

On Mon, 22 Sep 2014 17:58:41 -0400, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > I also find the way some problems are reported quite disturbing. How > much testing did you do ? On which platforms ? > > I can do this (in Pharo 3) without any problems (we're talking about > arbitrary Unicode characters in path names): > > ('/tmp' asFileReference / 'Ã©tÃ©') ensureCreateDirectory. > '/tmp/Ã©tÃ©' asFileReference exists. > ('/tmp/Ã©tÃ©' asFileReference / 'ÎÎ»Î»Î¬Î´Î±.txt') writeStreamDo: [ :out | > out << 'What about Greece ?' ]. > ('/tmp/Ã©tÃ©' asFileReference / 'ÎÎ»Î»Î¬Î´Î±.txt') exists. > ('/tmp/Ã©tÃ©' asFileReference / 'ÎÎ»Î»Î¬Î´Î±.txt') contents. > > And in a terminal, I get: > > $ ls /tmp/Ã©tÃ©/ÎÎ»Î»Î¬Î´Î±.txt > /tmp/Ã©tÃ©/ÎÎ»Î»Î¬Î´Î±.txt > > $ cat !$ > cat /tmp/Ã©tÃ©/ÎÎ»Î»Î¬Î´Î±.txt > What about Greece ? > > This is on Mac OS X. > > So this part fundamentally works in the image and on one VM. There might > of course be problems in how paths are used in certain places or on > certain VM/platforms. > Focusing purely on Unicode itself (not the encoding systems), a letter like Ã© can be represented as U+00E9 (LATIN SMALL LETTER E WITH ACUTE), or as U+0065 (LATIN SMALL LETTER E) followed by U+0301 (combining acute accent). These will appear identical to the user, but are emphatically *not* identical for most software. The way you're testing here, you will not hit any error relating to this concept, ever, because you're using Pharo for both generating and consuming the strings. At the very least, we'd need to generate a file named "Ã©tÃ©" with both forms explicitly and see what happens. Things get even more exciting, though, because Unix says that file names are simply arbitrary byte patterns that do not contain the null byte.* Thus, you can trivially create a file named "Ã©tÃ©" using Latin-1 encoding, and again using UTF-8 encoding, and again using UTF-7 encoding, and these might all be shown to the user as "identically" named, but I guarantee you that Pharo will not act sanely with all four of these. Even on Windows, where things are a bit saner (NTFS mandates UTF-16), and where an explicit normalization form is preferred (NFC), I just explicitly verified that I can trivially inject other normalization forms into the file system. Thus, you can still have two files named "Ã©tÃ©" that nevertheless have different names as far as the OS is concerned. In this case, as far as I can tell, Pharo assumes that all path names are Unicode, and does not do any work to convert strings to or from the various normalization schemes (looking in Path class>>canonicalizeElements:, Path class>>from:delimiter, and FileSystemStore>>pathFromString: here). There's therefore a pretty straightforward fix that Pharo could do: 1. Path would use ByteArrays as the actual canonical store, and provide convenience methods to see what the array decodes to in various encodings. The developer and application can make decisions about what encoding system they want to use. 2. The VM likely needs to be modified to handle this (didn't check) As much as I wish Hilaire provided more details in his bug report, it's worth keeping in mind that not all users, or even all programmers, understand the full implications of things like how various Unicode normalization and encoding schemes interact in practice with Unix's very vague concept of what a file name actually is, so I usually try to approach these bug reports carefully and with an open mind. --Benjamin * On OS X, HFS+ uses UTF-16 with an Apple-specific variant of NFD, whereas I do not believe this holds for e.g. UFS or FUSE-backed file systems, so things are a bit subtler there, but the general rule holds.

Sept. 24, 2014