May 2012 - Pharo-dev - lists.pharo.org

[Pharo-project] [update 2.0] #20060

by Esteban Lorenzano

20060 ----- Issue 5846: register correct message browser http://code.google.com/p/pharo/issues/detail?id=5846 Issue 5845: MCRepositoryGroup>>removeHTTPRepositoryLocationNamed: bad category http://code.google.com/p/pharo/issues/detail?id=5845

May 8, 2012

Re: [Pharo-project] Unicode/Latin1 handling in Pharo 1.3

by Henrik Sperre Johansen

On 08.05.2012 14:00, Nicolas Cellier wrote: > leadingChar was here merely for handling Han unification > http://en.wikipedia.org/wiki/Han_unification. > As I understand it, the language variant is encoded in leadingChar, > while the generic ideogram is encoded in the unicode value. From what I can tell, it (based on the indexes) initially also allowed for non-unicode WideStrings (see Character asUnicode, and it's use of EncodedCharSet). That was probably a bad idea in the first place since there is no 16-bit string type in Pharo, I guess adding that would be the original thought behind it... Later on it was changed to do Han unification, but: - The code for selecting/displaying font based on leadingChar is a mess, never tested, and no compatible font come installed in Pharo by default. - There is no code to, say, automatically add leadingChars based on current environment to character events, so even if it works, input might still display incorrectly. - Using it in Translation support requires storing in a special format which includes the leading characters. - The LanguageEnvironments that currently deal with this also have other roles, which is purely empirical and, in some cases, outdated (I'm looking at you, defaultSystemConverter) > We tried to clean a lot already and avoid using leadingChar, except > maybe for east-asian languages, so in case of umlaut, I would classify > this as a bug. > > Nicolas Yeah, that's clearly something gone entirely wrong when the translation table was initialized. 17 has never been a valid leadingChar. FWIW, it was reinitialized and works as expected in 1.4. TLDR of non-bug related stuff; If it were up to me, I'd say nuke leadingChar from orbit (which, as you say, is already done in most cases), then find some other way than what is currently done in LanguageEnvironment (if possible) to query the correct encoding of strings you pass to the system. Cheers, Henry

May 8, 2012

Re: [Pharo-project] Unicode/Latin1 handling in Pharo 1.3

by Norbert Hartl

Can you paste the code you used to come to this result? Norbert Am 08.05.2012 um 14:46 schrieb Torsten Bergmann: > Sven wrote >> On my Mac I get: >> ... >> (stream contents at: 6) codePoint 71303420 >> ('TeilrÃ¼ckzahlung' at: 6) codePoint 252 > > Hi Sven, > > on Windows I get: > > (stream contents at: 6) codePoint 252 > ('TeilrÃ¼ckzahlung' at: 6) codePoint 252 > > For > > $Ã¼ codePoint I also get 252 > > So on Windows (at least for my test) it does > not have the "leadingChar". > > Bye > T. > > -- > Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir > belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de >

May 8, 2012

Re: [Pharo-project] Improved version of StartupPreferences

by Mariano Martinez Peck

On Tue, May 8, 2012 at 3:03 PM, Sean P. DeNigris <sean(a)clipperadams.com>wrote: > > Mariano Martinez Peck wrote > > > > /Users/mariano/.config/pharo/general you have a script that does: > > > > Should be "/Users/mariano/.pharo/config/general", no? > No, the directory is actually /Users/mariano/.config/pharo/general Of course it can be changed if that's what people want. I have no idea, StartupPreference was like this before I touch it ;) It seems in linux they are stored that way so to avoid lots of .xxx in the $HOME. So you use one directory for all your conf ;) (thanks Pavel) Cheers > > -- > View this message in context: > http://forum.world.st/Improved-version-of-StartupPreferences-tp4615713p4617… > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. > > -- Mariano http://marianopeck.wordpress.com

May 8, 2012

Re: [Pharo-project] Improved version of StartupPreferences

by Sean P. DeNigris

Mariano Martinez Peck wrote > > /Users/mariano/.config/pharo/general you have a script that does: > Should be "/Users/mariano/.pharo/config/general", no? -- View this message in context: http://forum.world.st/Improved-version-of-StartupPreferences-tp4615713p4617… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

May 8, 2012

[Pharo-project] Unicode/Latin1 handling in Pharo 1.3

by Torsten Bergmann

Sven wrote >On my Mac I get: >... >(stream contents at: 6) codePoint 71303420 >('TeilrÃ¼ckzahlung' at: 6) codePoint 252 Hi Sven, on Windows I get: (stream contents at: 6) codePoint 252 ('TeilrÃ¼ckzahlung' at: 6) codePoint 252 For $Ã¼ codePoint I also get 252 So on Windows (at least for my test) it does not have the "leadingChar". Bye T. -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

May 8, 2012

Re: [Pharo-project] Unicode/Latin1 handling in Pharo 1.3

by Norbert Hartl

Am 08.05.2012 um 14:00 schrieb Nicolas Cellier: > leadingChar was here merely for handling Han unification > http://en.wikipedia.org/wiki/Han_unification. > As I understand it, the language variant is encoded in leadingChar, > while the generic ideogram is encoded in the unicode value. > We tried to clean a lot already and avoid using leadingChar, except > maybe for east-asian languages, so in case of umlaut, I would classify > this as a bug. > Ok, so there is a real tension to get rid of leadingChar. Would be good then to have Character>>#= using charCode instead of asciiValue? Norbert > 2012/5/8 Norbert Hartl <norbert(a)hartl.name>: >> >> Am 08.05.2012 um 11:41 schrieb Sven Van Caekenberghe: >> >>> Hi Holger, >>> >>> -- >>> Sven Van Caekenberghe >>> http://stfx.eu >>> Smalltalk is the Red Pill >>> >>> >>> >>> >>> On 08 May 2012, at 10:57, Holger Hans Peter Freyther wrote: >>> >>>> Hi, >>>> >>>> I am trying to read a file from disk that is in latin1 encoding and then try >>>> to compare the string with a string provided as literal and it fails. My test >>>> case can be seen below. I am using Pharo 1.3 on a Linux machine with a UTF-8 >>>> locale. Is there something obvious that I am doing wrong? >>>> >>>> >>>> | stream text | >>>> stream := (FileStream fileNamed: 'pharo_example_latin1.txt') >>>> converter: ISO885915TextConverter new; >>>> yourself. >>>> text := stream contents. >>>> text = 'TeilrÃ¼ckzahlung' >>>> >>>> <pharo_example_latin1.txt> >>> >>> This should work: >>> >>> | stream text | >>> stream := (FileStream fileNamed: '/Users/sven/Desktop/pharo_example_latin1.txt') >>> converter: Latin1TextConverter new; >>> yourself. >>> text := stream contents. >>> text = 'TeilrÃ¼ckzahlung'. >>> >>> or this (latest Zn code): >>> >>> | stream text | >>> stream := (FileStream fileNamed: '/Users/sven/Desktop/pharo_example_latin1.txt') >>> binary; >>> yourself. >>> text := (ZnCharacterEncoder newForEncoding: 'iso-8859-15') >>> decodeBytes: stream contents. >>> text = 'TeilrÃ¼ckzahlung'. >>> >>> there seems to be an issue with ISO885915TextConverter, check the umlaut encoding: it adds something called the leadingChar which I don't think is needed (I don't know why it even exists). >>> >> Yes, characters are mapped to characters with a leadingChar. Furthermore in Character the leadingChar value alters the output of asInteger, asciiValue, codePoint... because they all use the value of the character that includes leadingChar.But Latin1TextConverter doesn't do it so you get false for >> >> (ISO885915TextConverter new byteToUnicode: $Ã¼) = (Latin1TextConverter new byteToUnicode: $Ã¼) >> >> Because >> >> Character>>#= aCharacter >> "Primitive. Answer true if the receiver and the argument are the same >> object (have the same object pointer) and false otherwise. Optional. See >> Object documentation whatIsAPrimitive." >> >> ^ self == aCharacter or:[ >> aCharacter isCharacter and: [self asciiValue = aCharacter asciiValue]] >> >> is using asciiValue containing in one case 252 and in the other 71303420 (252 + leadingChar) >> >> So leadingChar can be considered harmful. It surely was useful in the 8bit days (that followed the 7bit days). But I personally cannot see why the encoding of character should persist if a character has been created. The approach with leadingChar is keeping the 128 >= asciiValue >= 255 and add the leadingChar for the encoding (comparable to code pages). Another approach would be to have a homogenous encoding inside the image. So a value of a character is always unicode (The 8bit trouble is solved because 8bit now is latin1). There would be only be unicode characters without encoding. And encoders that produce individual encodings. >> The "westeners first" approach then leads to a situation that everyone not living in the latin1 zone (or good ol' 7bit US) has to live with wide strings. >> >> This should be changed but I can see that's no easy task because the leadingChar is used throughout the system trying to help in collation things. It would be a bigger effort to clean this up. >> >> Norbert >> >> >> >

May 8, 2012

Re: [Pharo-project] A comparative article (was Re: [squeak-dev] [ANN] STON - Smalltalk Object Notation)

by Sven Van Caekenberghe

(The paper was indeed just updated a couple of days ago). On 08 May 2012, at 13:29, GÃ¶ran Krampe wrote: > - In general I believe in 99% of the cases the parsing system has to know what it is parsing, how the JSON looks and how it should be mapped onto objects. Making the JSON parser itsy bitsy smarter with type annotations doesn't help me there, I still need to know that I want to instantiate a PDFPage and put this Rectangle into it - but oh, perhaps not as a Rectangle, but perhaps as two points sent into a class side instantiation message or something. > > My point being that the type annotations doesn't "buy me" enough. (Just for the sake of the argument, I don't want to convince you) STON *does* buy you that: as long as we are talking about 'domain level objects' (a vague notion I agree) your Smalltalk object will serialize without any extra effort (see the ZnResponse example in the paper). You can customize some objects if you want to (mostly for readability, sometimes to fix some issue). Object implements #toSton: and #fromSton: by iterating over the instances variables. The issues are: Blocks, Classes and the fact that ephemeral instances are always serialized. Sven

May 8, 2012

Re: [Pharo-project] Unicode/Latin1 handling in Pharo 1.3

by Nicolas Cellier

leadingChar was here merely for handling Han unification http://en.wikipedia.org/wiki/Han_unification. As I understand it, the language variant is encoded in leadingChar, while the generic ideogram is encoded in the unicode value. We tried to clean a lot already and avoid using leadingChar, except maybe for east-asian languages, so in case of umlaut, I would classify this as a bug. Nicolas 2012/5/8 Norbert Hartl <norbert(a)hartl.name>: > > Am 08.05.2012 um 11:41 schrieb Sven Van Caekenberghe: > >> Hi Holger, >> >> -- >> Sven Van Caekenberghe >> http://stfx.eu >> Smalltalk is the Red Pill >> >> >> >> >> On 08 May 2012, at 10:57, Holger Hans Peter Freyther wrote: >> >>> Hi, >>> >>> I am trying to read a file from disk that is in latin1 encoding and then try >>> to compare the string with a string provided as literal and it fails. My test >>> case can be seen below. I am using Pharo 1.3 on a Linux machine with a UTF-8 >>> locale. Is there something obvious that I am doing wrong? >>> >>> >>> | stream text | >>> stream := (FileStream fileNamed: 'pharo_example_latin1.txt') >>> Â Â Â Â Â Â Â Â Â Â Â converter: ISO885915TextConverter new; >>> Â Â Â Â Â Â Â Â Â Â Â yourself. >>> text := stream contents. >>> text = 'TeilrÃ¼ckzahlung' >>> >>> <pharo_example_latin1.txt> >> >> This should work: >> >> | stream text | >> stream := (FileStream fileNamed: '/Users/sven/Desktop/pharo_example_latin1.txt') >> Â Â Â Â Â Â Â Â Â Â Â converter: Latin1TextConverter Â new; >> Â Â Â Â Â Â Â Â Â Â Â yourself. >> text := stream contents. >> text = 'TeilrÃ¼ckzahlung'. >> >> or this (latest Zn code): >> >> | stream text | >> stream := (FileStream fileNamed: '/Users/sven/Desktop/pharo_example_latin1.txt') >> Â Â Â Â Â Â Â Â Â Â Â binary; >> Â Â Â Â Â Â Â Â Â Â Â yourself. >> text := (ZnCharacterEncoder newForEncoding: 'iso-8859-15') >> Â Â Â Â Â Â Â decodeBytes: stream contents. >> text = 'TeilrÃ¼ckzahlung'. >> >> there seems to be an issue with ISO885915TextConverter, check the umlaut encoding: it adds something called the leadingChar which I don't think is needed (I don't know why it even exists). >> > Yes, characters are mapped to characters with a leadingChar. Furthermore in Character the leadingChar value alters the output of asInteger, asciiValue, codePoint... because they all use the value of the character that includes leadingChar.But Latin1TextConverter doesn't do it so you get false for > > (ISO885915TextConverter new byteToUnicode: $Ã¼) = (Latin1TextConverter new byteToUnicode: $Ã¼) > > Because > > Character>>#= aCharacter > Â Â Â Â "Primitive. Answer true if the receiver and the argument are the same > Â Â Â Â object (have the same object pointer) and false otherwise. Optional. See > Â Â Â Â Object documentation whatIsAPrimitive." > > Â Â Â Â ^ self == aCharacter or:[ > Â Â Â Â Â Â Â Â aCharacter isCharacter and: [self asciiValue = aCharacter asciiValue]] > > is using asciiValue containing in one case 252 and in the other 71303420 (252 + leadingChar) > > So leadingChar can be considered harmful. It surely was useful in the 8bit days (that followed the 7bit days). But I personally cannot see why the encoding of character should persist if a character has been created. The approach with leadingChar is keeping the 128 >= asciiValue >= 255 and add the leadingChar for the encoding (comparable to code pages). Another approach would be to have a homogenous encoding inside the image. So a value of a character is always unicode (The 8bit trouble is solved because 8bit now is latin1). There would be only be unicode characters without encoding. And encoders that produce individual encodings. > The "westeners first" approach then leads to a situation that everyone not living in the latin1 zone (or good ol' 7bit US) has to live with wide strings. > > This should be changed but I can see that's no easy task because the leadingChar is used throughout the system trying to help in collation things. It would be a bigger effort to clean this up. > > Norbert > > >

May 8, 2012

Re: [Pharo-project] Running in background with OSProcess

by David T. Lewis

On Mon, May 07, 2012 at 10:05:12PM -0700, Sean P. DeNigris wrote: > > David T. Lewis wrote > > > > ProxyPipeline command: '/bin/sleep 10' > > This gives you exactly what you expect to see. > > > > Okay, it worked! However, I'm confused by the string escaping. Because I'm > on a Mac, and only later discovered that meant I'm on Unix, I have lots of > paths with spaces and even [gasp] apostrophes in them (the second makes > Smalltalk strings a lot of fun). One thing that I loved about OSProcess is > that I felt like I had more control over strings (e.g. paths) in commands > because I could construct my paths in Smalltalk and then just double-quote > the final product. > > However, using ProxyPipeline, it seems that CommandShell has layered on its > own escaping rules that are different from Bash's. So, although I can write: > p := PipeableOSProcess command: '/usr/bin/java > -DJENKINS_HOME="/path/with/a > space/and/an/apostrophe/isn''t/fun/Pharo-1.4/.jenkins"...' > > with ProxyPipeline, I apparently have to write: > p2 := ProxyPipeline command: '/usr/bin/java -DJENKINS_HOME=/path/with/a\ > space/and/an/apostrophe/isn\''t/fun/Pharo-1.4/.jenkins...' > > I.e. "enclose in double-quotes" vs. "escape single-quotes and spaces with > preceding backslash" > > Can I somehow circumvent these extra rules so I don't have to adapt all my > commands? What is their purpose? > > Getting closer and closer to having this work... Hi Sean, I'm afraid that the simulation of unix shell behavior is far from perfect, and many things that you might expect from a real shell (notably including shell variables and some of the substitution behaviour you are experiencing here) are either missing from CommandShell, or behave differently for various reasons. In this case you are seeing a mashup of Smalltalk string escaping in a unix-like command shell. You are first composing a string in Smalltalk, so the "enclose in double-quotes" rules apply. Then you are (quite reasonably) expected that preceeding-backslash quoting should work in a unix shell, but it does not work here because I never put that into the shell syntax processing. Yes, this is confusing. No, there is no easy workaround or fix. I'm afraid that doing a complete and faithful reproduction of bash would be a fool's errand, just way too much work to be worth the trouble. Adding backslash escaping would not be hard to do but ... well, when it comes to adding features, where do you stop? ;) BTW, I would welcome patches to improve things like this, as long as they are supported with unit tests. Dave

May 8, 2012