December 2015 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Unicode Support

by EuanM

"Well. I do not agree with this. I agree with the quote. Can you explain a bit more about what you mean by abstract character and concept?" Guillermo The problem with the quote is, that *while true*, it *does not disambiguate* between: either compatibility character and abstract character; or character as composable component of an abstract character and character as the entire embodiment of an abstract character; Abstract character is the key concept of Unicode. Differentiation between abstract character and codepoints is the key differentiator of the Unicode approach and most previous approaches to character encoding, e,g, ASCII, EBCDIC, ISO Latin 1, etc Please see my previous posts which use the example of Angstrom, Capital A with circle (or whatever the canonical name is) and the composed sequence of "Capital A" and "circle above a letter" for a fuller explanation of the concept of "abstract character". On 9 December 2015 at 09:35, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > >> On 8 dic 2015, at 10:07 p.m., EuanM <euanmee(a)gmail.com> wrote: >> >> "No. a codepoint is the numerical value assigned to a character. An >> "encoded character" is the way a codepoint is represented in bytes >> using a given encoding." >> >> No. >> >> A codepoint may represent a component part of an abstract character, >> or may represent an abstract character, or it may do both (but not >> always at the same time). >> >> Codepoints represent a single encoding of a single concept. >> >> Sometimes that concept represents a whole abstract character. >> Sometimes it represent part of an abstract character. > > Well. I do not agree with this. I agree with the quote. > > Can you explain a bit more about what you mean by abstract character and concept? > >> >> This is the key difference between Unicode and most character encodings. >> >> A codepoint does not always represent a whole character. >> >> On 7 December 2015 at 13:06, Henrik Johansen >> <henrik.s.johansen(a)veloxit.no> wrote: >>> >>> On 07 Dec 2015, at 1:05 , EuanM <euanmee(a)gmail.com> wrote: >>> >>> Hi Henry, >>> >>> To be honest, at some point I'm going to long for the for the much >>> more succinct semantics of healthcare systems and sports scoring and >>> administration systems again. :-) >>> >>> codepoints are any of *either* >>> - the representation of a component of an abstract character, *or* >>> eg. "A" #(0041) as a component of >>> - the sole representation of the whole of an abstract character *or* of >>> - a representation of an abstract character provided for backwards >>> compatibility which is more properly represented by a series of >>> codepoints representing a composed character >>> >>> e.g. >>> >>> The "A" #(0041) as a codepoint can be: >>> the sole representation of the whole of an abstract character "A" #(0041) >>> >>> The representation of a component of the composed (i.e. preferred) >>> version of the abstract character â« #(0041 030a) >>> >>> â« (#00C5) represents one valid compatibility form of the abstract >>> character â« which is most properly represented by #(0041 030a). >>> >>> â« (#212b) also represents one valid compatibility form of the abstract >>> character â« which is most properly represented by #(0041 030a). >>> >>> With any luck, this satisfies both our semantic understandings of the >>> concept of "codepoint" >>> >>> Would you agree with that? >>> >>> In Unicode, codepoints are *NOT* an abstract numerical representation >>> of a text character. >>> >>> At least not as we generally understand the term "text character" from >>> our experience of non-Unicode character mappings. >>> >>> >>> I agree, they are numerical representations of what Unicode refers to as >>> characters. >>> >>> >>> codepoints represent "*encoded characters*" >>> >>> >>> No. a codepoint is the numerical value assigned to a character. An "encoded >>> character" is the way a codepoint is represented in bytes using a given >>> encoding. >>> >>> and "a *text element* ... >>> is represented by a sequence of one or more codepoints". (And the >>> term "text element" is deliberately left undefined in the Unicode >>> standard) >>> >>> Individual codepoints are very often *not* the encoded form of an >>> abstract character that we are interested in. Unless we are >>> communicating to or from another system (Which in some cases is the >>> Smalltalk ByteString class) >>> >>> >>> >>> >>> i.e. in other words >>> >>> *Some* individual codepoints *may* be a representation of a specific >>> *abstract character*, but only in special cases. >>> >>> The general case in Unicode is that Unicode defines (a) >>> representation(s) of a Unicode *abstract character*. >>> >>> The Unicode standard representation of an abstract character is a >>> composed sequence of codepoints, where in some cases that sequence is >>> as short as 1 codepoint. >>> >>> In other cases, Unicode has a compatibility alias of a single >>> codepoint which is *also* a representation of an abstract character >>> >>> There are some cases where an abstract character can be represented by >>> more than one single-codepoint compatibility codepoint. >>> >>> Cheers, >>> Euan >>> >>> >>> I agree you have a good grasp of the distinction between an abstract >>> character (characters and character sequences which should be treated >>> equivalent wrt, equality / sorting / display, etc.) and a character (which >>> each have a code point assigned). >>> That is besides the point both Sven and I tried to get through, which is the >>> difference between a code point and the encoded form(s) of said code point. >>> When you write: >>> "and therefore encodable in UTF-8 as compatibility codepoint e9 hex >>> and as the composed character #(0065 00b4) (all in hex) and as the >>> same composed character as both >>> #(feff 0065 00b4) and #(ffef 0065 00b4) when endianness markers are >>> included" >>> >>> I's quite clear you confuse the two. 0xFEFF is the codepoint of the >>> character used as bom. >>> When you state that it can be written ffef (I assume you meant FFFE), you >>> are again confusing the code point and its encoded value (an encoded value >>> which only occurs in UTF16/32, no less). >>> >>> When this distinction is clear, it might be easier to see that value in that >>> Strings are kept as Unicode code points arrays, and converted to encoded >>> forms when entering/exiting the system. >>> >>> Cheers, >>> Henry >>> >> > >

Dec. 9, 2015

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/50492 Home: https://github.com/pharo-project/pharo-core

Dec. 9, 2015

[pharo-project/pharo-core] e3f6b3: 50492

by GitHub

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: e3f6b3f61fc8f36d306bbd3d8e0f125a369528ba https://github.com/pharo-project/pharo-core/commit/e3f6b3f61fc8f36d306bbd3d… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-09 (Wed, 09 Dec 2015) Changed paths: R Morphic-Core.package/Morph.class/instance/stepping and presenter/startSteppingIn_.st R Morphic-Core.package/PasteUpMorph.class/instance/world state/addMorphs_.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50491.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50492.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50491.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50492.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 50492 17221 clean dirty packages after metacello update https://pharo.fogbugz.com/f/cases/17221 17219 Class Definitions not in Changes... again again https://pharo.fogbugz.com/f/cases/17219 17217 Remove PasteUpMorph>>#addMorphs: https://pharo.fogbugz.com/f/cases/17217 http://files.pharo.org/image/50/50492.zip

Dec. 9, 2015

Re: [Pharo-dev] Unicode Support

by H. Hirzel

See example with ANGSTROM Abstract Characters (Unicode) http://wiki.squeak.org/squeak/6256 On 12/9/15, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > >> On 8 dic 2015, at 10:07 p.m., EuanM <euanmee(a)gmail.com> wrote: >> >> "No. a codepoint is the numerical value assigned to a character. An >> "encoded character" is the way a codepoint is represented in bytes >> using a given encoding." >> >> No. >> >> A codepoint may represent a component part of an abstract character, >> or may represent an abstract character, or it may do both (but not >> always at the same time). >> >> Codepoints represent a single encoding of a single concept. >> >> Sometimes that concept represents a whole abstract character. >> Sometimes it represent part of an abstract character. > > Well. I do not agree with this. I agree with the quote. > > Can you explain a bit more about what you mean by abstract character and > concept? > >> >> This is the key difference between Unicode and most character encodings. >> >> A codepoint does not always represent a whole character. >> >> On 7 December 2015 at 13:06, Henrik Johansen >> <henrik.s.johansen(a)veloxit.no> wrote: >>> >>> On 07 Dec 2015, at 1:05 , EuanM <euanmee(a)gmail.com> wrote: >>> >>> Hi Henry, >>> >>> To be honest, at some point I'm going to long for the for the much >>> more succinct semantics of healthcare systems and sports scoring and >>> administration systems again. :-) >>> >>> codepoints are any of *either* >>> - the representation of a component of an abstract character, *or* >>> eg. "A" #(0041) as a component of >>> - the sole representation of the whole of an abstract character *or* of >>> - a representation of an abstract character provided for backwards >>> compatibility which is more properly represented by a series of >>> codepoints representing a composed character >>> >>> e.g. >>> >>> The "A" #(0041) as a codepoint can be: >>> the sole representation of the whole of an abstract character "A" >>> #(0041) >>> >>> The representation of a component of the composed (i.e. preferred) >>> version of the abstract character â« #(0041 030a) >>> >>> â« (#00C5) represents one valid compatibility form of the abstract >>> character â« which is most properly represented by #(0041 030a). >>> >>> â« (#212b) also represents one valid compatibility form of the abstract >>> character â« which is most properly represented by #(0041 030a). >>> >>> With any luck, this satisfies both our semantic understandings of the >>> concept of "codepoint" >>> >>> Would you agree with that? >>> >>> In Unicode, codepoints are *NOT* an abstract numerical representation >>> of a text character. >>> >>> At least not as we generally understand the term "text character" from >>> our experience of non-Unicode character mappings. >>> >>> >>> I agree, they are numerical representations of what Unicode refers to as >>> characters. >>> >>> >>> codepoints represent "*encoded characters*" >>> >>> >>> No. a codepoint is the numerical value assigned to a character. An >>> "encoded >>> character" is the way a codepoint is represented in bytes using a given >>> encoding. >>> >>> and "a *text element* ... >>> is represented by a sequence of one or more codepoints". (And the >>> term "text element" is deliberately left undefined in the Unicode >>> standard) >>> >>> Individual codepoints are very often *not* the encoded form of an >>> abstract character that we are interested in. Unless we are >>> communicating to or from another system (Which in some cases is the >>> Smalltalk ByteString class) >>> >>> >>> >>> >>> i.e. in other words >>> >>> *Some* individual codepoints *may* be a representation of a specific >>> *abstract character*, but only in special cases. >>> >>> The general case in Unicode is that Unicode defines (a) >>> representation(s) of a Unicode *abstract character*. >>> >>> The Unicode standard representation of an abstract character is a >>> composed sequence of codepoints, where in some cases that sequence is >>> as short as 1 codepoint. >>> >>> In other cases, Unicode has a compatibility alias of a single >>> codepoint which is *also* a representation of an abstract character >>> >>> There are some cases where an abstract character can be represented by >>> more than one single-codepoint compatibility codepoint. >>> >>> Cheers, >>> Euan >>> >>> >>> I agree you have a good grasp of the distinction between an abstract >>> character (characters and character sequences which should be treated >>> equivalent wrt, equality / sorting / display, etc.) and a character >>> (which >>> each have a code point assigned). >>> That is besides the point both Sven and I tried to get through, which is >>> the >>> difference between a code point and the encoded form(s) of said code >>> point. >>> When you write: >>> "and therefore encodable in UTF-8 as compatibility codepoint e9 hex >>> and as the composed character #(0065 00b4) (all in hex) and as the >>> same composed character as both >>> #(feff 0065 00b4) and #(ffef 0065 00b4) when endianness markers are >>> included" >>> >>> I's quite clear you confuse the two. 0xFEFF is the codepoint of the >>> character used as bom. >>> When you state that it can be written ffef (I assume you meant FFFE), >>> you >>> are again confusing the code point and its encoded value (an encoded >>> value >>> which only occurs in UTF16/32, no less). >>> >>> When this distinction is clear, it might be easier to see that value in >>> that >>> Strings are kept as Unicode code points arrays, and converted to encoded >>> forms when entering/exiting the system. >>> >>> Cheers, >>> Henry >>> >> > > >

Dec. 9, 2015

Re: [Pharo-dev] where is the AST?

by Marcus Denker

> On 09 Dec 2015, at 12:33, Ben Coman <btc(a)openinworld.com> wrote: > > #+ implementors first inspect for now the easiest is to get the compiledMethod: #+ implementors first compiledMethod inspect Marcus

Dec. 9, 2015

where is the AST?

by Ben Coman

I am probably misremembering, but I thought I used to be able to do something like... #+ implementors first inspect and spotter would show a CompiledMethod with a tab showing the Bytecode and AST, but I get a RGMethodDefinition that doesn't have those. How do I get those tabs up? This is just for a two minute example for demonstration purposes. cheers -ben

Dec. 9, 2015

Re: [Pharo-dev] Help needed: old issues issue tracker entries

by Ben Coman

On Wed, Dec 9, 2015 at 9:35 AM, EuanM <euanmee(a)gmail.com> wrote: > Excellent. Can I suggest that in future they are simply closed, rather > than re-assigned? The reassignment happens automatically in the background whenever an issue is "resolved". Its a part of Fogbugz we have no control over. cheers -ben > On 3 December 2015 at 06:35, Nicolai Hess <nicolaihess(a)gmail.com> wrote: >> >> Am 03.12.2015 1:08 vorm. schrieb "EuanM" <euanmee(a)gmail.com>: >>> >>> I had an issue assigned against me, which I have not the time nor the >>> knowledge to fix. >>> >>> (Inconstant aspect ratio of Desktop background image). >>> >>> Can I suggest it is given to someone who knows the Desktop GUI codebase? >>> >>> (If I was in a position to fix it, I would have just submitted a fix, >>> not a bug report) >> >> It was assigned to "everyone". And just assigned to you after a fix was >> provided. >> That issue is already closed. >> >>> >>> On 2 December 2015 at 13:42, webwarrior <reg(a)webwarrior.ws> wrote: >>> > Case 16725 - Merge changes from spec github repository. >>> > 2 subcases are resolved, 2 other need decision. Nothing time-consuming - >>> > just decide if we integrate them or mark as wontfix. >>> > >>> > >>> > Case 15927 - Menu is broken when contained in a list. >>> > This is much more complicated, it's even not clear which class >>> > (TransfromMorph or MenuItemMorph) is bugged. >>> > I tried to fix it myself, but had no success. >>> > >>> > >>> > >>> > -- >>> > View this message in context: >>> > http://forum.world.st/Help-needed-old-issues-issue-tracker-entries-tp486426… >>> > Sent from the Pharo Smalltalk Developers mailing list archive at >>> > Nabble.com. >>> > >>> >

Dec. 9, 2015

Re: [Pharo-dev] Unicode Support

by Sven Van Caekenberghe

> On 09 Dec 2015, at 10:35, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > >> On 8 dic 2015, at 10:07 p.m., EuanM <euanmee(a)gmail.com> wrote: >> >> "No. a codepoint is the numerical value assigned to a character. An >> "encoded character" is the way a codepoint is represented in bytes >> using a given encoding." >> >> No. >> >> A codepoint may represent a component part of an abstract character, >> or may represent an abstract character, or it may do both (but not >> always at the same time). >> >> Codepoints represent a single encoding of a single concept. >> >> Sometimes that concept represents a whole abstract character. >> Sometimes it represent part of an abstract character. > > Well. I do not agree with this. I agree with the quote. > > Can you explain a bit more about what you mean by abstract character and concept? I am pretty sure that this whole discussion does more harm than good for most people's understanding of Unicode. It is best and (mostly) correct to think of a Unicode string as a sequence of Unicode characters, each defined/identified by a code point (out of 10.000s covering all languages). That is what we have today in Pharo (with the distinction between ByteString and WideString as mostly invisible implementation details). To encode Unicode for external representation as bytes, we use UTF-8 like the rest of the modern world. So far, so good. Why all the confusion ? Because the world is a complex place and the Unicode standard tries to cover all possible things. Citing all these exceptions and special cases will make people crazy and give up. I am sure that most stopped reading this thread. Why then is there confusion about the seemingly simple concept of a character ? Because Unicode allows different ways to say the same thing. The simplest example in a common language is (the French letter Ã©) is LATIN SMALL LETTER E WITH ACUTE [U+00E9] which can also be written as LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT [U+0301] The former being a composed normal form, the latter a decomposed normal form. (And yes, it is even much more complicated than that, it goes on for 1000s of pages). In the above example, the concept of character/string is indeed fuzzy. HTH, Sven

Dec. 9, 2015

Re: [Pharo-dev] doing stupid things (image will become unusable)

by stepharo

Le 8/12/15 17:29, Marcus Denker a Ã©crit : > Hi, > > There used to be a mechanism for that (readonly variable bindings). > > It was there but not used: half the globals where readonly, all > globals added > after the experiment was done where ânormalâ. > > With the new âfirst classâ variables we could easily add that back in > a clean way. Pleassssssssse it would be gorgeous :) > > Marcus > >> On 08 Dec 2015, at 17:19, Nicolai Hess <nicolaihess(a)gmail.com >> <mailto:nicolaihess@gmail.com>> wrote: >> >> Object := nil >> >> you can evaluate that piece of code, but afterwards .... >> >> Opal checks for assignments to read only variables (method arguments >> for example) >> and signals an error if you try to modify those vars. >> But it does not check globals. >> >> Should all Globals (OCLiteralVariable with isGlobalVar == true) be >> read only ? >> or can we distinguish global vars and class bindings? >> >

Dec. 9, 2015

Re: [Pharo-dev] displayingProgress on demand

by stepharo

Thanks Hernan. I love your attitude: - packaging and sharing Keep pushing Stef Le 9/12/15 07:03, HernÃ¡n Morales Durand a Ã©crit : > http://www.smalltalkhub.com/#!/~hernan/ProgressBarInstaller > <http://www.smalltalkhub.com/#%21/%7Ehernan/ProgressBarInstaller> > > Public write access > > Cheers, > > HernÃ¡n > > > 2015-12-08 20:40 GMT-03:00 HernÃ¡n Morales Durand > <hernan.morales(a)gmail.com <mailto:hernan.morales@gmail.com>>: > > Hi John, > > Thank you for your feedback and help! > > I started a ProgressBarInstaller using your corrections. > Unfortunately SmalltalkHub is not working for me right now but > will check tomorrow and share the link. > > Best regards, > > HernÃ¡n > > > 2015-12-08 13:56 GMT-03:00 John Brant <brant(a)refactoryworkers.com > <mailto:brant@refactoryworkers.com>>: > > On 12/7/2015 5:27 PM, HernÃ¡n Morales Durand wrote: > > > but now I am failing to see how to replace #collect: with > #collect:thenDo: using the rewriter: > > http://ws.stfx.eu/MOWS947F000O > > > You don't need to first search using the RBParseTreeSearcher. > The RBParseTreeRewriter works with search patterns. What your > code is really doing is searching for the literal pattern that > it found from the RBParseTreeSearcher in the FooClass>>foo: > method: > > aString asFileReference entries > collect: [ : fileEntry | self openOnFileNamed: > fileEntry fullName ] > > It then tries to replace it with the pattern: > > `@collection collect: `@arg1 > thenDo: [ : fileEntry | > index := index + 1. > JobProgress progress: (0.1 * index); > title: 'Procesing...', index asString ] > > However, since neither `@collection nor `@arg1 appear in your > search pattern, you get an error. > > Here's a version that works: > > | cm rewriteRule replacement | > cm := FooClass >> #foo:. > replacement := '`@collection collect: `@arg1 > thenDo: [ : fileEntry | > index := index + 1. > JobProgress progress: (0.1 * index); > title: ''Procesing...'', index > asString ]'. > rewriteRule := RBParseTreeRewriter new > replace: '`@collection collect: `@arg1' > with: replacement. > rewriteRule executeTree: cm parseTree. > rewriteRule tree newSource > > > Essentially, I used the same search pattern you used, but I > used it for the RBParseTreeRewriter and removed all of the > RBParseTreeSearcher code. If you want to change all #collect: > sends in a method, then you should probably change > `@aCollection and `@arg1 to ``@aCollection and ``@arg1. > > > John Brant > > >

Dec. 9, 2015