- Pharo-dev - lists.pharo.org

[pharo-project/pharo-core] 320769: 50488

by GitHub

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 320769988140aadea83fe1a598e23830ab075c1d https://github.com/pharo-project/pharo-core/commit/320769988140aadea83fe1a5… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-07 (Mon, 07 Dec 2015) Changed paths: M Morphic-Base.package/TextMorph.class/instance/event handling/mouseDown_.st M Rubric.package/RubExtraSelectionDecorator.class/instance/drawing/computeSecondarySelectionShapesFrom_to_.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50487.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50488.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50487.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50488.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st R Watchpoints.package/CircularCollection.class/README.md R Watchpoints.package/CircularCollection.class/class/instance creation/new_.st R Watchpoints.package/CircularCollection.class/definition.st R Watchpoints.package/CircularCollection.class/instance/initialization/initialize.st R Watchpoints.package/CircularCollection.class/instance/initialization/initialize_.st Log Message: ----------- 50488 17189 remove check for PluggableTextMorph in TextMorph>>#moudeDown: https://pharo.fogbugz.com/f/cases/17189 17145 remove watchpoint package https://pharo.fogbugz.com/f/cases/17145 16962 Code pane should not show all spaces selected when one is selected https://pharo.fogbugz.com/f/cases/16962 http://files.pharo.org/image/50/50488.zip

Dec. 7, 2015

Re: [Pharo-dev] [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org

by H. Hirzel

As the issue is about potential reuse of GNU Smalltalk code snippets / classes regarding Unicode the most straightforward thing is to ask Paolo Bonzini (http://smalltalk.gnu.org/blog/bonzinip) if he permits the reuse under an MIT license. So far no code has been identified which would be needed under an MIT license. HTH Hannes On 12/7/15, Martin BÃ¤hr <mbaehr(a)email.archlab.tuwien.ac.at> wrote: > Excerpts from Martin BÃ¤hr's message of 2015-12-07 16:17:28 +0100: >> randal schwartz suggests a cleanroom approach, though i personally think >> that >> is overkill. if a cleanroom approach were really necessary than i would >> never >> be allowed to switch jobs because once i have been working on one >> companys >> private code i would be tainted for life, as i could accidentally >> reproduce the >> same code in my next job. >> >> however, either you get the GNU-smalltalk authors to relicense their >> unicode >> classes under the MIT license, or they will need to be rewritten from >> scratch. >> >> i would suggest asking them in the name of having a common code-base for >> all >> smalltalks. > > http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123817.… > is worth reading in that context: > > So it's mainly an issue of being willing to cooperate. I think all GNU > Smalltalk people (almost all?) are and it's also true at least for some > Squeak people. If somebody took a package (several thousands lines of > code, etc.) wholesale, ported it to Squeak, and licensed it under MIT on > SqueakMap that would be copyright violation. But everything can be done > "cum grano salis". Nobody is going to sue you if you copy an interface > from GNU Smalltalk, reimplement in Squeak (for fun!), but you did look > at the method comments in GNU Smalltalk -- and OMG maybe you got a > glimpse of the source code just below! Copyright does not protect > having similar implementations. > > I think we're all for interoperability and for exchange of opinions > within the communities. > > so there you have the open invitation to go and ask... > > and read this > http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123838.… > for a good summary on the issue. > > greetings, martin. > > -- > eKita - the online platform for your entire academic > life > -- > chief engineer > eKita.co > pike programmer pike.lysator.liu.se caudium.net > societyserver.org > secretary > beijinglug.org > mentor > fossasia.org > foresight developer foresightlinux.org > realss.com > unix sysadmin > Martin BÃ¤hr working in china > http://societyserver.org/mbaehr/ > >

Dec. 7, 2015

Re: [Pharo-dev] Quick debugging failed #assert:equals tests (Differ in comment)

by Denis Kudriashov

Nice to get it in Pharo. Many years ago I saw it in Dolphin 2015-12-07 12:50 GMT+01:00 Torsten Bergmann <astares(a)gmx.de>: > Hi > > the #assert:equals method is often used in Test cases: we have 2491 > senders in > recent Pharo5.0 Latest update: #50487. > > Anytime I have to debug a failing test using it I find it hard to compare > the > expected and actual value when it is a string in the debugger panes. > > Pharo tools come to the rescue because we already have a differ inside the > image. > > I would therefore propose to add the following comment into > TAssertable>>#assert:equals: > > assert: actual equals: expected > " DiffModel new > showOptions: true; > leftText: actual; > rightText: expected; > contextClass: nil; > openWithSpec " > > ^ self > assert: expected = actual > description: [self comparingStringBetween: actual and: > expected] > > With this additional comment one can quickly open the diff tool and check > the differences in the > expected vs. actual string. Directly by evaluating the commented > expression in the debugger. > > What do you think? Should we add it? Are there any other solutions to > quicly find the differences? > > Thanks > T. > >

Dec. 7, 2015

Re: [Pharo-dev] Hook "WACurrentRequestContext" into debugger?

by Mariano Martinez Peck

OK Max. In the last version you can find the "likely" final version. It's extremely simple, easy to use, and very very little code. Only one override. All WADynamicVariables subclasses handled automatically. All you need to do is to register the handler: app filter configuration at: #exceptionHandler put: WAPharoDebuggererErrorHandler. It works for exceptions and both Halt too. The only limit is only the "last opened debugger" will be correct and all previous will be wrong since they will be using another values for dynamic variables. I will see if people want to integrate this directly into Seaside. On Sat, Dec 5, 2015 at 11:36 AM, Max Leske <maxleske(a)gmail.com> wrote: > > On 05 Dec 2015, at 12:58, Mariano Martinez Peck <marianopeck(a)gmail.com> > wrote: > > Can you try with the latest version that does not use LPC if it works? > > > Works perfectly! > -- Mariano http://marianopeck.wordpress.com

Dec. 7, 2015

Re: [Pharo-dev] Quick debugging failed #assert:equals tests (Differ in comment)

by Bernardo Ezequiel Contreras

+1 On Mon, Dec 7, 2015 at 8:50 AM, Torsten Bergmann <astares(a)gmx.de> wrote: > Hi > > the #assert:equals method is often used in Test cases: we have 2491 > senders in > recent Pharo5.0 Latest update: #50487. > > Anytime I have to debug a failing test using it I find it hard to compare > the > expected and actual value when it is a string in the debugger panes. > > Pharo tools come to the rescue because we already have a differ inside the > image. > > I would therefore propose to add the following comment into > TAssertable>>#assert:equals: > > assert: actual equals: expected > " DiffModel new > showOptions: true; > leftText: actual; > rightText: expected; > contextClass: nil; > openWithSpec " > > ^ self > assert: expected = actual > description: [self comparingStringBetween: actual and: > expected] > > With this additional comment one can quickly open the diff tool and check > the differences in the > expected vs. actual string. Directly by evaluating the commented > expression in the debugger. > > What do you think? Should we add it? Are there any other solutions to > quicly find the differences? > > Thanks > T. > > -- Bernardo E.C. Sent from a cheap desktop computer in South America.

Dec. 7, 2015

Fwd: [squeak-dev] Unicode Support

by Ben Coman

Sorry to double post, I missed the cross-post. On Fri, Dec 4, 2015 at 7:42 PM, EuanM <euanmee(a)gmail.com> wrote: > I'm currently groping my way to seeing how feature-complete our > Unicode support is. I am doing this to establish what still needs to > be done to provide full Unicode support. > > This seems to me to be an area where it would be best to write it > once, and then have the same codebase incorporated into the Smalltalks > that most share a common ancestry. > > I am keen to get: equality-testing for strings; sortability for > strings which have ligatures and diacritic characters; and correct > round-tripping of data. > > Call to action: > ========== > > If you have comments on these proposals - such as "but we already have > that facility" or "the reason we do not have these facilities is > because they are dog-slow" - please let me know them. > > If you would like to help out, please let me know. > > If you have Unicode experience and expertise, and would like to be, or > would be willing to be, in the 'council of experts' for this project, > please let me know. > > If you have comments or ideas on anything mentioned in this email > > In the first instance, the initiative's website will be: > http://smalltalk.uk.to/unicode.html > > I have created a SqueakSource.com project called UnicodeSupport > > I want to avoid re-inventing any facilities which already exist. > Except where they prevent us reaching the goals of: > - sortable UTF8 strings > - sortable UTF16 strings > - equivalence testing of 2 UTF8 strings > - equivalence testing of 2 UTF16 strings > - round-tripping UTF8 strings through Smalltalk > - roundtripping UTF16 strings through Smalltalk. > As I understand it, we have limited Unicode support atm. > > Current state of play > =============== > ByteString gets converted to WideString when need is automagically detected. > > Is there anything else that currently exists? > > Definition of Terms > ============== > A quick definition of terms before I go any further: > > Standard terms from the Unicode standard > =============================== > a compatibility character : an additional encoding of a *normal* > character, for compatibility and round-trip conversion purposes. For > instance, a 1-byte encoding of a Latin character with a diacritic. > > Made-up terms > ============ > a convenience codepoint : a single codepoint which represents an item > that is also encoded as a string of codepoints. > > (I tend to use the terms compatibility character and compatibility > codepoint interchangably. The standard only refers to them as > compatibility characters. However, the standard is determined to > emphasise that characters are abstract and that codepoints are > concrete. So I think it is often more useful and productive to think > of compatibility or convenience codepoints). > > a composed character : a character made up of several codepoints > > Unicode encoding explained > ===================== > A convenience codepoint can therefore be thought of as a code point > used for a character which also has a composed form. > > The way Unicode works is that sometimes you can encode a character in > one byte, sometimes not. Sometimes you can encode it in two bytes, > sometimes not. > > You can therefore have a long stream of ASCII which is single-byte > Unicode. If there is an occasional Cyrillic or Greek character in the > stream, it would be represented either by a compatibility character or > by a multi-byte combination. > > Using compatibility characters can prevent proper sorting and > equivalence testing. > > Using "pure" Unicode, ie. "normal encodings", can cause compatibility > and round-tripping probelms. Although avoiding them can *also* cause > compatibility issues and round-tripping problems. > > Currently my thinking is: > > a Utf8String class > an Ordered collection, with 1 byte characters as the modal element, > but short arrays of wider strings where necessary > a Utf16String class > an Ordered collection, with 2 byte characters as the modal element, > but short arrays of wider strings > beginning with a 2-byte endianness indicator. > > Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. > > So my thinking is that Utf8String will contain convenience codepoints, > for round-tripping. And where there are multiple convenience > codepoints for a character, that it standardises on one. > > And that there is a Utf8SortableString which uses *only* normal characters. > > We then need methods to convert between the two. > > aUtf8String asUtf8SortableString > > and > > aUtf8SortableString asUtf8String > > > Sort orders are culture and context dependent - Sweden and Germany > have different sort orders for the same diacritic-ed characters. Some > countries have one order in general usage, and another for specific > usages, such as phone directories (e.g. UK and France) > > Similarly for Utf16 : Utf16String and Utf16SortableString and > conversion methods > > A list of sorted words would be a SortedCollection, and there could be > pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, > seOrder, ukOrder, etc > > along the lines of > aListOfWords := SortedCollection sortBlock: deOrder > > If a word is either a Utf8SortableString, or a well-formed Utf8String, > then we can perform equivalence testing on them trivially. > > To make sure a Utf8String is well formed, we would need to have a way > of cleaning up any convenience codepoints which were valid, but which > were for a character which has multiple equally-valid alternative > convenience codepoints, and for which the string currently had the > "wrong" convenience codepoint. (i.e for any character with valid > alternative convenience codepoints, we would choose one to be in the > well-formed Utf8String, and we would need a method for cleaning the > alternative convenience codepoints out of the string, and replacing > them with the chosen approved convenience codepoint. > > aUtf8String cleanUtf8String > > With WideString, a lot of the issues disappear - except > round-tripping(although I'm sure I have seen something recently about > 4-byte strings that also have an additional bit. Which would make > some Unicode characters 5-bytes long.) > > > (I'm starting to zone out now - if I've overlooked anything - obvious, > subtle, or somewhere in between, please let me know) > > Cheers, > Euan > Good initiative. Here is some info I've bookmarked over time... http://www.joelonsoftware.com/articles/Unicode.html >> The Single Most Important Fact About Encodings - It does not make sense to have a string without knowing what encoding it uses. You can no longer stick your head in the sand and pretend that "plain" text is ASCII. There Ain't No Such Thing As Plain Text. If you have a string, in memory, in a file, or in an email message, you have to know what encoding it is in or you cannot interpret it or display it to users correctly. http://kunststube.net/encoding/ >> So what does it mean for a language to natively support or not support Unicode? It basically refers to whether the language assumes that one character equals one byte or not. >> What does it mean for a language to support Unicode then? Javascript for example supports Unicode. In fact, any string in Javascript is UTF-16 encoded. In fact, it's the only thing Javascript deals with. You cannot have a string in Javascript that is not UTF-16 encoded. Javascript worships Unicode to the extent that there's no facility to deal with any other encoding in the core language. >> Other languages are simply encoding-aware. Internally they store strings in a particular encoding, http://cafe.elharo.com/programming/the-ten-commandments-of-unicode/ >> 1. I am Unicode, thy character set. Thou shalt have no other character sets before me. >> 4. Thou shalt not refer to Unicode as a two-byte character set. >> 6. Thou shalt count and index Unicode characters, not UTF-16 code points. >> 7. Thou shalt use UTF-8 as the preferred encoding wherever possible. https://xkcd.com/1137/ >> ruomuh https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conc… >> Strings and Characters in Swift http://oleb.net/blog/2014/07/swift-strings/ Interesting article, though I don't yet grok it all, I thought it worth sharing... >> SwiftÊ¼s string implemenation makes working with Unicode easier and significantly less error-prone. >> Strings in Swift are represented by the String type. A String is a collection of Charactervalues. A Swift Character represents one perceived character (what a person thinks of as a single character, called a grapheme). Since Unicode often uses two or more code points(called a grapheme cluster) to form one perceived character, this implies that a Charactercan be composed of multiple Unicode scalar values if they form a single grapheme cluster. (Unicode scalar is the term for any Unicode code point except surrogate pair characters, which are used to encode UTF-16.) >> This change has the potential to prevent many common errors when dealing with string lengths or substrings. It is a huge difference to most other Unicode-aware string libraries (including NSString) where the building blocks of a string are usually UTF-16 code units or single Unicode scalars. cheers -ben

Dec. 7, 2015

Re: [Pharo-dev] Unicode Support

by Henrik Johansen

> On 07 Dec 2015, at 2:06 , Henrik Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> >> codepoints represent "*encoded characters*" > > No. a codepoint is the numerical value assigned to a character. An "encoded character" is the way a codepoint is represented in bytes using a given encoding. You were right on this point, I see I remembered the terminology of this incorrectly. http://www.unicode.org/versions/Unicode8.0.0/ch02.pdf figure 2.8 does use "encoded characters" for the mapping of abstract characters to its equivalent codepoint (s/ sequences). What I thought it meant is better described as a codepoint's byte output using an "encoding scheme". An accurate description following that terminology, would be that Pharo/Squeak Strings keep data in UTF32 encoding form, where 1 codepoint = 1 code unit, dynamically switched between Latin1 (ByteStrings) and UTF32 (WideStrings) encoding schemes as needed. With the same terminology, the difference between a code point, a code unit, how an encoding scheme represents a codepoint as code units/bytes, are the concepts it is important to distinguish. Quite a mouthful though! Cheers Henry

Dec. 7, 2015

Re: [Pharo-dev] [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)

by Henrik Johansen

> On 07 Dec 2015, at 12:18 , EuanM <euanmee(a)gmail.com> wrote: > > Hannes, > > The Unicode standard provide compatibility codepoints for > compatibility purposes and prefers all characters to be represented > composed form - as that way they are comparable and sortable. > > (Some composed characters have *more than one* compatibility codepoint. > > The canonical example is the composed character #(0041 030a) which can > be represented by EITHER the compatibility codepoint #(00c5) "Latin > Capital Letter A with Ring" above OR by #(212b) "Angstrom sign" ) > > On 7 December 2015 at 08:17, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: >> On 12/7/15, EuanM <euanmee(a)gmail.com> wrote: >>> My current thinking for collation sequences: >>> >>> All strings being collated have had all compatibility codepoints >>> expanded into composed sequences. >> >> What does the Unicode manual suggest? (www.unicode.org reference?) >> While nice, it is by no means required to keep/convert the strings to Normalizion form D, as long as the operation returns the same results as if they were (http://unicode.org/reports/tr10/#Main_Algorithm) "Step 1. Produce a normalized form of each input string, applying S1.1. S1.1 <http://unicode.org/reports/tr10/#S1.1> Use the Unicode canonical algorithm to decompose characters according to the canonical mappings. That is, put the string into Normalization Form D (see [UAX15 <http://unicode.org/reports/tr10/#UAX15>]). Conformant implementations may skip this step in certain circumstances, as long as they get the same results. For techniques that may be useful in such an approach, seeSection 6.5, Avoiding Normalization <http://unicode.org/reports/tr10/#Avoiding_Normalization>." So it's a tradeoff between whether you complicate other string operations that create/modify strings to maintain normalized data, or take the hit when collating. Cheers, Henry

Dec. 7, 2015

Re: [Pharo-dev] Unicode Support

by Henrik Johansen

> On 07 Dec 2015, at 1:05 , EuanM <euanmee(a)gmail.com> wrote: > > Hi Henry, > > To be honest, at some point I'm going to long for the for the much > more succinct semantics of healthcare systems and sports scoring and > administration systems again. :-) > > codepoints are any of *either* > - the representation of a component of an abstract character, *or* > eg. "A" #(0041) as a component of > - the sole representation of the whole of an abstract character *or* of > - a representation of an abstract character provided for backwards > compatibility which is more properly represented by a series of > codepoints representing a composed character > > e.g. > > The "A" #(0041) as a codepoint can be: > the sole representation of the whole of an abstract character "A" #(0041) > > The representation of a component of the composed (i.e. preferred) > version of the abstract character â« #(0041 030a) > > â« (#00C5) represents one valid compatibility form of the abstract > character â« which is most properly represented by #(0041 030a). > > â« (#212b) also represents one valid compatibility form of the abstract > character â« which is most properly represented by #(0041 030a). > > With any luck, this satisfies both our semantic understandings of the > concept of "codepoint" > > Would you agree with that? > > In Unicode, codepoints are *NOT* an abstract numerical representation > of a text character. > > At least not as we generally understand the term "text character" from > our experience of non-Unicode character mappings. I agree, they are numerical representations of what Unicode refers to as characters. > > codepoints represent "*encoded characters*" No. a codepoint is the numerical value assigned to a character. An "encoded character" is the way a codepoint is represented in bytes using a given encoding. > and "a *text element* ... > is represented by a sequence of one or more codepoints". (And the > term "text element" is deliberately left undefined in the Unicode > standard) > > Individual codepoints are very often *not* the encoded form of an > abstract character that we are interested in. Unless we are > communicating to or from another system (Which in some cases is the > Smalltalk ByteString class) > > i.e. in other words > > *Some* individual codepoints *may* be a representation of a specific > *abstract character*, but only in special cases. > > The general case in Unicode is that Unicode defines (a) > representation(s) of a Unicode *abstract character*. > > The Unicode standard representation of an abstract character is a > composed sequence of codepoints, where in some cases that sequence is > as short as 1 codepoint. > > In other cases, Unicode has a compatibility alias of a single > codepoint which is *also* a representation of an abstract character > > There are some cases where an abstract character can be represented by > more than one single-codepoint compatibility codepoint. > > Cheers, > Euan I agree you have a good grasp of the distinction between an abstract character (characters and character sequences which should be treated equivalent wrt, equality / sorting / display, etc.) and a character (which each have a code point assigned). That is besides the point both Sven and I tried to get through, which is the difference between a code point and the encoded form(s) of said code point. When you write: "and therefore encodable in UTF-8 as compatibility codepoint e9 hex and as the composed character #(0065 00b4) (all in hex) and as the same composed character as both #(feff 0065 00b4) and #(ffef 0065 00b4) when endianness markers are included" I's quite clear you confuse the two. 0xFEFF is the codepoint of the character used as bom. When you state that it can be written ffef (I assume you meant FFFE), you are again confusing the code point and its encoded value (an encoded value which only occurs in UTF16/32, no less). When this distinction is clear, it might be easier to see that value in that Strings are kept as Unicode code points arrays, and converted to encoded forms when entering/exiting the system. Cheers, Henry

Dec. 7, 2015

Re: [Pharo-dev] Help needed: old issues issue tracker entries

by Esteban Lorenzano

> On 07 Dec 2015, at 13:42, Peter Uhnak <i.uhnak(a)gmail.com> wrote: > >> Can *YOU* help to get it to 637? > > Without activity/reply for some time and no clear action --- maybe close > as stale?: > > * https://pharo.fogbugz.com/f/cases/10047/Regex-not-independant-of-platform > * https://pharo.fogbugz.com/f/cases/13479/ValueHolder-and-NewValueHolder-do-n… > * https://pharo.fogbugz.com/f/cases/14500/Manual-class-replacement-unwanted-o… > * https://pharo.fogbugz.com/f/cases/15098/Default-button-size > > How do I push VM change?: > > * https://pharo.fogbugz.com/f/cases/14759/Linux-X11-VM-default-encoding-shoul… Iâlll review and accept it itâs been there from some time but I remember I didnât merge at the time for some reasonâ¦ now I just forget what was (shame on me, I didnât take a note :( )â¦ so I will re-review it :) Esteban > > -- > Peter >

Dec. 7, 2015