December 2015 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org

by EuanM

Yes, clean room approaches are overkill. Obviousness in code has long been found to be a protection against copyright. For instance, you cannot copyright a programme in BASIC that says 10 Print "Hello World" 20 GOTO 10 (And by copyright, I mean prevent anyone else in the world from typing that programme and distributing their copy). One of the great advantage of writing weird, convoluted or just plain dense clever code is that it becomes non-obvious and therefore indubitably covered under copyright. The mere fact that code has been written somewhere, does *not* make it unusable elsewhere. (As an aside, I tend to write plain, straightforward code. Asserting copyright on a lot of it would be hard). One of the great advantages of not doing my coding in the USA is that Stallman would have to come after me in my jurisdiction, not his. Probably a special case of security through obscurity. Usual disclaimer. IANAL. Just interested in these issues from having developed legal support systems (including litigation support systems). As Hannes says, we haven't hit an area of the project where it is an issue yet. But personally, I'm quite relaxed about looking at the Gnu Smalltalk codebase. I'm pretty certain I can re-implement without violating copyright. In the same way I can prÃ©cis and paraphrase a short story without violating the copyright of the short story. Can any of you tell me the story of Hansel and Gretel without violating the copyright of some recently published re-telling of the tale? I'm pretty certain you could. Please remember that it is much easier to avoid copyright violation than it is to avoid patent violation, in these sorts of cases. None of the Gnu Smalltalk base is patented. Cheers, Euan (back to reading the ICU) On 7 December 2015 at 16:06, Martin BÃ¤hr <mbaehr(a)email.archlab.tuwien.ac.at> wrote: > Excerpts from Martin BÃ¤hr's message of 2015-12-07 16:17:28 +0100: >> randal schwartz suggests a cleanroom approach, though i personally think that >> is overkill. if a cleanroom approach were really necessary than i would never >> be allowed to switch jobs because once i have been working on one companys >> private code i would be tainted for life, as i could accidentally reproduce the >> same code in my next job. >> >> however, either you get the GNU-smalltalk authors to relicense their unicode >> classes under the MIT license, or they will need to be rewritten from scratch. >> >> i would suggest asking them in the name of having a common code-base for all >> smalltalks. > > http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123817.… > is worth reading in that context: > > So it's mainly an issue of being willing to cooperate. I think all GNU > Smalltalk people (almost all?) are and it's also true at least for some > Squeak people. If somebody took a package (several thousands lines of > code, etc.) wholesale, ported it to Squeak, and licensed it under MIT on > SqueakMap that would be copyright violation. But everything can be done > "cum grano salis". Nobody is going to sue you if you copy an interface > from GNU Smalltalk, reimplement in Squeak (for fun!), but you did look > at the method comments in GNU Smalltalk -- and OMG maybe you got a > glimpse of the source code just below! Copyright does not protect > having similar implementations. > > I think we're all for interoperability and for exchange of opinions > within the communities. > > so there you have the open invitation to go and ask... > > and read this > http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123838.… > for a good summary on the issue. > > greetings, martin. > > -- > eKita - the online platform for your entire academic life > -- > chief engineer eKita.co > pike programmer pike.lysator.liu.se caudium.net societyserver.org > secretary beijinglug.org > mentor fossasia.org > foresight developer foresightlinux.org realss.com > unix sysadmin > Martin BÃ¤hr working in china http://societyserver.org/mbaehr/

Dec. 9, 2015

Re: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support))

by EuanM

http://www.unicode.org/reports/tr15/#Stable_Code_Points Table 7, the discussion of Ligatures, (which uses the ligature of "ffi" as its example) Every time I think I'm about to grokk this standard, something like this crops up. On 9 December 2015 at 00:43, Dale Henrichs <dale.henrichs(a)gemtalksystems.com> wrote: > > > On 12/08/2015 04:33 PM, EuanM wrote: >> >> Reading up http://www.unicode.org/reports/tr15/#Examples >> >> The Unicode standard seems to require you never to make >> aGermanStrasse >> equivalent in a sort order to the ligatured version, >> aGermanStraÃe > > I didn't see anything in the examples related to this ... could you be a bit > more specific? > > Dale

Dec. 9, 2015

Re: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support))

by Dale Henrichs

On 12/08/2015 04:33 PM, EuanM wrote: > Reading up http://www.unicode.org/reports/tr15/#Examples > > The Unicode standard seems to require you never to make > aGermanStrasse > equivalent in a sort order to the ligatured version, > aGermanStraÃe I didn't see anything in the examples related to this ... could you be a bit more specific? Dale

Dec. 9, 2015

SecureSession version 9 is officially released

by Robert Withers

Good evening, After some sprints over the past couple of days, the SecureSession project has reached an Official Release. This project implements a Certificateless secure connection, with no third parties. It is a 6 leg handshake. Here is the wiki: https://github.com/SqueakCryptographySquad/Mushroom/wiki/SecureSession thank You, Robert

Dec. 9, 2015

Re: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support))

by EuanM

Reading up http://www.unicode.org/reports/tr15/#Examples The Unicode standard seems to require you never to make aGermanStrasse equivalent in a sort order to the ligatured version, aGermanStraÃe This seems counter-intuitive to me. Is there a reason for this? Have I just simply picked this up wrongly? On 8 December 2015 at 21:35, Dale Henrichs <dale.henrichs(a)gemtalksystems.com> wrote: > Euan, > > What I meant is that you can't _always_ use the code point for collation, > i.e., sorting based on the value of code points is not always correct[1]. > > If I'm not mistaken the fully-composed and fully-decomposed forms can only > be used for testing the equivalence of two strings[2] ... > > The Main Algorithm[3], starts by producing a normalized form of the string, > but the subsequent steps (produce array, form sort key and compare) involves > table lookups among other things .... > > Once you've produced a sort key for a string, the sort key does use "binary > comparison" for collating , which is a byte by byte numeric comparison ... > > Dale > > [1] http://www.unicode.org/reports/tr10/#Common_Misperceptions > [2] http://unicode.org/reports/tr15/pdtr15.html > [3] http://www.unicode.org/reports/tr10/#Main_Algorithm > > > On 12/08/2015 12:22 PM, EuanM wrote: >> >> Dale - is that you can't depend on the value of a codepoint >> *unless the string is either in fully-composed form >> (or has just been fully-decomposed from a fully-composed form) * >> >> OR are there circumstances where even those two cases cannot be relied >> upon? >> >> On 8 December 2015 at 19:20, Dale Henrichs >> <dale.henrichs(a)gemtalksystems.com> wrote: >>> >>> >>> On 12/07/2015 11:31 PM, H. Hirzel wrote: >>>> >>>> Dale >>>> >>>> Thank you for your answer with links to the ICU library and the notes >>>> about classes in Gemstone. Noteworthy that you have a class Utf8 as a >>>> subclass of ByteArray. >>>> >>>> I understand that Gemstone uses the ICU library and thus does not >>>> implement the algorithms in Smalltalk. >>>> >>>> I am currently looking into what the ICU library provides. >>>> >>>> I found as well a Ruby library [2] which implements CLDR [3] >>>> >>>> It has methods like this >>>> >>>> "Alphabetize a list using regular Ruby sort:" >>>> >>>> $> ["Art", "Wasa", "Ãlg", "Ved"].sort >>>> $> ["Art", "Ved", "Wasa", "Ãlg"] >>>> >>>> Alphabetize a list using TwitterCLDRâs locale-aware sort: >>>> >>>> $> ["Art", "Wasa", "Ãlg", "Ved"].localize(:de).sort.to_a >>>> $> ["Ãlg", "Art", "Ved", "Wasa"] >>>> >>>> I hope that given such an example it would not be too difficult to >>>> reimplement a similar sort algorithm in Squeak/Cuis/Pharo. Currently >>>> the interest is in getting sorting done in a cross-dialect-way. >>>> >>> I think that the issue (from a performance perspective) is that you can't >>> depend upon the value of the code point when doing collation --- the main >>> algorithm[5] is pretty much table based --- In addition to the different >>> sort orders based on characters there are even more arcane sort rules >>> where >>> characters at the end of a word can affect the sort order of the word >>> (for >>> more info see[4]). >>> >>> It is worth looking at the Conformance section of the Unicode spec[1] as >>> there are different levels of collation conformance ..... >>> >>> ICU conforms[2] to to UTS #10[3], the highest level of conformance ... >>> >>> It looks like TwitterCLDR[6] uses the Main Algorithm[5] with >>> tailoring[7]. >>> They don't claim to be conformant to the Unicode Collation Algorithm[3], >>> but >>> they are covering a big chunk of the standard use cases .... >>> >>> Dale >>> >>> [1] http://unicode.org/reports/tr10/#Conformance >>> [2] http://userguide.icu-project.org/collation >>> [3] http://www.unicode.org/reports/tr10/ >>> [4] http://www.unicode.org/reports/tr10/#Introduction >>> [5] http://www.unicode.org/reports/tr10/#Main_Algorithm >>> [6] >>> >>> https://blog.twitter.com/2012/twittercldr-improving-internationalization-su… >>> [7] http://unicode.org/reports/tr10/#Tailoring > >

Dec. 9, 2015

Re: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support))

by EuanM

Hi Todd, it's taken me til now to put my thoughts into words on this issue. I think we should make it work first. This will allow us to gain more insight into the issues, and create documentation about the process that we, as a community, understand. If ICU is the right way to go, we can *then* "make it right". On 8 December 2015 at 22:36, Todd Blanchard <tblanchard(a)mac.com> wrote: > I just want to second Dale's endorsement of the ICU library. It has been > around a long time (originally developed by Taligent) and it provides the > base unicode capabilities for an awful lot of software. > > I think it would make more sense to bring icu into Smalltalk as a > NativeBoost library than to spend resources reimplementing and maintaining > it. > > -Todd Blanchard > > On Dec 8, 2015, at 11:20, Dale Henrichs <dale.henrichs(a)gemtalksystems.com> > wrote: > > On 12/07/2015 11:31 PM, H. Hirzel wrote: > > Dale > > Thank you for your answer with links to the ICU library and the notes > about classes in Gemstone. Noteworthy that you have a class Utf8 as a > subclass of ByteArray. > > I understand that Gemstone uses the ICU library and thus does not > implement the algorithms in Smalltalk. > > I am currently looking into what the ICU library provides. > >

Dec. 9, 2015

Re: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support))

by EuanM

Equally old are the NextStep Object C functions which are now embodied within MacOS X. It might also be informative for us to see what's done there. On 8 December 2015 at 23:50, EuanM <euanmee(a)gmail.com> wrote: > Dale, > > yes - sorting based on the value of codepoints is almost always > guaranteed to be wrong. Sorting is an application-specific issue, not > a technical Unicode issue, as there is more than one canonical sort > order per culture, and there is often more than one culture per > writing system. > > e.g. ISO Latin 1 / Latin 9 > covers these cultures (amongst others) > English (2 sort orders); Spanish; French (2 sort orders); German (2 > sort orders); Swedish; etc > > German sort order differs from Swedish for the same characters, etc > > Todd, > > My thinking is that if we implement fully-composed strings as > heterogenous arrays, we sidestep a lot of the complexity of the ICU. > > If it turns out that the performance is terrible, we can then seek to > incorporate the ICU. > > > On 8 December 2015 at 22:36, Todd Blanchard <tblanchard(a)mac.com> wrote: >> I just want to second Dale's endorsement of the ICU library. It has been >> around a long time (originally developed by Taligent) and it provides the >> base unicode capabilities for an awful lot of software. >> >> I think it would make more sense to bring icu into Smalltalk as a >> NativeBoost library than to spend resources reimplementing and maintaining >> it. >> >> -Todd Blanchard >> >> On Dec 8, 2015, at 11:20, Dale Henrichs <dale.henrichs(a)gemtalksystems.com> >> wrote: >> >> On 12/07/2015 11:31 PM, H. Hirzel wrote: >> >> Dale >> >> Thank you for your answer with links to the ICU library and the notes >> about classes in Gemstone. Noteworthy that you have a class Utf8 as a >> subclass of ByteArray. >> >> I understand that Gemstone uses the ICU library and thus does not >> implement the algorithms in Smalltalk. >> >> I am currently looking into what the ICU library provides. >> >>

Dec. 8, 2015

Re: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support))

by EuanM

Dale, yes - sorting based on the value of codepoints is almost always guaranteed to be wrong. Sorting is an application-specific issue, not a technical Unicode issue, as there is more than one canonical sort order per culture, and there is often more than one culture per writing system. e.g. ISO Latin 1 / Latin 9 covers these cultures (amongst others) English (2 sort orders); Spanish; French (2 sort orders); German (2 sort orders); Swedish; etc German sort order differs from Swedish for the same characters, etc Todd, My thinking is that if we implement fully-composed strings as heterogenous arrays, we sidestep a lot of the complexity of the ICU. If it turns out that the performance is terrible, we can then seek to incorporate the ICU. On 8 December 2015 at 22:36, Todd Blanchard <tblanchard(a)mac.com> wrote: > I just want to second Dale's endorsement of the ICU library. It has been > around a long time (originally developed by Taligent) and it provides the > base unicode capabilities for an awful lot of software. > > I think it would make more sense to bring icu into Smalltalk as a > NativeBoost library than to spend resources reimplementing and maintaining > it. > > -Todd Blanchard > > On Dec 8, 2015, at 11:20, Dale Henrichs <dale.henrichs(a)gemtalksystems.com> > wrote: > > On 12/07/2015 11:31 PM, H. Hirzel wrote: > > Dale > > Thank you for your answer with links to the ICU library and the notes > about classes in Gemstone. Noteworthy that you have a class Utf8 as a > subclass of ByteArray. > > I understand that Gemstone uses the ICU library and thus does not > implement the algorithms in Smalltalk. > > I am currently looking into what the ICU library provides. > >

Dec. 8, 2015

Re: [Pharo-dev] displayingProgress on demand

by HernÃ¡n Morales Durand

Hi John, Thank you for your feedback and help! I started a ProgressBarInstaller using your corrections. Unfortunately SmalltalkHub is not working for me right now but will check tomorrow and share the link. Best regards, HernÃ¡n 2015-12-08 13:56 GMT-03:00 John Brant <brant(a)refactoryworkers.com>: > On 12/7/2015 5:27 PM, HernÃ¡n Morales Durand wrote: > >> >> but now I am failing to see how to replace #collect: with >> #collect:thenDo: using the rewriter: >> >> http://ws.stfx.eu/MOWS947F000O >> > > You don't need to first search using the RBParseTreeSearcher. The > RBParseTreeRewriter works with search patterns. What your code is really > doing is searching for the literal pattern that it found from the > RBParseTreeSearcher in the FooClass>>foo: method: > > aString asFileReference entries > collect: [ : fileEntry | self openOnFileNamed: fileEntry fullName ] > > It then tries to replace it with the pattern: > > `@collection collect: `@arg1 > thenDo: [ : fileEntry | > index := index + 1. > JobProgress progress: (0.1 * index); > title: 'Procesing...', index asString ] > > However, since neither `@collection nor `@arg1 appear in your search > pattern, you get an error. > > Here's a version that works: > > | cm rewriteRule replacement | > cm := FooClass >> #foo:. > replacement := '`@collection collect: `@arg1 > thenDo: [ : fileEntry | > index := index + 1. > JobProgress progress: (0.1 * index); > title: ''Procesing...'', index asString ]'. > rewriteRule := RBParseTreeRewriter new > replace: '`@collection collect: `@arg1' > with: replacement. > rewriteRule executeTree: cm parseTree. > rewriteRule tree newSource > > > Essentially, I used the same search pattern you used, but I used it for > the RBParseTreeRewriter and removed all of the RBParseTreeSearcher code. If > you want to change all #collect: sends in a method, then you should > probably change `@aCollection and `@arg1 to ``@aCollection and ``@arg1. > > > John Brant > >

Dec. 8, 2015

Re: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support))

by Todd Blanchard

I just want to second Dale's endorsement of the ICU library. It has been around a long time (originally developed by Taligent) and it provides the base unicode capabilities for an awful lot of software. I think it would make more sense to bring icu into Smalltalk as a NativeBoost library than to spend resources reimplementing and maintaining it. -Todd Blanchard > On Dec 8, 2015, at 11:20, Dale Henrichs <dale.henrichs(a)gemtalksystems.com> wrote: > > On 12/07/2015 11:31 PM, H. Hirzel wrote: >> Dale >> >> Thank you for your answer with links to the ICU library and the notes >> about classes in Gemstone. Noteworthy that you have a class Utf8 as a >> subclass of ByteArray. >> >> I understand that Gemstone uses the ICU library and thus does not >> implement the algorithms in Smalltalk. >> >> I am currently looking into what the ICU library provides.

Dec. 8, 2015