- Pharo-dev - lists.pharo.org

[pharo-project/pharo-core]
by GitHub Dec. 16, 2015

Dec. 16, 2015

Branch: refs/tags/50500 Home: https://github.com/pharo-project/pharo-core

1 0

[pharo-project/pharo-core] 8ec2c9: 50500
by GitHub Dec. 16, 2015

Dec. 16, 2015

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 8ec2c9995556d7decba6455aa9d8eb44fe6575ef https://github.com/pharo-project/pharo-core/commit/8ec2c9995556d7decba6455a… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-16 (Wed, 16 Dec 2015) Changed paths: M FFI-NB.package/FFICalloutAPI.class/instance/action/function_module_.st A FFI-NB.package/FFIMethodRegistry.class/class/cleanup/cleanUp_.st M FFI-NB.package/FFIMethodRegistry.class/instance/accessing/checkPlatform.st M FFI-NB.package/FFIMethodRegistry.class/instance/accessing/reset.st M FFI-NB.package/extension/ExternalData/instance/+.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50499.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50500.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50499.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50500.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st M System-OSEnvironments.package/Win32Environment.class/instance/private/getEnv_.st Log Message: ----------- 50500 17239 monticello: understand to new immediate classes (spur migration) https://pharo.fogbugz.com/f/cases/17239 http://files.pharo.org/image/50/50500.zip

1 0

Question about errno and NB FFI
by Mariano Martinez Peck Dec. 15, 2015

Dec. 15, 2015

Hi guys, I am using NB FFI and I am calling a function that could answer a NULL* in case of error and that it should have set the number on *errno.* So now I have some questions: Say the code is this: run | whatever | whatever := self *primitiveWhatever*. ^ popenFile ifNil: [ self *primitivePerror*. Imagine that the C lib called from #primitiveWhatever was the one setting *errno.* My question is... the following #primitivePerror is a DIFFERENT system call. Couldn't that happened that another error happened before the first system call and time I am executing the second one and so the "last error" (errno) is actually the wrong one I am thinking of? Even worst, I may think that the scope of the "global variable errno" may existed at the scope of the invoked lib only? In this case I don't have above problem, but then....how can I get such number once the NB primitive finished (so that I can call, for example, strerror() ? Thanks in advance, -- Mariano http://marianopeck.wordpress.com

2 2

Compact representation of source code history
by Stephan Eggermont Dec. 15, 2015

Dec. 15, 2015

The current format we use for storing source code is not optimal for archival and analysis purposes. Each mcz stores all source code. That makes it difficult and slow. Today I've experimented with an archive format that combines many mcz's and should be able to reconstruct all individual ones. I defined a MCProject, representing a project repository. The definitions are stored in an OCLiteralSet Object subclass: #MCProject instanceVariableNames: 'location infos definitions repository' classVariableNames: '' package: 'MonticelloProjects' For each filename found in the repository I load the MCVersion and its snapshot. MCProject>>read | filenames | repository := MCHttpRepository location: location user: '' password: ''. filenames := repository readableFileNames. filenames do: [ :each | self read: each ] MCproject>>read: aFileName "Needs a rate limiter!!!" |mcVersion| mcVersion := repository loadNotCachedVersionFromFileNamed: aFileName. mcVersion snapshot. self parse: mcVersion. repository flushCache For each unique package in those MCVersions, I add a MCPackageInfo, defined as Object subclass: #MCPackageInfo instanceVariableNames: 'packageName packageVersions' classVariableNames: '' package: 'MonticelloProjects' MCProject>>parse: aVersion |info| info := self ensureInfo: aVersion package. info addVersion: aVersion in: self a MCPackageVersion then stores the info and the unique definition that is stored in the project, eliminating the duplicates. MCPackageInfo>>addVersion: aMcVersion in: aProject |packageVersion| packageVersion := MCPackageVersion new info: aMcVersion info; yourself. self packageVersions add: packageVersion. aMcVersion snapshot definitions do: [ :aDefinition | packageVersion definitions add: (aProject definitions add: aDefinition) ]. As long as #= and #hash are correctly defined for all MCDefinitions, this should make it possible too eliminate all duplicate definitions and have a full history. On my Documentation repo this already saves a factor 7, when saving this compressed as a Fuel file. On large repositories with a high change rate (Roassal2?) the compression will be significantly higher. There are several other normalizations that can reduce the size further: - make recategorization explicit - normalize MCVersionInfo data: explicit author, compact timestamp. I'd be interested in further ideas for this, and situations where this approach wouldn't work. Stephan

2 4

Re: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support))
by H. Hirzel Dec. 15, 2015

Dec. 15, 2015

Dale Thank you for your answer with links to the ICU library and the notes about classes in Gemstone. Noteworthy that you have a class Utf8 as a subclass of ByteArray. I understand that Gemstone uses the ICU library and thus does not implement the algorithms in Smalltalk. I am currently looking into what the ICU library provides. I found as well a Ruby library [2] which implements CLDR [3] It has methods like this "Alphabetize a list using regular Ruby sort:" $> ["Art", "Wasa", "Ãlg", "Ved"].sort $> ["Art", "Ved", "Wasa", "Ãlg"] Alphabetize a list using TwitterCLDRâs locale-aware sort: $> ["Art", "Wasa", "Ãlg", "Ved"].localize(:de).sort.to_a $> ["Ãlg", "Art", "Ved", "Wasa"] I hope that given such an example it would not be too difficult to reimplement a similar sort algorithm in Squeak/Cuis/Pharo. Currently the interest is in getting sorting done in a cross-dialect-way. --Hannes [2] https://blog.twitter.com/2012/twittercldr-improving-internationalization-su… [3] Unicode Common Locale Data Repository http://cldr.unicode.org/index On 12/7/15, Dale Henrichs <dale.henrichs(a)gemtalksystems.com> wrote: > Hannes, > > For GemStone, we are using the ICU library[1]. We have Unicode7, > Unicode16 and Unicode32 classes (subclasses of CharacterCollection) for > internal Strings and the class Utf8 (a subclass of ByteArray) for Utf8 > encoded strings ... > > The ICU library provides the primitive implementations for working with > the Unicode* and Utf8 classes > > When we started considering Unicode support, we looked at what it would > take to support collation - our main reason for looking at Unicode in > the first place) -- and we saw just how complicated the collation rules > can be[2], we were glad to see that someone had already done the hard > work[1]... > > Reconciling our legacy String implementations (String, DoubleByteString, > and QuadByteString) with the Unicode* classes was also interesting, > because the rules for Unicode equality and our legacy equality > implementation were not quite compatible. > > If you are interested in more information, I can share additional > details ... > > Dale > > [1] http://site.icu-project.org/ > [2] http://unicode.org/reports/tr10/ > > On 12/07/2015 11:54 AM, H. Hirzel wrote: >> Hello >> >> According to http://www.unicode.org/cldr/charts/27/collation/de.html the >> German >> phonebook sort order is >> >> a A Ã¤ Ã ÄÌ ÄÌ Ç Ç áº¡Ì áº Ì á¸Ì á¸Ì b B c C d D e E f F g G h H i I j J k K >> l L m M n N o O Ã¶ Ã Ç«Ì ÇªÌ È« Èª Æ¡Ì Æ Ì á»£Ì á»¢Ì á»Ì á»Ì p P q Q r R s S ss Ã t >> T u U Ã¼ Ã Ç Ç Ç Ç Ç Ç Å³Ì Å²Ì Ç Ç Æ°Ì Æ¯Ì á»±Ì á»°Ì á»¥Ì á»¤Ì á¹³Ì á¹²Ì á¹·Ì á¹¶Ì á¹µÌ á¹´Ì v >> V w W x X y Y z Z >> >> I wonder why it looks like this. A lot of characters which never >> appear in a German text. >> >> >> For Spanish there is 'traditional' and 'standard' >> >> http://www.unicode.org/cldr/charts/27/collation/es.html >> >> standard a A Ã¡ Ã b B c C d D e E Ã© Ã f F g G h H i I Ã Ã j J k K l L m >> M n N Ã± Ã ÅÌ ÅÌ á¹Ì á¹Ì á¹Ì á¹Ì á¹Ì á¹Ì o O Ã³ Ã p P q Q r R s S t T u U Ãº Ã >> Ã¼ Ã v V w W x X y Y z Z >> >> traditional a A Ã¡ Ã b B c C ch Ch CH cÄ¥ CÄ¥ CÄ¤ cÈ CÈ CÈ cá¸§ Cá¸§ Cá¸¦ cá¸£ Cá¸£ >> Cá¸¢ cá¸© Cá¸© Cá¸¨ cá¸¥ Cá¸¥ Cá¸¤ cá¸« Cá¸« Cá¸ª cáº Cáº d D e E Ã© Ã f F g G h H i I Ã Ã j >> J k K l L ll Ll LL lÄº LÄº LÄ¹ lÄ¾ LÄ¾ LÄ½ lÄ¼ LÄ¼ LÄ» lá¸· Lá¸· Lá¸¶ lá¸¹ Lá¸¹ Lá¸¸ lá¸½ Lá¸½ >> Lá¸¼ lá¸» Lá¸» Lá¸º m M n N Ã± Ã ÅÌ ÅÌ á¹Ì á¹Ì á¹Ì á¹Ì á¹Ì á¹Ì o O Ã³ Ã p P q Q r R s >> S t T u U Ãº Ã Ã¼ Ã v V w W x X y Y z Z >> >> And French is not easily found >> http://www.unicode.org/cldr/charts/27/collation/index.html >> or seems to be defined elsewhere >> >> http://unicode.org/repos/cldr/tags/release-27/common/collation/fr.xml >> >> Suggestions and hints are welcome >> >> --Hannes >> _______________________________________________ >> Cuis mailing list >> Cuis(a)jvuletich.org >> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org > > > _______________________________________________ > Cuis mailing list > Cuis(a)jvuletich.org > http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org >

8 24

[pharo-project/pharo-core] 6a48f4: 50499
by GitHub Dec. 15, 2015

Dec. 15, 2015

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 6a48f484555b8ff8ae9faa567347dc8615bebbf2 https://github.com/pharo-project/pharo-core/commit/6a48f484555b8ff8ae9faa56… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-15 (Tue, 15 Dec 2015) Changed paths: M FFI-NB.package/FFIMethodRegistry.class/class/class initialization/initialize.st A FFI-NB.package/FFIMethodRegistry.class/class/class initialization/initializeStartUpList.st M FFI-NB.package/FFIMethodRegistry.class/instance/accessing/checkPlatform.st A FFI-NB.package/FFIMethodRegistry.class/instance/accessing/reset.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50498.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50499.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50498.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50499.st Log Message: ----------- 50499 17241 ffi-nb is not cleaning compiled methods. then they fail when platform changes https://pharo.fogbugz.com/f/cases/17241 http://files.pharo.org/image/50/50499.zip

2 1

[pharo-project/pharo-core]
by GitHub Dec. 15, 2015

Dec. 15, 2015

Branch: refs/tags/50499 Home: https://github.com/pharo-project/pharo-core

1 0

review, validation and test development for GenericGF
by Robert Withers Dec. 15, 2015

Dec. 15, 2015

Hi all, sticking with the distribution for discussing Crypto in the base, I am currently trying to stabilize a port of Reed-Solomon such that FEC can be added to SecureSession. I am doing this work in SecureSession. I just posted an update of the GenericGF class, with the Java implementation attached here. I need this critical code reviewed and tested. You need Cryptography and SecureSession to be included in the catalog. If you can help me with the former, I could help with the latter, if that suits you. Please let me know! Please load Cryptography(-Pharo) and SecureSession.23 for review. "ported & refactored GenericGF from the java code. This includes reindexing from 0 to 1. Having no tests, hopefully someone can give a review of this specific class. Where some tests to be developed at this scope, that would be stellar." I appreciate. -- . .. .. ^,^ best, robert

1 1

Re: [Pharo-dev] Unicode Support - Normalization NFD -- #decomposedStringWithCanonicalMapping
by H. Hirzel Dec. 15, 2015

Dec. 15, 2015

On 12/13/15, stepharo <stepharo(a)free.fr> wrote: > >> It is best and (mostly) correct to think of a Unicode string as a sequence >> of Unicode characters, each defined/identified by a code point (out of >> 10.000s covering all languages). That is what we have today in Pharo (with >> the distinction between ByteString and WideString as mostly invisible >> implementation details). >> >> To encode Unicode for external representation as bytes, we use UTF-8 like >> the rest of the modern world. >> So far, so good. >> ...> > > like me ;) > I will wait for a conclusion with code :) > > Stef > Some code with illustration by the e acute example see below >> Why then is there confusion about the seemingly simple concept of a >> character ? Because Unicode allows different ways to say the same thing. >> The simplest example in a common language is (the French letter Ã©) is >> >> LATIN SMALL LETTER E WITH ACUTE [U+00E9] >> >> which can also be written as >> >> LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT [U+0301] >> >> The former being a composed normal form, the latter a decomposed normal >> form. (And yes, it is even much more complicated than that, it goes on for >> 1000s of pages). >> >> In the above example, the concept of character/string is indeed fuzzy. >> >> HTH, >> >> Sven The text below shows how to deal with the Unicode e acute example brought up by Sven in terms of comparing strings. Currently Pharo and Cuis do not do Normalization of strings. Limited support is in Squeak. It will be shown how NFD normalization may be implemented. Swift programming language ----------------------------------------- How does the Swift programming language [1] deal with Unicode strings? // "Voulez-vous un cafÃ©?" using LATIN SMALL LETTER E WITH ACUTE let eAcuteQuestion = "Voulez-vous un caf\u{E9}?" // "Voulez-vous un café?" using LATIN SMALL LETTER E and COMBINING ACUTE ACCENT let combinedEAcuteQuestion = "Voulez-vous un caf\u{65}\u{301}?" if eAcuteQuestion == combinedEAcuteQuestion { print("These two strings are considered equal") } // prints "These two strings are considered equal" The equality operator uses the NFD (Normalization Form Decomposed)[2] form for the comparison appyling a method #decomposedStringWithCanonicalMapping[3] Squeak / Pharo ----------------------- Comparison without NFD [3] "Voulez-vous un cafÃ©?" eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter asString, '?'. eAcuteQuestion = combinedEAcuteQuestion false eAcuteQuestion == combinedEAcuteQuestion false The result is false. A Unicode conformant application however should return *true*. Reason for this is that Squeak / Pharo strings are not put into NFD before testing for equality = Squeak Unicode strings may be tested for Unicode conformant equality by converting them to NFD before testing. Squeak using NFD asDecomposedUnicode[4] transforms a string into NFD for cases where a Unicode code point if decomposed, is decomposed only to two code points [5]. This is so because when initializing [6] the Unicode Character Database in Squeak this is a limitation imposed by the code which reads UnicodeData.txt [7][8]. This is not a necessary limitation. The code may be rewritten at the price of a more complex implementation of #asDecomposedUnicode. "Voulez-vous un cafÃ©?" eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter asString, '?'. eAcuteQuestion asDecomposedUnicode = combinedEAcuteQuestion asDecomposedUnicode true Conclusion ------------------ Implementing a method like #decomposedStringWithCanonicalMapping (swift) which puts a string into NFD (Normalization Form D) is an important building block towards better Unicode compliance. A Squeak proposal is given by [4]. It needs to be reviewed.extended. It should probably be extended for cases where there are more than two code points in the decomposed form (3 or more?) The implementing of NFD comparison gives us an equality test for a comparatively small effort for simple cases covering a large number of use cases (Languages using the Latin script). The algorithm is table driven by the UCD [8]. From this follows an simple but important fact for conformant implementations need runtime access to information from the Unicode Character Database [UCD][9]. [1] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swif… [2] http://www.unicode.org/glossary/#normalization_form_d [3] https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Found… [4] String asDecomposedUnicode http://wiki.squeak.org/squeak/6250 [5] http://www.unicode.org/glossary/#code_point [6] Unicode initialize http://wiki.squeak.org/squeak/6248 [7] http://www.unicode.org/Public/UNIDATA/UnicodeData.txt [8] Unicode Character Database documentation http://unicode.org/reports/tr44/ [9] http://www.unicode.org/reports/tr23/

1 0

Windows compilation performance issues
by Thierry Goubier Dec. 15, 2015

Dec. 15, 2015

Hi all, I just did a teaching lab session today in Brest, and I noticed that Pharo on Microsoft Windows compiles very slowly (loading Roassal2 stable on Pharo4). What Windows tools shows is a very low cpu load, and a high disk load with very low bandwidth (0.1Mbit/s instead of 16Mbit/s). Does anybody knows why? Thierry

3 2