- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Unicode Support // e acute example --> decomposition in Pharo?

by Sven Van Caekenberghe

Hi Hannes, My detailed comments/answers below, after quoting 2 of your emails: > On 10 Dec 2015, at 22:17, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: > > Hello Sven > > On 12/9/15, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> The simplest example in a common language is (the French letter Ã©) is >> >> LATIN SMALL LETTER E WITH ACUTE [U+00E9] >> >> which can also be written as >> >> LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT [U+0301] >> >> The former being a composed normal form, the latter a decomposed normal >> form. (And yes, it is even much more complicated than that, it goes on for >> 1000s of pages). >> >> In the above example, the concept of character/string is indeed fuzzy. >> >> HTH, >> >> Sven > > Thanks for this example. I have created a wiki page with it > > I wonder what the Pharo equivalent is of the following Squeak expression > > $Ã© asString asDecomposedUnicode > > Regards > > Hannes You also wrote: > The text below shows how to deal with the Unicode e acute example > brought up by Sven in terms of comparing strings. Currently Pharo and > Cuis do not do Normalization of strings. Limited support is in Squeak. > It will be shown how NFD normalization may be implemented. > > > Swift programming language > ----------------------------------------- > > How does the Swift programming language [1] deal with Unicode strings? > > // "Voulez-vous un cafÃ©?" using LATIN SMALL LETTER E WITH ACUTE > let eAcuteQuestion = "Voulez-vous un caf\u{E9}?" > > // "Voulez-vous un café?" using LATIN SMALL LETTER E and > COMBINING ACUTE ACCENT > let combinedEAcuteQuestion = "Voulez-vous un caf\u{65}\u{301}?" > > if eAcuteQuestion == combinedEAcuteQuestion { > print("These two strings are considered equal") > } > // prints "These two strings are considered equal" > > The equality operator uses the NFD (Normalization Form Decomposed)[2] > form for the comparison appyling a method > #decomposedStringWithCanonicalMapping[3] > > > Squeak / Pharo > ----------------------- > > Comparison without NFD [3] > > > "Voulez-vous un cafÃ©?" > eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. > combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter > asString, '?'. > > > eAcuteQuestion = combinedEAcuteQuestion > false > > eAcuteQuestion == combinedEAcuteQuestion > false > > The result is false. A Unicode conformant application however should > return *true*. > > Reason for this is that Squeak / Pharo strings are not put into NFD > before testing for equality = > > > Squeak Unicode strings may be tested for Unicode conformant equality > by converting them to NFD before testing. > > > > Squeak using NFD > > asDecomposedUnicode[4] transforms a string into NFD for cases where a > Unicode code point if decomposed, is decomposed only to two code > points [5]. This is so because when initializing [6] the Unicode > Character Database in Squeak this is a limitation imposed by the code > which reads UnicodeData.txt [7][8]. This is not a necessary > limitation. The code may be rewritten at the price of a more complex > implementation of #asDecomposedUnicode. > > "Voulez-vous un cafÃ©?" > eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. > combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter > asString, '?'. > > > eAcuteQuestion asDecomposedUnicode = > combinedEAcuteQuestion asDecomposedUnicode > > true > > > > Conclusion > ------------------ > > Implementing a method like #decomposedStringWithCanonicalMapping > (swift) which puts a string into NFD (Normalization Form D) is an > important building block towards better Unicode compliance. A Squeak > proposal is given by [4]. It needs to be reviewed.extended. > > It should probably be extended for cases where there are more than > two code points in the decomposed form (3 or more?) > > The implementing of NFD comparison gives us an equality test for a > comparatively small effort for simple cases covering a large number of > use cases (Languages using the Latin script). > > The algorithm is table driven by the UCD [8]. From this follows an > simple but important fact for conformant implementations need runtime > access to information from the Unicode Character Database [UCD][9]. > > > [1] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swif… > [2] http://www.unicode.org/glossary/#normalization_form_d > [3] https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Found… > [4] String asDecomposedUnicode http://wiki.squeak.org/squeak/6250 > [5] http://www.unicode.org/glossary/#code_point > [6] Unicode initialize http://wiki.squeak.org/squeak/6248 > [7] http://www.unicode.org/Public/UNIDATA/UnicodeData.txt > [8] Unicode Character Database documentation http://unicode.org/reports/tr44/ > [9] http://www.unicode.org/reports/tr23/ Today, we have a Unicode and CombinedCharacter class in Pharo, and there is different but similar Unicode code in Squeak. These are too simple (even though they might work, partially). The scope of the original threads is way too wide: a new string type, normalisation, collation, being cross dialect, mixing all kinds of character and encoding definitions. All interesting, but not much will come out of it. But the point that we cannot leave proper text string handling to an outside library is indeed key. That is why a couple of people in the Pharo community (myself included) started an experimental, proof of concept, prototype project, that aims to improve Unicode support. We will announce it to a wider public when we feel we have something to show for. The goal is in the first place to understand and implement the fundamental algorithms, starting with the 4 forms of Normalisation. But we're working on collation/sorting too. This work is of course being done for/in Pharo, using some of the facilities only available there. It probably won't be difficult to port, but we can't be bothered with probability right now. What we started with is loading UCD data and making it available as a nice objects (30.000 of them). So now you can do things like $Ã© unicodeCharacterData. => "U+00E9 LATIN SMALL LETTER E WITH ACUTE (LATIN SMALL LETTER E ACUTE)" $Ã© unicodeCharacterData uppercase asCharacter. => "$Ã" $Ã© unicodeCharacterData decompositionMapping. => "#(101 769)" There is also a cool GT Inspector view: Next we started implementing a normaliser. It was rather easy to get support for simpler languages going. The next code snippets use explicit code arrays, because copying decomposed diacritics to my mail client does not work (they get automatically composed), in a Pharo Workspace this does work nicely with plain strings. The higher numbers are the diacritics. (normalizer decomposeString: 'les Ã©lÃ¨ves FranÃ§ais') collect: #codePoint as: Array. => "#(108 101 115 32 101 769 108 101 768 118 101 115 32 70 114 97 110 99 807 97 105 115)" (normalizer decomposeString: 'DÃ¼sseldorf KÃ¶nigsallee') collect: #codePoint as: Array. => "#(68 117 776 115 115 101 108 100 111 114 102 32 75 111 776 110 105 103 115 97 108 108 101 101)" normalizer composeString: (#(108 101 115 32 101 769 108 101 768 118 101 115 32 70 114 97 110 99 807 97 105 115) collect: #asCharacter as: String). => "'les Ã©lÃ¨ves FranÃ§ais'" normalizer composeString: (#(68 117 776 115 115 101 108 100 111 114 102 32 75 111 776 110 105 103 115 97 108 108 101 101) collect: #asCharacter as: String). => "'DÃ¼sseldorf KÃ¶nigsallee'" However, the real algorithm following the official specification (and other elements of Unicode that interact with it) is way more complicated (think about all those special languages/scripts out there). We're focused on understanding/implementing that now. Next, unit tests were added (of course). As well as a test that uses http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt to run about 75.000 individual test cases to check conformance to the official Unicode Normalization specification. Right now (with super cool hangul / jamo code by Henrik), we hit the following stats: #testNFC 16998/18593 (91.42%) #testNFD 16797/18593 (90.34%) #testNFKC 13321/18593 (71.65%) #testNFKD 16564/18593 (89.09%) Way better than the naive implementations, but not yet there. We are also experimenting and thinking a lot about how to best implement all this, trying out different models/ideas/apis/representations. It will move slowly, but you will hear from us again in the coming weeks/months. Sven PS: Pharo developers with a good understanding of this subject area that want to help, let me know and we'll put you in the loop. Hacking and specification reading are required ;-)

Dec. 16, 2015

Re: [Pharo-dev] [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support))

by H. Hirzel

On 12/16/15, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Todd, > > On Tue, Dec 15, 2015 at 3:46 PM, Todd Blanchard <tblanchard(a)mac.com> wrote: > >> Hi Eliot, >> >> On Dec 15, 2015, at 13:46, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >> >> Just so you know, I will dig my heels in as deeply as I am able to >> prevent >> the use of C++ libraries in the VM. It destroys the simulator, which is >> the most important thing we have for VM development productivity. As far >> as I'm concerned any use of external libraries to implement core >> functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) >> are >> built upon. >> >> >> OK, I defer to you because you certainly know more about the VM internals >> and what does and doesn't work well than anyone else. >> >> So I guess I would like to know your recommendation for 1) how best to >> store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we >> get to worry about endian). >> > > Raw Unicode, either as 8-bit, 16-bit or 32-bit. When creating a String it > should start as an 8-bit-per-Unicode-character string. Attempts to store > Character values that won't fit cause the String to become a String whose > element size is large enough to accommodate the character. This is the case: see tests here http://wiki.squeak.org/squeak/6316 In Spur, > become: is cheap so this growth pays only for the reallocation and copying > of the at a, not for an expensive heap scan necessary to do the become:. Could you elaborate on this please? > > >> Bearing in mind that both representations are variable length and so >> while >> accessing the n'th byte/word is O(1), accessing the n'th character is >> necessarily O(n) unless you know you have no surrogates in your string. >> > > Right, so UTF-8 and UTF-16 are not convenient representations and to be > provided only for interchange. +1 Squeak/Pharo uses 8bit/32bit (UTF-32) internally and UTF-8 externally. There are converters for UTF-8 and UTF-16. > >> >> Also...since NSString has been mentioned...it is worth noting that >> NSString is built atop CFString (source code here: >> https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c) which >> does a fair job of optimizing memory by using bytes where it can and >> shorts >> where it cannot. It is also worth noting that characterAt: actually does >> the wrong thing, since it assumes characters are no bigger than FFFF >> rather >> than 10FFFF. >> > > Yes, and Squeak (and AFAIA, Pharo) has been doing this for ages. If one > has become: it is very easy to manage. Now with Spur not only do we have > become:, we have a fairly fast become:. > > Does this make sense? > > >> Also...I'll just toss in this very nice article on unicode and how >> NSString deals with it. >> https://www.objc.io/issues/9-strings/unicode/ >> >> -Todd Blanchard >> > > _,,,^..^,,,_ > best, Eliot >

Dec. 16, 2015

Re: [Pharo-dev] AsmJit

by philippe.back＠highoctane.be

Couldn't we reuse the AsmJIT DSL and generate some C code with Asm directives through Slang? And call that C code through FFI? Seems cleaner and more debuggable. And could be moved to x64. Phil On Dec 16, 2015 6:49 AM, "ClÃ©ment Bera" <bera.clement(a)gmail.com> wrote: > > > > 2015-12-15 20:22 GMT+01:00 Jan Vrany <jan.vrany(a)fit.cvut.cz>: >> >> Hi Esteban, Eliot, Clemont >> >> thanks very much for elaborate answer. I'm aware of all what you said >> (well, most :-), but my actual needs are very low and primitive. >> >> All I need now is something that allows me to turn x86_64 assembly >> into a machine code using a sane Smalltalk API, which I can later copy >> to VM code space. Nothing more, nothing less :-) I do not fancy >> spending time writing yet-another-x86-assembler hence my interest in >> asmjit. > > > As far as I know AsmJIT has stable support for x86 but not x86_64 >> >> >> Thanks! Jan >> >> >> >> On Tue, 2015-12-15 at 18:34 +0100, ClÃ©ment Bera wrote: >> > We're getting away from Jan initial question here ... >> > >> > I think the point here is that Cog has: >> > - 2 stable back ends: ARMv5 and x86 >> > - 2 backends stable in the simulator with production planned for ~ >> > April: x64 and MIPS >> > and Tim is willing to do the ARMv8 backend. >> > >> > All the 4, and soon 5, cog back ends are maintained: >> > - x86 and x64 maintained by Eliot >> > - MIPS maintained by Ryan >> > - ARMv5 and soon ARMv8 maintained by Tim >> > >> > Each backend requires months of work. >> > >> > Do you want to maintain 10 backends instead of 5 ? Do want to spend >> > time to implement 5 backends or 10 ? >> > >> > I don't. Only the x86 backend is duplicated right now in AsmJIT. >> > >> > So one idea is to reuse Cog's backends from the image instead of >> > AsmJIT. To do so, we can use encodings available in the sista >> > extended instruction set to tell Cog what machine code to generate. A >> > project named uFFI aims, among other things, at doing this, by >> > providing a unified interface between the Cog backends and AsmJIT. >> > >> > >> > 2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: >> > > Hi Jan, >> > > >> > > > On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany(a)fit.cvut.cz> >> > > wrote: >> > > > >> > > > Hi guys, >> > > > >> > > > two queations: >> > > > >> > > > (i) Is AsmJit going to be developed any more or it's abandoned >> > > > as well as native boost? >> > > >> > > AsmJIT is effectively being abandoned but NativeBoost is not. >> > > >> > > The key limitation of AsmJIT is that it was not designed to be >> > > cross platform; it is effectively an x86 assembler. As such it's >> > > use gets in the way of ARM and x86_64 (I am currently getting the C >> > > version of 64-bit Cog Spur working on x86_64, given that it is >> > > working in the simulator). >> > > >> > > Another limitation is that it doesn't play that well with the VM's >> > > JIT. Igor and I never managed to work on integrating it better. >> > > The VM's job is managing code and Igor's approach was to hack; >> > > eliminating execution protection in the entire heap, instead of >> > > extending the support that either the Alien plugin's callback >> > > support or the JIT's executable method zone provides. Making the >> > > entire heap executable is /not/ a sensible approach. >> > > >> > > But there is a better way! A key component of the Sista adaptive >> > > optimizer/speculative inlined that ClÃ©ment is currently stabilizing >> > > (!!) is a set of bytecodes that encode unsafe operations like >> > > at:put: without bounds, type or store checks. For example, the >> > > normal at:put: is about a hundred instructions, checks for >> > > smallinteger indices, differentiates between byte, 32-but long and >> > > pointer objects and does a store check. But one of the Sista codes >> > > for at:put: generates about two instructions, one to adjust the >> > > index, the other to do the store. Distaste job is to analyze code >> > > and inline methods using these unsafe bytecodes where they are >> > > proven to be safe, hence increasing performance. >> > > >> > > Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, >> > > being executed by the VM, can work on an interpreter VM or be >> > > converted to machine code by the JIT. >> > > >> > > So our plan is to extend these bytecodes with ones that support >> > > marshaling arguments for NativeBoost calls. Ronie Salgado has >> > > already extended his lowcode scheme to define these instructions >> > > and sometime soon (hopefully 2016) we shall rewrite NativeBoost to >> > > target these bytecodes. >> > > >> > > HTH >> > > >> > > > (ii)Where can I find latest AsmJit? I'm properly confused: >> > > > >> > > > * Is is the one in latest Pharo 4.0 (5.0) image? >> > > > * Is it the one here: http://smalltalkhub.com/#!/~Pharo/AsmJi >> > > t ? >> > > > (the one in the image seem to be based on completely >> > > disjunct >> > > > set of .mcz than those in the repo above). >> > > > >> > > > Best, Jan >> > > >> > > _,,,^..^,,,_ (phone) >> > > >> >

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by Esteban Lorenzano

Hi, > On 16 Dec 2015, at 08:21, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Hi Esteban, > > On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <estebanlm(a)gmail.com <mailto:estebanlm@gmail.com>> wrote: > >> >>> On 15 Dec 2015, at 13:18, Norbert Hartl <norbert(a)hartl.name <mailto:norbert@hartl.name>> wrote: >>> >>> We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. >>> I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel? >> >> I think is not possible, but I will let fuel maintainers to have the last word :) > > > Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate. Oh wellâ¦ of course is possible. What I meant is that I think current implementation of Fuel does not support it, just that :) but again, Iâm not sure: Martin and Mariano can for sure say something more detailed about :) (problem is Martin is on vacations after getting his PhD and Mariano didnât followed in close the spur migration) cheers, Esteban > > In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not. > >> >> Esteban >> >>> >>> thanks, >>> >>> Norbert >>> >>> >>> primitive #value: in Character class failed >>> Stacktrace >>> >>> Character class(Object)>>primitiveFailed: >>> Character class(Object)>>primitiveFailed >>> Character class>>value: >>> Character class>>materializeFrom: >>> FLHookPrimitiveCluster>>materializeInstanceWith: >>> FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: >>> FLMaterialization>>clusterInstancesStep >>> [ self clusterInstancesStep ] in FLMaterialization>>instancesStep >>> SmallInteger(Integer)>>timesRepeat: >>> FLMaterialization>>instancesStep >>> FLMaterialization>>run >>> [ :aDecoder | >>> (FLMaterialization with: aDecoder) >>> run; >>> yourself ] in FLMaterializer>>setDefaultMaterialization >>> FLMaterializer>>materializeFrom: >>> FLMaterializer class>>materializeFromByteArray: >>> MAPExampleModels class>>readModelFromSelector: >>> MAPExampleModels class>>readModelNamed: >>> [ self readModelNamed: aString ] in MAPExampleModels class>>named: >>> [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: >>> Dictionary>>at:ifAbsent: >>> Dictionary>>at:ifAbsentPut: >>> MAPExampleModels class>>named: >>> MAPTest>>model >>> MAPTest>>tcapModule >>> MAPTest>>testInsertSubscriberDataBitStringAccess >>> [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: >>> [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: >>> Dictionary>>at:ifAbsent: >>> Dictionary>>at:ifAbsentPut: >>> Given>>produceReturnValueAt: >>> MAPTest(Phexample)>>performTest >>> >>> >>

Dec. 16, 2015

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/50500 Home: https://github.com/pharo-project/pharo-core

Dec. 16, 2015

[pharo-project/pharo-core] 8ec2c9: 50500

by GitHub

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 8ec2c9995556d7decba6455aa9d8eb44fe6575ef https://github.com/pharo-project/pharo-core/commit/8ec2c9995556d7decba6455a… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-16 (Wed, 16 Dec 2015) Changed paths: M FFI-NB.package/FFICalloutAPI.class/instance/action/function_module_.st A FFI-NB.package/FFIMethodRegistry.class/class/cleanup/cleanUp_.st M FFI-NB.package/FFIMethodRegistry.class/instance/accessing/checkPlatform.st M FFI-NB.package/FFIMethodRegistry.class/instance/accessing/reset.st M FFI-NB.package/extension/ExternalData/instance/+.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50499.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50500.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50499.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50500.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st M System-OSEnvironments.package/Win32Environment.class/instance/private/getEnv_.st Log Message: ----------- 50500 17239 monticello: understand to new immediate classes (spur migration) https://pharo.fogbugz.com/f/cases/17239 http://files.pharo.org/image/50/50500.zip

Dec. 16, 2015

Re: [Pharo-dev] Are Critical Tools Becoming Too Brittle?

by Max Leske

> On 15 Dec 2015, at 23:21, Sean P. DeNigris <sean(a)clipperadams.com> wrote: > > I just tried to inspect a domain object and I froze my image and had to force > quit. How do I debug this? And, more importantly, when an error is > encountered in a critical, basic tool like this, should there be some > reasonable bare-bones alternative, like the emergency debugger? Not being > able to inspect an object without crashing seems like a pretty crippling > situationâ¦ Did you send #basicInspect? That should still give you the original minimal inspector. > > > > ----- > Cheers, > Sean > -- > View this message in context: http://forum.world.st/Are-Critical-Tools-Becoming-Too-Brittle-tp4867206.html > Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com. >

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by Eliot Miranda

Hi Esteban, > On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > >> On 15 Dec 2015, at 13:18, Norbert Hartl <norbert(a)hartl.name> wrote: >> >> We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. >> I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel? > > I think is not possible, but I will let fuel maintainers to have the last word :) Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate. In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not. > > Esteban > >> >> thanks, >> >> Norbert >> >> >> primitive #value: in Character class failed >> Stacktrace >> >> Character class(Object)>>primitiveFailed: >> Character class(Object)>>primitiveFailed >> Character class>>value: >> Character class>>materializeFrom: >> FLHookPrimitiveCluster>>materializeInstanceWith: >> FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: >> FLMaterialization>>clusterInstancesStep >> [ self clusterInstancesStep ] in FLMaterialization>>instancesStep >> SmallInteger(Integer)>>timesRepeat: >> FLMaterialization>>instancesStep >> FLMaterialization>>run >> [ :aDecoder | >> (FLMaterialization with: aDecoder) >> run; >> yourself ] in FLMaterializer>>setDefaultMaterialization >> FLMaterializer>>materializeFrom: >> FLMaterializer class>>materializeFromByteArray: >> MAPExampleModels class>>readModelFromSelector: >> MAPExampleModels class>>readModelNamed: >> [ self readModelNamed: aString ] in MAPExampleModels class>>named: >> [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: >> Dictionary>>at:ifAbsent: >> Dictionary>>at:ifAbsentPut: >> MAPExampleModels class>>named: >> MAPTest>>model >> MAPTest>>tcapModule >> MAPTest>>testInsertSubscriberDataBitStringAccess >> [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: >> [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: >> Dictionary>>at:ifAbsent: >> Dictionary>>at:ifAbsentPut: >> Given>>produceReturnValueAt: >> MAPTest(Phexample)>>performTest >

Dec. 16, 2015

Re: [Pharo-dev] AsmJit

by ClÃ©ment Bera

2015-12-15 20:22 GMT+01:00 Jan Vrany <jan.vrany(a)fit.cvut.cz>: > Hi Esteban, Eliot, Clemont > > thanks very much for elaborate answer. I'm aware of all what you said > (well, most :-), but my actual needs are very low and primitive. > > All I need now is something that allows me to turn x86_64 assembly > into a machine code using a sane Smalltalk API, which I can later copy > to VM code space. Nothing more, nothing less :-) I do not fancy > spending time writing yet-another-x86-assembler hence my interest in > asmjit. > As far as I know AsmJIT has stable support for x86 but not x86_64 > > Thanks! Jan > > > > On Tue, 2015-12-15 at 18:34 +0100, ClÃ©ment Bera wrote: > > We're getting away from Jan initial question here ... > > > > I think the point here is that Cog has: > > - 2 stable back ends: ARMv5 and x86 > > - 2 backends stable in the simulator with production planned for ~ > > April: x64 and MIPS > > and Tim is willing to do the ARMv8 backend. > > > > All the 4, and soon 5, cog back ends are maintained: > > - x86 and x64 maintained by Eliot > > - MIPS maintained by Ryan > > - ARMv5 and soon ARMv8 maintained by Tim > > > > Each backend requires months of work. > > > > Do you want to maintain 10 backends instead of 5 ? Do want to spend > > time to implement 5 backends or 10 ? > > > > I don't. Only the x86 backend is duplicated right now in AsmJIT. > > > > So one idea is to reuse Cog's backends from the image instead of > > AsmJIT. To do so, we can use encodings available in the sista > > extended instruction set to tell Cog what machine code to generate. A > > project named uFFI aims, among other things, at doing this, by > > providing a unified interface between the Cog backends and AsmJIT. > > > > > > 2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: > > > Hi Jan, > > > > > > > On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany(a)fit.cvut.cz> > > > wrote: > > > > > > > > Hi guys, > > > > > > > > two queations: > > > > > > > > (i) Is AsmJit going to be developed any more or it's abandoned > > > > as well as native boost? > > > > > > AsmJIT is effectively being abandoned but NativeBoost is not. > > > > > > The key limitation of AsmJIT is that it was not designed to be > > > cross platform; it is effectively an x86 assembler. As such it's > > > use gets in the way of ARM and x86_64 (I am currently getting the C > > > version of 64-bit Cog Spur working on x86_64, given that it is > > > working in the simulator). > > > > > > Another limitation is that it doesn't play that well with the VM's > > > JIT. Igor and I never managed to work on integrating it better. > > > The VM's job is managing code and Igor's approach was to hack; > > > eliminating execution protection in the entire heap, instead of > > > extending the support that either the Alien plugin's callback > > > support or the JIT's executable method zone provides. Making the > > > entire heap executable is /not/ a sensible approach. > > > > > > But there is a better way! A key component of the Sista adaptive > > > optimizer/speculative inlined that ClÃ©ment is currently stabilizing > > > (!!) is a set of bytecodes that encode unsafe operations like > > > at:put: without bounds, type or store checks. For example, the > > > normal at:put: is about a hundred instructions, checks for > > > smallinteger indices, differentiates between byte, 32-but long and > > > pointer objects and does a store check. But one of the Sista codes > > > for at:put: generates about two instructions, one to adjust the > > > index, the other to do the store. Distaste job is to analyze code > > > and inline methods using these unsafe bytecodes where they are > > > proven to be safe, hence increasing performance. > > > > > > Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, > > > being executed by the VM, can work on an interpreter VM or be > > > converted to machine code by the JIT. > > > > > > So our plan is to extend these bytecodes with ones that support > > > marshaling arguments for NativeBoost calls. Ronie Salgado has > > > already extended his lowcode scheme to define these instructions > > > and sometime soon (hopefully 2016) we shall rewrite NativeBoost to > > > target these bytecodes. > > > > > > HTH > > > > > > > (ii)Where can I find latest AsmJit? I'm properly confused: > > > > > > > > * Is is the one in latest Pharo 4.0 (5.0) image? > > > > * Is it the one here: http://smalltalkhub.com/#!/~Pharo/AsmJi > > > t ? > > > > (the one in the image seem to be based on completely > > > disjunct > > > > set of .mcz than those in the repo above). > > > > > > > > Best, Jan > > > > > > _,,,^..^,,,_ (phone) > > > > >

Dec. 16, 2015

Re: [Pharo-dev] Are Critical Tools Becoming Too Brittle?

by Tudor Girba

Hi, Indeed, please help us isolate that case. Cheers, Doru > On Dec 15, 2015, at 11:49 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > I assume you are using the GT inspector. Do you have custom gt inspector presentations ? Are you sure your #printString is OK ? > > There is also a 'Basic Inspect It' menu item ... > > Can you isolate it so that we can see it, try it ? > >> On 15 Dec 2015, at 23:21, Sean P. DeNigris <sean(a)clipperadams.com> wrote: >> >> I just tried to inspect a domain object and I froze my image and had to force >> quit. How do I debug this? And, more importantly, when an error is >> encountered in a critical, basic tool like this, should there be some >> reasonable bare-bones alternative, like the emergency debugger? Not being >> able to inspect an object without crashing seems like a pretty crippling >> situation... >> >> >> >> ----- >> Cheers, >> Sean >> -- >> View this message in context: http://forum.world.st/Are-Critical-Tools-Becoming-Too-Brittle-tp4867206.html >> Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com. >> > > -- www.tudorgirba.com www.feenk.com "From an abstract enough point of view, any two things are similar."

Dec. 16, 2015