- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] AsmJit

by Eliot Miranda

On Tue, Dec 15, 2015 at 1:55 PM, Jan Vrany <jan.vrany(a)fit.cvut.cz> wrote: > Hi Eliot, > > > What's the use case? What do you mean "copy to VM code space"? I > > want to know because I want to give you a viable direction, but I > > need to know more about your intended use first. > > I really appreciate it, but to make it clear: I'm not going > to use Cog/Spur VMs for these experiments. > > By "copy to VM code space" I mean installing a code into > the area in which VM keeps machine code. I think you call > this "code zone" in Cog. This effectively allows me to > generate a code for a method from the "image" itself. > OK, then you'll want to use the simulator to develop and test a suitable primitive. Right now that means using Squeak 5/Spur or Squeak 4.x/Cog to run the VMMaker image. See http://www.squeakvm.org/svn/squeak/branches/Cog/image and http://www.mirandabanda.org/cogblog/build-image for information on how best to build a VMMaker.oscog image. You'll need to understand a /lot/ of details about how the JIT embeds metadata in the VM. Personally I think you'll get further by providing an interface to the Cogit's assembler API so that, for example, the image-level code doesn't have to mimic all the abort/prolog/frame build code in a method and/or the metadata, but instead says to the Cogit :create me a method with standard prolog, and standard metadata for the following sequence of instructions". And do you think you'll want to embed blocks in your generated code? Also (I can see you're being cagey about the experiments, but that's your choice) I think you'll find life much easier if you eschew generating machine code directly and instead extend the inline primitive byte code used to encode Sista's unsafe operations, and there-by generate the machine code you want using "normal" byte codes. This would be great for us, because you'd be extending the VM with instructions we can all use, and great for you, because you're going to have to do far less work in getting your experiments to work, and your experiment will automatically be cross-platform. > > Jan > > > > > > > > > > Thanks! Jan > > > > > > > > > > > > On Tue, 2015-12-15 at 18:34 +0100, ClÃ©ment Bera wrote: > > > > We're getting away from Jan initial question here ... > > > > > > > > I think the point here is that Cog has: > > > > - 2 stable back ends: ARMv5 and x86 > > > > - 2 backends stable in the simulator with production planned for > > > ~ > > > > April: x64 and MIPS > > > > and Tim is willing to do the ARMv8 backend. > > > > > > > > All the 4, and soon 5, cog back ends are maintained: > > > > - x86 and x64 maintained by Eliot > > > > - MIPS maintained by Ryan > > > > - ARMv5 and soon ARMv8 maintained by Tim > > > > > > > > Each backend requires months of work. > > > > > > > > Do you want to maintain 10 backends instead of 5 ? Do want to > > > spend > > > > time to implement 5 backends or 10 ? > > > > > > > > I don't. Only the x86 backend is duplicated right now in AsmJIT. > > > > > > > > So one idea is to reuse Cog's backends from the image instead of > > > > AsmJIT. To do so, we can use encodings available in the sista > > > > extended instruction set to tell Cog what machine code to > > > generate. A > > > > project named uFFI aims, among other things, at doing this, by > > > > providing a unified interface between the Cog backends and > > > AsmJIT. > > > > > > > > > > > > 2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com > > > >: > > > > > Hi Jan, > > > > > > > > > > > On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany(a)fit.cvut.cz > > > > > > > > > wrote: > > > > > > > > > > > > Hi guys, > > > > > > > > > > > > two queations: > > > > > > > > > > > > (i) Is AsmJit going to be developed any more or it's > > > abandoned > > > > > > as well as native boost? > > > > > > > > > > AsmJIT is effectively being abandoned but NativeBoost is not. > > > > > > > > > > The key limitation of AsmJIT is that it was not designed to be > > > > > cross platform; it is effectively an x86 assembler. As such > > > it's > > > > > use gets in the way of ARM and x86_64 (I am currently getting > > > the C > > > > > version of 64-bit Cog Spur working on x86_64, given that it is > > > > > working in the simulator). > > > > > > > > > > Another limitation is that it doesn't play that well with the > > > VM's > > > > > JIT. Igor and I never managed to work on integrating it > > > better. > > > > > The VM's job is managing code and Igor's approach was to hack; > > > > > eliminating execution protection in the entire heap, instead of > > > > > extending the support that either the Alien plugin's callback > > > > > support or the JIT's executable method zone provides. Making > > > the > > > > > entire heap executable is /not/ a sensible approach. > > > > > > > > > > But there is a better way! A key component of the Sista > > > adaptive > > > > > optimizer/speculative inlined that ClÃ©ment is currently > > > stabilizing > > > > > (!!) is a set of bytecodes that encode unsafe operations like > > > > > at:put: without bounds, type or store checks. For example, the > > > > > normal at:put: is about a hundred instructions, checks for > > > > > smallinteger indices, differentiates between byte, 32-but long > > > and > > > > > pointer objects and does a store check. But one of the Sista > > > codes > > > > > for at:put: generates about two instructions, one to adjust the > > > > > index, the other to do the store. Distaste job is to analyze > > > code > > > > > and inline methods using these unsafe bytecodes where they are > > > > > proven to be safe, hence increasing performance. > > > > > > > > > > Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, > > > and, > > > > > being executed by the VM, can work on an interpreter VM or be > > > > > converted to machine code by the JIT. > > > > > > > > > > So our plan is to extend these bytecodes with ones that support > > > > > marshaling arguments for NativeBoost calls. Ronie Salgado has > > > > > already extended his lowcode scheme to define these > > > instructions > > > > > and sometime soon (hopefully 2016) we shall rewrite NativeBoost > > > to > > > > > target these bytecodes. > > > > > > > > > > HTH > > > > > > > > > > > (ii)Where can I find latest AsmJit? I'm properly confused: > > > > > > > > > > > > * Is is the one in latest Pharo 4.0 (5.0) image? > > > > > > * Is it the one here: http://smalltalkhub.com/#!/~Pharo/A > > > smJi > > > > > t ? > > > > > > (the one in the image seem to be based on completely > > > > > disjunct > > > > > > set of .mcz than those in the repo above). > > > > > > > > > > > > Best, Jan > > > > > > > > > > _,,,^..^,,,_ (phone) > > > > > > > > > > > > > > > > > -- > > _,,,^..^,,,_ > > best, Eliot > > -- _,,,^..^,,,_ best, Eliot

Dec. 16, 2015

Re: [Pharo-dev] potential to include cryptography in base image

by Ben Coman

On Wed, Dec 16, 2015 at 3:59 AM, Robert Withers <robert.w.withers(a)gmail.com> wrote: > Thanks for the links. I am ready to copy my ConfigurationOfCryptography to > the MetaRepoForPharo50, though I did my work in Pharo 4, if that is ok. That would be MetaRepoForPharo40 :) > Could you add me so I can publish there, please? > > robert > > > On 12/15/2015 12:55 PM, Peter UhnÃ¡k wrote: >>> >>> Alright, please let me know. If someone shows me how to get this into >>> configurations and the Catalog, we'll get it done. >> >> You just need to commit your configuration to MetaRepoForPharo50 (this >> should be public repo; if you can't commit ask on the mailing list and >> someone will add you). >> >> But before you do that, open Versionner (World > Tools > Versionner), >> right click your configuration and select "add catalog methods". >> This will add class-side methods to the Configuration class which you >> can modify to write things like description, website, tags, etc. I hadn't noticed this feature before - maybe its been added since I last used Versioneer, so I'm asking my question a bit blind (from my day job)... Is it really *obvious* where these methods are created? Perhaps it would be good to have text fields or a code pane tab for each, to really encourage people to fill these in. cheers -ben >> >> You can also use the Veresionner to create your ConfigurationOf if you >> don't have one already, more info on Configurations can be found in >> DeepIntoPharo (Metacello chapter) >> >> https://ci.inria.fr/pharo-contribution/view/Books/job/DeepIntoPharo/lastSuc… >> or look through other configurations in the image or search through >> mailing list archives, there should be plenty of information. >> >> Peter >> > > -- > . .. .. ^,^ best, robert >

Dec. 16, 2015

Re: [Pharo-dev] [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support))

by Eliot Miranda

Hi Todd, On Tue, Dec 15, 2015 at 3:46 PM, Todd Blanchard <tblanchard(a)mac.com> wrote: > Hi Eliot, > > On Dec 15, 2015, at 13:46, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Just so you know, I will dig my heels in as deeply as I am able to prevent > the use of C++ libraries in the VM. It destroys the simulator, which is > the most important thing we have for VM development productivity. As far > as I'm concerned any use of external libraries to implement core > functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) are > built upon. > > > OK, I defer to you because you certainly know more about the VM internals > and what does and doesn't work well than anyone else. > > So I guess I would like to know your recommendation for 1) how best to > store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we > get to worry about endian). > Raw Unicode, either as 8-bit, 16-bit or 32-bit. When creating a String it should start as an 8-bit-per-Unicode-character string. Attempts to store Character values that won't fit cause the String to become a String whose element size is large enough to accommodate the character. In Spur, become: is cheap so this growth pays only for the reallocation and copying of the at a, not for an expensive heap scan necessary to do the become:. > Bearing in mind that both representations are variable length and so while > accessing the n'th byte/word is O(1), accessing the n'th character is > necessarily O(n) unless you know you have no surrogates in your string. > Right, so UTF-8 and UTF-16 are not convenient representations and to be provided only for interchange. > > Also...since NSString has been mentioned...it is worth noting that > NSString is built atop CFString (source code here: > https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c) which > does a fair job of optimizing memory by using bytes where it can and shorts > where it cannot. It is also worth noting that characterAt: actually does > the wrong thing, since it assumes characters are no bigger than FFFF rather > than 10FFFF. > Yes, and Squeak (and AFAIA, Pharo) has been doing this for ages. If one has become: it is very easy to manage. Now with Spur not only do we have become:, we have a fairly fast become:. Does this make sense? > Also...I'll just toss in this very nice article on unicode and how > NSString deals with it. > https://www.objc.io/issues/9-strings/unicode/ > > -Todd Blanchard > _,,,^..^,,,_ best, Eliot

Dec. 16, 2015

Re: [Pharo-dev] potential to include cryptography in base image

by Robert Withers

Do I need to do something with the catalog? It wasnât clear to me how to proceed on that so Iâll be cautious and work on RS-FEC. thanks, robert > On Dec 15, 2015, at 4:57 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Before you copy it to the 50 repo, you should test it on 50 ! > >> On 15 Dec 2015, at 20:59, Robert Withers <robert.w.withers(a)gmail.com> wrote: >> >> Thanks for the links. I am ready to copy my ConfigurationOfCryptography to the MetaRepoForPharo50, though I did my work in Pharo 4, if that is ok. Could you add me so I can publish there, please? >> >> robert >> >> On 12/15/2015 12:55 PM, Peter UhnÃ¡k wrote: >>>> Alright, please let me know. If someone shows me how to get this into >>>> configurations and the Catalog, we'll get it done. >>> You just need to commit your configuration to MetaRepoForPharo50 (this >>> should be public repo; if you can't commit ask on the mailing list and >>> someone will add you). >>> >>> But before you do that, open Versionner (World > Tools > Versionner), >>> right click your configuration and select "add catalog methods". >>> This will add class-side methods to the Configuration class which you >>> can modify to write things like description, website, tags, etc. >>> >>> You can also use the Veresionner to create your ConfigurationOf if you >>> don't have one already, more info on Configurations can be found in >>> DeepIntoPharo (Metacello chapter) >>> https://ci.inria.fr/pharo-contribution/view/Books/job/DeepIntoPharo/lastSuc… >>> or look through other configurations in the image or search through >>> mailing list archives, there should be plenty of information. >>> >>> Peter >>> >> >> -- >> . .. .. ^,^ best, robert >> > >

Dec. 16, 2015

Re: [Pharo-dev] [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support))

by Todd Blanchard

Hi Eliot, > On Dec 15, 2015, at 13:46, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Just so you know, I will dig my heels in as deeply as I am able to prevent the use of C++ libraries in the VM. It destroys the simulator, which is the most important thing we have for VM development productivity. As far as I'm concerned any use of external libraries to implement core functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) are built upon. OK, I defer to you because you certainly know more about the VM internals and what does and doesn't work well than anyone else. So I guess I would like to know your recommendation for 1) how best to store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we get to worry about endian). Bearing in mind that both representations are variable length and so while accessing the n'th byte/word is O(1), accessing the n'th character is necessarily O(n) unless you know you have no surrogates in your string. Also...since NSString has been mentioned...it is worth noting that NSString is built atop CFString (source code here: https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c <https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c>) which does a fair job of optimizing memory by using bytes where it can and shorts where it cannot. It is also worth noting that characterAt: actually does the wrong thing, since it assumes characters are no bigger than FFFF rather than 10FFFF. Also...I'll just toss in this very nice article on unicode and how NSString deals with it. https://www.objc.io/issues/9-strings/unicode/ <https://www.objc.io/issues/9-strings/unicode/> -Todd Blanchard

Dec. 15, 2015

Re: [Pharo-dev] [Vm-dev] Re: Question about errno and NB FFI

by Mariano Martinez Peck

On Tue, Dec 15, 2015 at 6:52 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > > On Tue, Dec 15, 2015 at 1:34 PM, Mariano Martinez Peck < > marianopeck(a)gmail.com> wrote: > >> Hi guys, >> >> I am using NB FFI and I am calling a function that could answer a NULL* >> in case of error and that it should have set the number on *errno.* >> >> So now I have some questions: >> >> Say the code is this: >> >> run >> | whatever | >> whatever := self *primitiveWhatever*. >> ^ popenFile ifNil: [ >> self *primitivePerror*. >> >> Imagine that the C lib called from #primitiveWhatever was the one setting >> *errno.* >> My question is... the following #primitivePerror is a DIFFERENT system >> call. Couldn't that happened that another error happened before the first >> system call and time I am executing the second one and so the "last error" >> (errno) is actually the wrong one I am thinking of? >> > > Yes, which is why it's really important that the FFI be able to be told to > answer errno on error. See e.g. the VW FFI where IIRC one can specify that > if a call answers -1, the FFI call can fail with a primitive error code > that includes errno. > > Yes, that would be very nice. At least for -1 and NULL. It seems Python FFI does this as well. > >> Even worst, I may think that the scope of the "global variable errno" >> may existed at the scope of the invoked lib only? In this case I don't have >> above problem, but then....how can I get such number once the NB primitive >> finished (so that I can call, for example, strerror() ? >> > > Well, there are two issues here. One is that one should use waterer the > underlying platform defines for getting "errno", for example some platforms > will say something like #define errno get_errno(), to get the errno > specific to the current thread. > It's funny there is no standard way (POSIX function?) for it. And there is for example for perror() which basically prints errno in stderr, but none standard function to get last errno of current thread? > Second, all one's libraries need to use the same approach so that errno is > shared between the VM and the libraries, otherwise your library may not set > the errno seen by the VM and the FFI. > > I didn't understand this last sentence. > > >> Thanks in advance, >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> > > > > -- > _,,,^..^,,,_ > best, Eliot > > -- Mariano http://marianopeck.wordpress.com

Dec. 15, 2015

Re: [Pharo-dev] Are Critical Tools Becoming Too Brittle?

by Sven Van Caekenberghe

I assume you are using the GT inspector. Do you have custom gt inspector presentations ? Are you sure your #printString is OK ? There is also a 'Basic Inspect It' menu item ... Can you isolate it so that we can see it, try it ? > On 15 Dec 2015, at 23:21, Sean P. DeNigris <sean(a)clipperadams.com> wrote: > > I just tried to inspect a domain object and I froze my image and had to force > quit. How do I debug this? And, more importantly, when an error is > encountered in a critical, basic tool like this, should there be some > reasonable bare-bones alternative, like the emergency debugger? Not being > able to inspect an object without crashing seems like a pretty crippling > situation... > > > > ----- > Cheers, > Sean > -- > View this message in context: http://forum.world.st/Are-Critical-Tools-Becoming-Too-Brittle-tp4867206.html > Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com. >

Dec. 15, 2015

Re: [Pharo-dev] potential to include cryptography in base image

by Robert Withers

Alright tested, issue found and fixed. I'm not sure how to proceed. On 12/15/2015 04:57 PM, Sven Van Caekenberghe wrote: > Before you copy it to the 50 repo, you should test it on 50 ! > >> On 15 Dec 2015, at 20:59, Robert Withers <robert.w.withers(a)gmail.com> wrote: >> >> Thanks for the links. I am ready to copy my ConfigurationOfCryptography to the MetaRepoForPharo50, though I did my work in Pharo 4, if that is ok. Could you add me so I can publish there, please? >> >> robert >> >> On 12/15/2015 12:55 PM, Peter UhnÃ¡k wrote: >>>> Alright, please let me know. If someone shows me how to get this into >>>> configurations and the Catalog, we'll get it done. >>> You just need to commit your configuration to MetaRepoForPharo50 (this >>> should be public repo; if you can't commit ask on the mailing list and >>> someone will add you). >>> >>> But before you do that, open Versionner (World > Tools > Versionner), >>> right click your configuration and select "add catalog methods". >>> This will add class-side methods to the Configuration class which you >>> can modify to write things like description, website, tags, etc. >>> >>> You can also use the Veresionner to create your ConfigurationOf if you >>> don't have one already, more info on Configurations can be found in >>> DeepIntoPharo (Metacello chapter) >>> https://ci.inria.fr/pharo-contribution/view/Books/job/DeepIntoPharo/lastSuc… >>> or look through other configurations in the image or search through >>> mailing list archives, there should be plenty of information. >>> >>> Peter >>> >> -- >> . .. .. ^,^ best, robert >> > -- . .. .. ^,^ best, robert

Dec. 15, 2015

Are Critical Tools Becoming Too Brittle?

by Sean P. DeNigris

I just tried to inspect a domain object and I froze my image and had to force quit. How do I debug this? And, more importantly, when an error is encountered in a critical, basic tool like this, should there be some reasonable bare-bones alternative, like the emergency debugger? Not being able to inspect an object without crashing seems like a pretty crippling situation... ----- Cheers, Sean -- View this message in context: http://forum.world.st/Are-Critical-Tools-Becoming-Too-Brittle-tp4867206.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Dec. 15, 2015

Re: [Pharo-dev] potential to include cryptography in base image

by Sven Van Caekenberghe

Before you copy it to the 50 repo, you should test it on 50 ! > On 15 Dec 2015, at 20:59, Robert Withers <robert.w.withers(a)gmail.com> wrote: > > Thanks for the links. I am ready to copy my ConfigurationOfCryptography to the MetaRepoForPharo50, though I did my work in Pharo 4, if that is ok. Could you add me so I can publish there, please? > > robert > > On 12/15/2015 12:55 PM, Peter UhnÃ¡k wrote: >>> Alright, please let me know. If someone shows me how to get this into >>> configurations and the Catalog, we'll get it done. >> You just need to commit your configuration to MetaRepoForPharo50 (this >> should be public repo; if you can't commit ask on the mailing list and >> someone will add you). >> >> But before you do that, open Versionner (World > Tools > Versionner), >> right click your configuration and select "add catalog methods". >> This will add class-side methods to the Configuration class which you >> can modify to write things like description, website, tags, etc. >> >> You can also use the Veresionner to create your ConfigurationOf if you >> don't have one already, more info on Configurations can be found in >> DeepIntoPharo (Metacello chapter) >> https://ci.inria.fr/pharo-contribution/view/Books/job/DeepIntoPharo/lastSuc… >> or look through other configurations in the image or search through >> mailing list archives, there should be plenty of information. >> >> Peter >> > > -- > . .. .. ^,^ best, robert >

Dec. 15, 2015