- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] AsmJit

by Eliot Miranda

Hi Phillippe, On Wed, Dec 16, 2015 at 2:14 AM, philippe.back(a)highoctane.be < philippe.back(a)gmail.com> wrote: > Couldn't we reuse the AsmJIT DSL and generate some C code with Asm > directives through Slang? > C is too high-level for the intended purposes (e.g. performing FFI calls) and Slang is not at all genial purpose or suitable for being used at run-time. Its tightly coupled to the VMaker. And call that C code through FFI? > Since the main use case for this system is to implement the marshalling code for FFI calls this doesn't work. > Seems cleaner and more debuggable. > And could be moved to x64. > > Phil > > On Dec 16, 2015 6:49 AM, "ClÃ©ment Bera" <bera.clement(a)gmail.com> wrote: > > > > > > > > 2015-12-15 20:22 GMT+01:00 Jan Vrany <jan.vrany(a)fit.cvut.cz>: > >> > >> Hi Esteban, Eliot, Clemont > >> > >> thanks very much for elaborate answer. I'm aware of all what you said > >> (well, most :-), but my actual needs are very low and primitive. > >> > >> All I need now is something that allows me to turn x86_64 assembly > >> into a machine code using a sane Smalltalk API, which I can later copy > >> to VM code space. Nothing more, nothing less :-) I do not fancy > >> spending time writing yet-another-x86-assembler hence my interest in > >> asmjit. > > > > > > As far as I know AsmJIT has stable support for x86 but not x86_64 > >> > >> > >> Thanks! Jan > >> > >> > >> > >> On Tue, 2015-12-15 at 18:34 +0100, ClÃ©ment Bera wrote: > >> > We're getting away from Jan initial question here ... > >> > > >> > I think the point here is that Cog has: > >> > - 2 stable back ends: ARMv5 and x86 > >> > - 2 backends stable in the simulator with production planned for ~ > >> > April: x64 and MIPS > >> > and Tim is willing to do the ARMv8 backend. > >> > > >> > All the 4, and soon 5, cog back ends are maintained: > >> > - x86 and x64 maintained by Eliot > >> > - MIPS maintained by Ryan > >> > - ARMv5 and soon ARMv8 maintained by Tim > >> > > >> > Each backend requires months of work. > >> > > >> > Do you want to maintain 10 backends instead of 5 ? Do want to spend > >> > time to implement 5 backends or 10 ? > >> > > >> > I don't. Only the x86 backend is duplicated right now in AsmJIT. > >> > > >> > So one idea is to reuse Cog's backends from the image instead of > >> > AsmJIT. To do so, we can use encodings available in the sista > >> > extended instruction set to tell Cog what machine code to generate. A > >> > project named uFFI aims, among other things, at doing this, by > >> > providing a unified interface between the Cog backends and AsmJIT. > >> > > >> > > >> > 2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: > >> > > Hi Jan, > >> > > > >> > > > On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany(a)fit.cvut.cz> > >> > > wrote: > >> > > > > >> > > > Hi guys, > >> > > > > >> > > > two queations: > >> > > > > >> > > > (i) Is AsmJit going to be developed any more or it's abandoned > >> > > > as well as native boost? > >> > > > >> > > AsmJIT is effectively being abandoned but NativeBoost is not. > >> > > > >> > > The key limitation of AsmJIT is that it was not designed to be > >> > > cross platform; it is effectively an x86 assembler. As such it's > >> > > use gets in the way of ARM and x86_64 (I am currently getting the C > >> > > version of 64-bit Cog Spur working on x86_64, given that it is > >> > > working in the simulator). > >> > > > >> > > Another limitation is that it doesn't play that well with the VM's > >> > > JIT. Igor and I never managed to work on integrating it better. > >> > > The VM's job is managing code and Igor's approach was to hack; > >> > > eliminating execution protection in the entire heap, instead of > >> > > extending the support that either the Alien plugin's callback > >> > > support or the JIT's executable method zone provides. Making the > >> > > entire heap executable is /not/ a sensible approach. > >> > > > >> > > But there is a better way! A key component of the Sista adaptive > >> > > optimizer/speculative inlined that ClÃ©ment is currently stabilizing > >> > > (!!) is a set of bytecodes that encode unsafe operations like > >> > > at:put: without bounds, type or store checks. For example, the > >> > > normal at:put: is about a hundred instructions, checks for > >> > > smallinteger indices, differentiates between byte, 32-but long and > >> > > pointer objects and does a store check. But one of the Sista codes > >> > > for at:put: generates about two instructions, one to adjust the > >> > > index, the other to do the store. Distaste job is to analyze code > >> > > and inline methods using these unsafe bytecodes where they are > >> > > proven to be safe, hence increasing performance. > >> > > > >> > > Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, > >> > > being executed by the VM, can work on an interpreter VM or be > >> > > converted to machine code by the JIT. > >> > > > >> > > So our plan is to extend these bytecodes with ones that support > >> > > marshaling arguments for NativeBoost calls. Ronie Salgado has > >> > > already extended his lowcode scheme to define these instructions > >> > > and sometime soon (hopefully 2016) we shall rewrite NativeBoost to > >> > > target these bytecodes. > >> > > > >> > > HTH > >> > > > >> > > > (ii)Where can I find latest AsmJit? I'm properly confused: > >> > > > > >> > > > * Is is the one in latest Pharo 4.0 (5.0) image? > >> > > > * Is it the one here: http://smalltalkhub.com/#!/~Pharo/AsmJi > >> > > t ? > >> > > > (the one in the image seem to be based on completely > >> > > disjunct > >> > > > set of .mcz than those in the repo above). > >> > > > > >> > > > Best, Jan > >> > > > >> > > _,,,^..^,,,_ (phone) > >> > > > >> > > > -- _,,,^..^,,,_ best, Eliot

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by Robert Withers

Seriously, I think of the I3 work from VW, so CORBA interface schemas are dynamically generated. The same could be done with AVRO schemas, then you could easily read and analyze BigData data, which is pretty happening in the market and still just getting started. Fuel would be a key part of that answer, I can't wait to do more with Fuel. regards, robert On 12/16/2015 10:06 AM, Robert Withers wrote: > I don't know; I'd be surprised. > > There are 2 parts: encoding and schema. Fuel would be great for it. > It is binary. A Squeak/Pharo port would follow the usual get it to > work, work right, work fast! :) > > It's perfect for BigData as it is the recommended Hadoop storage > format. hint...hint.. > > cheers, > robert > > On 12/16/2015 09:57 AM, H. Hirzel wrote: >> Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/ >> >> On 12/16/15, Robert Withers <robert.w.withers(a)gmail.com> wrote: >>> Please consider Avro. >>> >>> robert >>> >>> On 12/16/2015 08:58 AM, H. Hirzel wrote: >>>> If you want to move data to Java then you probably go for JSON or a >>>> particuar XML format. >>>> >>>> On 12/16/15, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: >>>>> No, it is a Smalltalk format. >>>>> >>>>> On 12/16/15, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: >>>>>> 2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: >>>>>> >>>>>>> It probably should be noted here as well that >>>>>>> >>>>>>> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ >>>>>>> >>>>>> Just to know does java implementation exists? >>>>>> >>> -- >>> . .. .. ^,^ best, robert >>> >>> > -- . .. .. ^,^ best, robert

Dec. 16, 2015

MacroRecorder updates (+ another demo)

by Gustavo Santos

Hello guys, thanks for the support so far. I've been working on your suggestions. Some of them concerned that the example in the first demo video was too "magic". :-P So I made another video showing how the developer can manually configure the tool. Take a look: https://youtu.be/3xc3j3PjIYQ ( first demo here: https://youtu.be/gW3l4O-WEGs ) Also, I kindly remind you that I'm interested in usage data: - if you systematically modified code in the past, show me. - if you created a custom transformation with the tool, show me; or - if you had problems using the tool, show me too! :-) Feel free to provide any feedback. Cheers, -- Gustavo Santos http://gustavojss.github.io/

Dec. 16, 2015

Re: [Pharo-dev] Error when using Git

by Alexandre Bergel

Ok, thanks Alexandre > On Dec 16, 2015, at 4:00 PM, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > Le 16/12/2015 15:40, Alexandre Bergel a Ã©crit : >> Hi! >> >> I get the following error: >> >> Any idea whatâs wrong? > > Yes. Don't know exactly but this is a git error. > > [1] Apparently it's a bug in the recent versions of git, and this happens to Mac OS X users of git :( > > Thierry > > [1] http://stackoverflow.com/questions/27330446/getting-a-fatal-error-in-git-fo… > >> Cheers, >> Alexandre >> >> â >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >> Alexandre Bergel http://www.bergel.eu >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >> >> >> > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by Robert Withers

I don't know; I'd be surprised. There are 2 parts: encoding and schema. Fuel would be great for it. It is binary. A Squeak/Pharo port would follow the usual get it to work, work right, work fast! :) It's perfect for BigData as it is the recommended Hadoop storage format. hint...hint.. cheers, robert On 12/16/2015 09:57 AM, H. Hirzel wrote: > Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/ > > On 12/16/15, Robert Withers <robert.w.withers(a)gmail.com> wrote: >> Please consider Avro. >> >> robert >> >> On 12/16/2015 08:58 AM, H. Hirzel wrote: >>> If you want to move data to Java then you probably go for JSON or a >>> particuar XML format. >>> >>> On 12/16/15, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: >>>> No, it is a Smalltalk format. >>>> >>>> On 12/16/15, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: >>>>> 2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: >>>>> >>>>>> It probably should be noted here as well that >>>>>> >>>>>> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ >>>>>> >>>>> Just to know does java implementation exists? >>>>> >> -- >> . .. .. ^,^ best, robert >> >> -- . .. .. ^,^ best, robert

Dec. 16, 2015

Re: [Pharo-dev] Error when using Git

by Thierry Goubier

Le 16/12/2015 15:40, Alexandre Bergel a Ã©crit : > Hi! > > I get the following error: > > Any idea whatâs wrong? Yes. Don't know exactly but this is a git error. [1] Apparently it's a bug in the recent versions of git, and this happens to Mac OS X users of git :( Thierry [1] http://stackoverflow.com/questions/27330446/getting-a-fatal-error-in-git-fo… > Cheers, > Alexandre > > â > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > >

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by H. Hirzel

Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/ On 12/16/15, Robert Withers <robert.w.withers(a)gmail.com> wrote: > Please consider Avro. > > robert > > On 12/16/2015 08:58 AM, H. Hirzel wrote: >> If you want to move data to Java then you probably go for JSON or a >> particuar XML format. >> >> On 12/16/15, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: >>> No, it is a Smalltalk format. >>> >>> On 12/16/15, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: >>>> 2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: >>>> >>>>> It probably should be noted here as well that >>>>> >>>>> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ >>>>> >>>> Just to know does java implementation exists? >>>> > > -- > . .. .. ^,^ best, robert > >

Dec. 16, 2015

Re: [Pharo-dev] Fuel materialization on spur

by Robert Withers

Please consider Avro. robert On 12/16/2015 08:58 AM, H. Hirzel wrote: > If you want to move data to Java then you probably go for JSON or a > particuar XML format. > > On 12/16/15, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: >> No, it is a Smalltalk format. >> >> On 12/16/15, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: >>> 2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel(a)gmail.com>: >>> >>>> It probably should be noted here as well that >>>> >>>> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ >>>> >>> Just to know does java implementation exists? >>> -- . .. .. ^,^ best, robert

Dec. 16, 2015

Re: [Pharo-dev] [Vm-dev] Re: [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support))

by Eliot Miranda

Hi Hannes, > On Dec 16, 2015, at 2:22 AM, H. Hirzel <hannes.hirzel(a)gmail.com> wrote: > > >> On 12/16/15, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >> Hi Todd, >> >>> On Tue, Dec 15, 2015 at 3:46 PM, Todd Blanchard <tblanchard(a)mac.com> wrote: >>> >>> Hi Eliot, >>> >>> On Dec 15, 2015, at 13:46, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >>> >>> Just so you know, I will dig my heels in as deeply as I am able to >>> prevent >>> the use of C++ libraries in the VM. It destroys the simulator, which is >>> the most important thing we have for VM development productivity. As far >>> as I'm concerned any use of external libraries to implement core >>> functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) >>> are >>> built upon. >>> >>> >>> OK, I defer to you because you certainly know more about the VM internals >>> and what does and doesn't work well than anyone else. >>> >>> So I guess I would like to know your recommendation for 1) how best to >>> store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we >>> get to worry about endian). >> >> Raw Unicode, either as 8-bit, 16-bit or 32-bit. When creating a String it >> should start as an 8-bit-per-Unicode-character string. Attempts to store >> Character values that won't fit cause the String to become a String whose >> element size is large enough to accommodate the character. > > This is the case: see tests here http://wiki.squeak.org/squeak/6316 > > In Spur, >> become: is cheap so this growth pays only for the reallocation and copying >> of the at a, not for an expensive heap scan necessary to do the become:. > > Could you elaborate on this please? Well, there are two presentations online, one at ESUG 2014, and one at Splash 2015, a paper at ISMM 2015, and a blog post if you want a full account, but... Spur supports lazy become via forwarders. When an object is becommed, it is turned into a forwarder to the object it becomes, and when a pair of objects are becommed each is copied and the two originals turned into forwarders to the opposite copy. Immediately after the forwarding the receiver in each stack frame in the stack zone is scanned to follow forwarders so that no read barrier is needed when accessing inst vars. Forwarders have a unique class index so any message send to a forwarder will fail. The check for sends to forwarders needs to be done only just before a full lookup. A forwarder looks different to other objects (they have a unique format field in their object header) so any primitive that encounters a forwarder in its operands will fail (since primitives validate their arguments). So in primitive failure the VM checks for forwarders amongst the operands and if any are found, they are followed and the primitive is retried. There are read barriers in some operations, hence I call the scheme a partial read barrier. The main cost is the stack zone scan which must be performed for pointer objects (since only these can have inst vars). This takes very little time on current machinery. Becoming bit objects like strings is faster because no scan is needed. >>> Bearing in mind that both representations are variable length and so >>> while >>> accessing the n'th byte/word is O(1), accessing the n'th character is >>> necessarily O(n) unless you know you have no surrogates in your string. >> >> Right, so UTF-8 and UTF-16 are not convenient representations and to be >> provided only for interchange. > > +1 > > Squeak/Pharo uses 8bit/32bit (UTF-32) internally and UTF-8 externally. > There are converters for UTF-8 and UTF-16. > >>> Also...since NSString has been mentioned...it is worth noting that >>> NSString is built atop CFString (source code here: >>> https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c) which >>> does a fair job of optimizing memory by using bytes where it can and >>> shorts >>> where it cannot. It is also worth noting that characterAt: actually does >>> the wrong thing, since it assumes characters are no bigger than FFFF >>> rather >>> than 10FFFF. >> >> Yes, and Squeak (and AFAIA, Pharo) has been doing this for ages. If one >> has become: it is very easy to manage. Now with Spur not only do we have >> become:, we have a fairly fast become:. >> >> Does this make sense? >> >> >>> Also...I'll just toss in this very nice article on unicode and how >>> NSString deals with it. >>> https://www.objc.io/issues/9-strings/unicode/ >>> >>> -Todd Blanchard >> >> _,,,^..^,,,_ >> best, Eliot

Dec. 16, 2015

Error when using Git

by Alexandre Bergel

Hi! I get the following error: Any idea whatâs wrong? Cheers, Alexandre â _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 16, 2015