We're getting away from Jan initial question here ... I think the point here is that Cog has: - 2 stable back ends: ARMv5 and x86 - 2 backends stable in the simulator with production planned for ~ April: x64 and MIPS and Tim is willing to do the ARMv8 backend. All the 4, and soon 5, cog back ends are maintained: - x86 and x64 maintained by Eliot - MIPS maintained by Ryan - ARMv5 and soon ARMv8 maintained by Tim Each backend requires months of work. Do you want to maintain 10 backends instead of 5 ? Do want to spend time to implement 5 backends or 10 ? I don't. Only the x86 backend is duplicated right now in AsmJIT. So one idea is to reuse Cog's backends from the image instead of AsmJIT. To do so, we can use encodings available in the sista extended instruction set to tell Cog what machine code to generate. A project named uFFI aims, among other things, at doing this, by providing a unified interface between the Cog backends and AsmJIT. 2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

Hi Jan,

...

On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:

Hi guys,

two queations:

(i) Is AsmJit going to be developed any more or it's abandoned as well as native boost?

AsmJIT is effectively being abandoned but NativeBoost is not.

The key limitation of AsmJIT is that it was not designed to be cross platform; it is effectively an x86 assembler. As such it's use gets in the way of ARM and x86_64 (I am currently getting the C version of 64-bit Cog Spur working on x86_64, given that it is working in the simulator).

Another limitation is that it doesn't play that well with the VM's JIT. Igor and I never managed to work on integrating it better. The VM's job is managing code and Igor's approach was to hack; eliminating execution protection in the entire heap, instead of extending the support that either the Alien plugin's callback support or the JIT's executable method zone provides. Making the entire heap executable is /not/ a sensible approach.

But there is a better way! A key component of the Sista adaptive optimizer/speculative inlined that Clément is currently stabilizing (!!) is a set of bytecodes that encode unsafe operations like at:put: without bounds, type or store checks. For example, the normal at:put: is about a hundred instructions, checks for smallinteger indices, differentiates between byte, 32-but long and pointer objects and does a store check. But one of the Sista codes for at:put: generates about two instructions, one to adjust the index, the other to do the store. Distaste job is to analyze code and inline methods using these unsafe bytecodes where they are proven to be safe, hence increasing performance.

Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, being executed by the VM, can work on an interpreter VM or be converted to machine code by the JIT.

So our plan is to extend these bytecodes with ones that support marshaling arguments for NativeBoost calls. Ronie Salgado has already extended his lowcode scheme to define these instructions and sometime soon (hopefully 2016) we shall rewrite NativeBoost to target these bytecodes.

HTH

...

(ii)Where can I find latest AsmJit? I'm properly confused:

* Is is the one in latest Pharo 4.0 (5.0) image? * Is it the one here: http://smalltalkhub.com/#!/~Pharo/AsmJit ? (the one in the image seem to be based on completely disjunct set of .mcz than those in the repo above).

Best, Jan

_,,,^..^,,,_ (phone)

Hi Jan, On Tue, Dec 15, 2015 at 11:22 AM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:

Hi Esteban, Eliot, Clemont

thanks very much for elaborate answer. I'm aware of all what you said (well, most :-), but my actual needs are very low and primitive.

All I need now is something that allows me to turn x86_64 assembly into a machine code using a sane Smalltalk API, which I can later copy to VM code space. Nothing more, nothing less :-) I do not fancy spending time writing yet-another-x86-assembler hence my interest in asmjit.

What's the use case? What do you mean "copy to VM code space"? I want to know because I want to give you a viable direction, but I need to know more about your intended use first.

Thanks! Jan

On Tue, 2015-12-15 at 18:34 +0100, Clément Bera wrote:

...

We're getting away from Jan initial question here ...

I think the point here is that Cog has: - 2 stable back ends: ARMv5 and x86 - 2 backends stable in the simulator with production planned for ~ April: x64 and MIPS and Tim is willing to do the ARMv8 backend.

All the 4, and soon 5, cog back ends are maintained: - x86 and x64 maintained by Eliot - MIPS maintained by Ryan - ARMv5 and soon ARMv8 maintained by Tim

Each backend requires months of work.

Do you want to maintain 10 backends instead of 5 ? Do want to spend time to implement 5 backends or 10 ?

I don't. Only the x86 backend is duplicated right now in AsmJIT.

So one idea is to reuse Cog's backends from the image instead of AsmJIT. To do so, we can use encodings available in the sista extended instruction set to tell Cog what machine code to generate. A project named uFFI aims, among other things, at doing this, by providing a unified interface between the Cog backends and AsmJIT.

2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Jan,

...

On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:

Hi guys,

two queations:

(i) Is AsmJit going to be developed any more or it's abandoned as well as native boost?

AsmJIT is effectively being abandoned but NativeBoost is not.

The key limitation of AsmJIT is that it was not designed to be cross platform; it is effectively an x86 assembler. As such it's use gets in the way of ARM and x86_64 (I am currently getting the C version of 64-bit Cog Spur working on x86_64, given that it is working in the simulator).

Another limitation is that it doesn't play that well with the VM's JIT. Igor and I never managed to work on integrating it better. The VM's job is managing code and Igor's approach was to hack; eliminating execution protection in the entire heap, instead of extending the support that either the Alien plugin's callback support or the JIT's executable method zone provides. Making the entire heap executable is /not/ a sensible approach.

But there is a better way! A key component of the Sista adaptive optimizer/speculative inlined that Clément is currently stabilizing (!!) is a set of bytecodes that encode unsafe operations like at:put: without bounds, type or store checks. For example, the normal at:put: is about a hundred instructions, checks for smallinteger indices, differentiates between byte, 32-but long and pointer objects and does a store check. But one of the Sista codes for at:put: generates about two instructions, one to adjust the index, the other to do the store. Distaste job is to analyze code and inline methods using these unsafe bytecodes where they are proven to be safe, hence increasing performance.

Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, being executed by the VM, can work on an interpreter VM or be converted to machine code by the JIT.

So our plan is to extend these bytecodes with ones that support marshaling arguments for NativeBoost calls. Ronie Salgado has already extended his lowcode scheme to define these instructions and sometime soon (hopefully 2016) we shall rewrite NativeBoost to target these bytecodes.

HTH

...

(ii)Where can I find latest AsmJit? I'm properly confused:

* Is is the one in latest Pharo 4.0 (5.0) image? * Is it the one here: http://smalltalkhub.com/#!/~Pharo/AsmJi t ? (the one in the image seem to be based on completely disjunct set of .mcz than those in the repo above).

Best, Jan

_,,,^..^,,,_ (phone)

-- _,,,^..^,,,_ best, Eliot

On Tue, Dec 15, 2015 at 1:55 PM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:

Hi Eliot,

...

What's the use case? What do you mean "copy to VM code space"? I want to know because I want to give you a viable direction, but I need to know more about your intended use first.

I really appreciate it, but to make it clear: I'm not going to use Cog/Spur VMs for these experiments.

By "copy to VM code space" I mean installing a code into the area in which VM keeps machine code. I think you call this "code zone" in Cog. This effectively allows me to generate a code for a method from the "image" itself.

OK, then you'll want to use the simulator to develop and test a suitable primitive. Right now that means using Squeak 5/Spur or Squeak 4.x/Cog to run the VMMaker image. See http://www.squeakvm.org/svn/squeak/branches/Cog/image and http://www.mirandabanda.org/cogblog/build-image for information on how best to build a VMMaker.oscog image. You'll need to understand a /lot/ of details about how the JIT embeds metadata in the VM. Personally I think you'll get further by providing an interface to the Cogit's assembler API so that, for example, the image-level code doesn't have to mimic all the abort/prolog/frame build code in a method and/or the metadata, but instead says to the Cogit :create me a method with standard prolog, and standard metadata for the following sequence of instructions". And do you think you'll want to embed blocks in your generated code? Also (I can see you're being cagey about the experiments, but that's your choice) I think you'll find life much easier if you eschew generating machine code directly and instead extend the inline primitive byte code used to encode Sista's unsafe operations, and there-by generate the machine code you want using "normal" byte codes. This would be great for us, because you'd be extending the VM with instructions we can all use, and great for you, because you're going to have to do far less work in getting your experiments to work, and your experiment will automatically be cross-platform.

Jan

...

...

Thanks! Jan

On Tue, 2015-12-15 at 18:34 +0100, Clément Bera wrote:

...

We're getting away from Jan initial question here ...

I think the point here is that Cog has: - 2 stable back ends: ARMv5 and x86 - 2 backends stable in the simulator with production planned for ~ April: x64 and MIPS and Tim is willing to do the ARMv8 backend.

All the 4, and soon 5, cog back ends are maintained: - x86 and x64 maintained by Eliot - MIPS maintained by Ryan - ARMv5 and soon ARMv8 maintained by Tim

Each backend requires months of work.

Do you want to maintain 10 backends instead of 5 ? Do want to spend time to implement 5 backends or 10 ?

I don't. Only the x86 backend is duplicated right now in AsmJIT.

So one idea is to reuse Cog's backends from the image instead of AsmJIT. To do so, we can use encodings available in the sista extended instruction set to tell Cog what machine code to generate. A project named uFFI aims, among other things, at doing this, by providing a unified interface between the Cog backends and AsmJIT.

2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com :

...

Hi Jan,

...

On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany@fit.cvut.cz

...

wrote:

...

Hi guys,

two queations:

(i) Is AsmJit going to be developed any more or it's

abandoned

...

as well as native boost?

AsmJIT is effectively being abandoned but NativeBoost is not.

The key limitation of AsmJIT is that it was not designed to be cross platform; it is effectively an x86 assembler. As such it's use gets in the way of ARM and x86_64 (I am currently getting the C version of 64-bit Cog Spur working on x86_64, given that it is working in the simulator).

Another limitation is that it doesn't play that well with the VM's JIT. Igor and I never managed to work on integrating it better. The VM's job is managing code and Igor's approach was to hack; eliminating execution protection in the entire heap, instead of extending the support that either the Alien plugin's callback support or the JIT's executable method zone provides. Making the entire heap executable is /not/ a sensible approach.

But there is a better way! A key component of the Sista adaptive optimizer/speculative inlined that Clément is currently stabilizing (!!) is a set of bytecodes that encode unsafe operations like at:put: without bounds, type or store checks. For example, the normal at:put: is about a hundred instructions, checks for smallinteger indices, differentiates between byte, 32-but long and pointer objects and does a store check. But one of the Sista codes for at:put: generates about two instructions, one to adjust the index, the other to do the store. Distaste job is to analyze code and inline methods using these unsafe bytecodes where they are proven to be safe, hence increasing performance.

Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, being executed by the VM, can work on an interpreter VM or be converted to machine code by the JIT.

So our plan is to extend these bytecodes with ones that support marshaling arguments for NativeBoost calls. Ronie Salgado has already extended his lowcode scheme to define these instructions and sometime soon (hopefully 2016) we shall rewrite NativeBoost to target these bytecodes.

HTH

...

(ii)Where can I find latest AsmJit? I'm properly confused:

* Is is the one in latest Pharo 4.0 (5.0) image? * Is it the one here: http://smalltalkhub.com/#!/~Pharo/A smJi t ? (the one in the image seem to be based on completely disjunct set of .mcz than those in the repo above).

Best, Jan

_,,,^..^,,,_ (phone)

-- _,,,^..^,,,_ best, Eliot

-- _,,,^..^,,,_ best, Eliot

Couldn't we reuse the AsmJIT DSL and generate some C code with Asm directives through Slang? And call that C code through FFI? Seems cleaner and more debuggable. And could be moved to x64. Phil On Dec 16, 2015 6:49 AM, "Clément Bera" <bera.clement@gmail.com> wrote:

2015-12-15 20:22 GMT+01:00 Jan Vrany <jan.vrany@fit.cvut.cz>:

...

Hi Esteban, Eliot, Clemont

thanks very much for elaborate answer. I'm aware of all what you said (well, most :-), but my actual needs are very low and primitive.

All I need now is something that allows me to turn x86_64 assembly into a machine code using a sane Smalltalk API, which I can later copy to VM code space. Nothing more, nothing less :-) I do not fancy spending time writing yet-another-x86-assembler hence my interest in asmjit.

As far as I know AsmJIT has stable support for x86 but not x86_64

...

Thanks! Jan

On Tue, 2015-12-15 at 18:34 +0100, Clément Bera wrote:

...

We're getting away from Jan initial question here ...

I think the point here is that Cog has: - 2 stable back ends: ARMv5 and x86 - 2 backends stable in the simulator with production planned for ~ April: x64 and MIPS and Tim is willing to do the ARMv8 backend.

All the 4, and soon 5, cog back ends are maintained: - x86 and x64 maintained by Eliot - MIPS maintained by Ryan - ARMv5 and soon ARMv8 maintained by Tim

Each backend requires months of work.

Do you want to maintain 10 backends instead of 5 ? Do want to spend time to implement 5 backends or 10 ?

I don't. Only the x86 backend is duplicated right now in AsmJIT.

So one idea is to reuse Cog's backends from the image instead of AsmJIT. To do so, we can use encodings available in the sista extended instruction set to tell Cog what machine code to generate. A project named uFFI aims, among other things, at doing this, by providing a unified interface between the Cog backends and AsmJIT.

2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Jan,

...

On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:

Hi guys,

two queations:

(i) Is AsmJit going to be developed any more or it's abandoned as well as native boost?

AsmJIT is effectively being abandoned but NativeBoost is not.

The key limitation of AsmJIT is that it was not designed to be cross platform; it is effectively an x86 assembler. As such it's use gets in the way of ARM and x86_64 (I am currently getting the C version of 64-bit Cog Spur working on x86_64, given that it is working in the simulator).

Another limitation is that it doesn't play that well with the VM's JIT. Igor and I never managed to work on integrating it better. The VM's job is managing code and Igor's approach was to hack; eliminating execution protection in the entire heap, instead of extending the support that either the Alien plugin's callback support or the JIT's executable method zone provides. Making the entire heap executable is /not/ a sensible approach.

But there is a better way! A key component of the Sista adaptive optimizer/speculative inlined that Clément is currently stabilizing (!!) is a set of bytecodes that encode unsafe operations like at:put: without bounds, type or store checks. For example, the normal at:put: is about a hundred instructions, checks for smallinteger indices, differentiates between byte, 32-but long and pointer objects and does a store check. But one of the Sista codes for at:put: generates about two instructions, one to adjust the index, the other to do the store. Distaste job is to analyze code and inline methods using these unsafe bytecodes where they are proven to be safe, hence increasing performance.

Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, being executed by the VM, can work on an interpreter VM or be converted to machine code by the JIT.

So our plan is to extend these bytecodes with ones that support marshaling arguments for NativeBoost calls. Ronie Salgado has already extended his lowcode scheme to define these instructions and sometime soon (hopefully 2016) we shall rewrite NativeBoost to target these bytecodes.

HTH

...

(ii)Where can I find latest AsmJit? I'm properly confused:

* Is is the one in latest Pharo 4.0 (5.0) image? * Is it the one here: http://smalltalkhub.com/#!/~Pharo/AsmJi t ? (the one in the image seem to be based on completely disjunct set of .mcz than those in the repo above).

Best, Jan

_,,,^..^,,,_ (phone)

Eliot, The C code generator has been externalized, so it is not dependent on VMMaker. http://www.smalltalkhub.com/#!/~PavelKrivanek/CCodeGenerator As far as we can have an ASM DSL that would be working along with that generator, there is no reason why we couldn't have assembly code part emitted by that CCodeGenerator as inlined asm bits ( https://gcc.gnu.org/onlinedocs/gcc/Using-Assembly-Language-with-C.html) e.g. https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html#Extended-Asm So, AsmJit syntax would extend CCodeGenerator and output C and inlined asm that would then be compiled and invoked with FFI ( https://clementbera.wordpress.com/2013/06/19/optimizing-pharo-to-c-speed-wit... ) This may not be the use case for writing FFI but that is a use case for writing and invoking ASMx86 in Smalltalk (or whatever else, based on the AsmJIT syntax for a given architecture + backend for CCodeGenerator). What do you think? Phil On Wed, Dec 16, 2015 at 6:06 PM, Eliot Miranda <eliot.miranda@gmail.com> wrote:

Hi Phillippe,

On Wed, Dec 16, 2015 at 2:14 AM, philippe.back@highoctane.be < philippe.back@gmail.com> wrote:

...

Couldn't we reuse the AsmJIT DSL and generate some C code with Asm directives through Slang?

C is too high-level for the intended purposes (e.g. performing FFI calls) and Slang is not at all genial purpose or suitable for being used at run-time. Its tightly coupled to the VMaker.

And call that C code through FFI?

...

Since the main use case for this system is to implement the marshalling code for FFI calls this doesn't work.

...

Seems cleaner and more debuggable. And could be moved to x64.

Phil

On Dec 16, 2015 6:49 AM, "Clément Bera" <bera.clement@gmail.com> wrote:

...

2015-12-15 20:22 GMT+01:00 Jan Vrany <jan.vrany@fit.cvut.cz>:

...

Hi Esteban, Eliot, Clemont

thanks very much for elaborate answer. I'm aware of all what you said (well, most :-), but my actual needs are very low and primitive.

All I need now is something that allows me to turn x86_64 assembly into a machine code using a sane Smalltalk API, which I can later copy to VM code space. Nothing more, nothing less :-) I do not fancy spending time writing yet-another-x86-assembler hence my interest in asmjit.

As far as I know AsmJIT has stable support for x86 but not x86_64

...

Thanks! Jan

On Tue, 2015-12-15 at 18:34 +0100, Clément Bera wrote:

...

We're getting away from Jan initial question here ...

I think the point here is that Cog has: - 2 stable back ends: ARMv5 and x86 - 2 backends stable in the simulator with production planned for ~ April: x64 and MIPS and Tim is willing to do the ARMv8 backend.

All the 4, and soon 5, cog back ends are maintained: - x86 and x64 maintained by Eliot - MIPS maintained by Ryan - ARMv5 and soon ARMv8 maintained by Tim

Each backend requires months of work.

Do you want to maintain 10 backends instead of 5 ? Do want to spend time to implement 5 backends or 10 ?

I don't. Only the x86 backend is duplicated right now in AsmJIT.

So one idea is to reuse Cog's backends from the image instead of AsmJIT. To do so, we can use encodings available in the sista extended instruction set to tell Cog what machine code to generate. A project named uFFI aims, among other things, at doing this, by providing a unified interface between the Cog backends and AsmJIT.

2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Jan,

...

On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:

Hi guys,

two queations:

(i) Is AsmJit going to be developed any more or it's abandoned as well as native boost?

AsmJIT is effectively being abandoned but NativeBoost is not.

The key limitation of AsmJIT is that it was not designed to be cross platform; it is effectively an x86 assembler. As such it's use gets in the way of ARM and x86_64 (I am currently getting the C version of 64-bit Cog Spur working on x86_64, given that it is working in the simulator).

Another limitation is that it doesn't play that well with the VM's JIT. Igor and I never managed to work on integrating it better. The VM's job is managing code and Igor's approach was to hack; eliminating execution protection in the entire heap, instead of extending the support that either the Alien plugin's callback support or the JIT's executable method zone provides. Making the entire heap executable is /not/ a sensible approach.

But there is a better way! A key component of the Sista adaptive optimizer/speculative inlined that Clément is currently stabilizing (!!) is a set of bytecodes that encode unsafe operations like at:put: without bounds, type or store checks. For example, the normal at:put: is about a hundred instructions, checks for smallinteger indices, differentiates between byte, 32-but long and pointer objects and does a store check. But one of the Sista codes for at:put: generates about two instructions, one to adjust the index, the other to do the store. Distaste job is to analyze code and inline methods using these unsafe bytecodes where they are proven to be safe, hence increasing performance.

Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, being executed by the VM, can work on an interpreter VM or be converted to machine code by the JIT.

So our plan is to extend these bytecodes with ones that support marshaling arguments for NativeBoost calls. Ronie Salgado has already extended his lowcode scheme to define these instructions and sometime soon (hopefully 2016) we shall rewrite NativeBoost to target these bytecodes.

HTH

...

(ii)Where can I find latest AsmJit? I'm properly confused:

* Is is the one in latest Pharo 4.0 (5.0) image? * Is it the one here:

http://smalltalkhub.com/#!/~Pharo/AsmJi

...

...

...

...

t ?

...

(the one in the image seem to be based on completely

disjunct

...

set of .mcz than those in the repo above).

Best, Jan

_,,,^..^,,,_ (phone)

-- _,,,^..^,,,_ best, Eliot

Hi, I get you. What I mean is to use AsmJIT "way of writing assembly" *DSL*, not the actual part generating the machine code. The DSL would translate into inlined asm code in a C file, that in turn is indeed compiled into and object file, linked into a DLL, and the reloaded as a module, and invoked back through FFI. Hope it is clearer. Phil On Wed, Dec 16, 2015 at 9:01 PM, Eliot Miranda <eliot.miranda@gmail.com> wrote:

Hi Phil,

On Wed, Dec 16, 2015 at 10:14 AM, phil@highoctane.be <phil@highoctane.be> wrote:

...

Eliot,

The C code generator has been externalized, so it is not dependent on VMMaker.

http://www.smalltalkhub.com/#!/~PavelKrivanek/CCodeGenerator

As far as we can have an ASM DSL that would be working along with that generator, there is no reason why we couldn't have assembly code part emitted by that CCodeGenerator as inlined asm bits ( https://gcc.gnu.org/onlinedocs/gcc/Using-Assembly-Language-with-C.html)

e.g. https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html#Extended-Asm

Yes, but you have to invoke gcc to translate the C and extended asm into assembler, and an assembler to translate that output into machine code, except that the assembler produces object files, not raw machine code.

...

So, AsmJit syntax would extend CCodeGenerator and output C and inlined asm that would then be compiled and invoked with FFI ( https://clementbera.wordpress.com/2013/06/19/optimizing-pharo-to-c-speed-wit... )

This may not be the use case for writing FFI but that is a use case for writing and invoking ASMx86 in Smalltalk (or whatever else, based on the AsmJIT syntax for a given architecture + backend for CCodeGenerator).

What do you think?

I don't think it solves the problem. AsmJIT is a source to machine-code translator but Slang is a source to source translator. So I don't see how it solves the key problem, which is to generate the machine code for FFI call argument marshalling according to the rules of the underlying platform's ABI. Maybe I'm being dense, but it doesn't seem to me to be relevant.

Right now, the Cog JIT contains an assembler and translates byte coded methods into sequences of assembler instructions and these into machine code embedded in CogMethod objects in the VM. AsmJIT contains an assembler which translates assembler instructions to machine code and embeds those machine code sequences in the trailer of compiled methods and then contains a hack to get the Cog JIT to call that machine code. None of these assembler to machine code steps are provided (at least not conveniently) by your proposed tool chain above. There is obvious duplication in having Cog contain an assembler and AsmJIT contain an assembler. But Cog also contains an escape sequence of byte codes we are using to generate efficient, unsafe code. This can be used to generate the marshalling code we need, and do it in a cross-platform manner, avoiding the duplication inherent in AsmJIT above Cog, and avoiding the hacks necessary to get Cog to be able to use the code embedded in method trailers (such as making the entire heap executable, and hacking Cog to call these escape sequences). I'm making myself parse here articulating what I think is an elegant, general and powerful architecture. But people find it boring. I don't understand. I think I'm going to shut up :-/

Phil

...

On Wed, Dec 16, 2015 at 6:06 PM, Eliot Miranda <eliot.miranda@gmail.com> wrote:

...

Hi Phillippe,

On Wed, Dec 16, 2015 at 2:14 AM, philippe.back@highoctane.be < philippe.back@gmail.com> wrote:

...

Couldn't we reuse the AsmJIT DSL and generate some C code with Asm directives through Slang?

C is too high-level for the intended purposes (e.g. performing FFI calls) and Slang is not at all genial purpose or suitable for being used at run-time. Its tightly coupled to the VMaker.

And call that C code through FFI?

...

Since the main use case for this system is to implement the marshalling code for FFI calls this doesn't work.

...

Seems cleaner and more debuggable. And could be moved to x64.

Phil

On Dec 16, 2015 6:49 AM, "Clément Bera" <bera.clement@gmail.com> wrote:

...

2015-12-15 20:22 GMT+01:00 Jan Vrany <jan.vrany@fit.cvut.cz>:

...

Hi Esteban, Eliot, Clemont

thanks very much for elaborate answer. I'm aware of all what you said (well, most :-), but my actual needs are very low and primitive.

All I need now is something that allows me to turn x86_64 assembly into a machine code using a sane Smalltalk API, which I can later

copy

...

...

to VM code space. Nothing more, nothing less :-) I do not fancy spending time writing yet-another-x86-assembler hence my interest in asmjit.

As far as I know AsmJIT has stable support for x86 but not x86_64

...

Thanks! Jan

On Tue, 2015-12-15 at 18:34 +0100, Clément Bera wrote:

...

We're getting away from Jan initial question here ...

I think the point here is that Cog has: - 2 stable back ends: ARMv5 and x86 - 2 backends stable in the simulator with production planned for ~ April: x64 and MIPS and Tim is willing to do the ARMv8 backend.

All the 4, and soon 5, cog back ends are maintained: - x86 and x64 maintained by Eliot - MIPS maintained by Ryan - ARMv5 and soon ARMv8 maintained by Tim

Each backend requires months of work.

Do you want to maintain 10 backends instead of 5 ? Do want to spend time to implement 5 backends or 10 ?

I don't. Only the x86 backend is duplicated right now in AsmJIT.

So one idea is to reuse Cog's backends from the image instead of AsmJIT. To do so, we can use encodings available in the sista extended instruction set to tell Cog what machine code to

generate. A

...

...

project named uFFI aims, among other things, at doing this, by providing a unified interface between the Cog backends and AsmJIT.

2015-12-15 16:43 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com :

...

Hi Jan,

...

On Dec 15, 2015, at 3:06 AM, Jan Vrany <jan.vrany@fit.cvut.cz> wrote:

Hi guys,

two queations:

(i) Is AsmJit going to be developed any more or it's abandoned as well as native boost?

AsmJIT is effectively being abandoned but NativeBoost is not.

The key limitation of AsmJIT is that it was not designed to be cross platform; it is effectively an x86 assembler. As such it's use gets in the way of ARM and x86_64 (I am currently getting the C version of 64-bit Cog Spur working on x86_64, given that it is working in the simulator).

Another limitation is that it doesn't play that well with the VM's JIT. Igor and I never managed to work on integrating it better. The VM's job is managing code and Igor's approach was to hack; eliminating execution protection in the entire heap, instead of extending the support that either the Alien plugin's callback support or the JIT's executable method zone provides. Making the entire heap executable is /not/ a sensible approach.

But there is a better way! A key component of the Sista adaptive optimizer/speculative inlined that Clément is currently stabilizing (!!) is a set of bytecodes that encode unsafe operations like at:put: without bounds, type or store checks. For example, the normal at:put: is about a hundred instructions, checks for smallinteger indices, differentiates between byte, 32-but long and pointer objects and does a store check. But one of the Sista codes for at:put: generates about two instructions, one to adjust the index, the other to do the store. Distaste job is to analyze code and inline methods using these unsafe bytecodes where they are proven to be safe, hence increasing performance.

Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, being executed by the VM, can work on an interpreter VM or be converted to machine code by the JIT.

So our plan is to extend these bytecodes with ones that support marshaling arguments for NativeBoost calls. Ronie Salgado has already extended his lowcode scheme to define these instructions and sometime soon (hopefully 2016) we shall rewrite NativeBoost to target these bytecodes.

HTH

...

(ii)Where can I find latest AsmJit? I'm properly confused:

* Is is the one in latest Pharo 4.0 (5.0) image? * Is it the one here: http://smalltalkhub.com/#!/~Pharo/AsmJi t ? (the one in the image seem to be based on completely disjunct set of .mcz than those in the repo above).

Best, Jan

_,,,^..^,,,_ (phone)

_,,,^..^,,,_ best, Eliot

_,,,^..^,,,_ flummoxed, Eliot