SWIG for Native Boost FFI
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ . It works by creating a C side code and Smalltalk that can be filed-in in a image, which contains a class with all the NativeBoost-FFI callouts, a class for the global module, classes for each C++ class and SWIGTYPE_* classes for opaque types or types that SWIG couldn't wrap. It currently can generate bindings for C/C++ functions, constants, define based constants, opaque structures, methods, enums, and for C++ classes, nested classes, member methods, member fields, member constants, member static functions, variables and constants. The overloaded functions support currently is not so great, since I am currently using the arguments name to build the selector. I have to still decide a better strategy for the overloaded methods. WARNING: automatic garbage collection of the C++ classes is not yet ready, but it will be soon. Downcasting of C++ objects is also not supported yet. Greetings Ronie Salgado
Wow! I guess this complement well NativeBoost right? Alexandre On Jan 22, 2014, at 5:25 PM, Ronie Salgado <roniesalg@gmail.com> wrote:
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
It works by creating a C side code and Smalltalk that can be filed-in in a image, which contains a class with all the NativeBoost-FFI callouts, a class for the global module, classes for each C++ class and SWIGTYPE_* classes for opaque types or types that SWIG couldn't wrap.
It currently can generate bindings for C/C++ functions, constants, define based constants, opaque structures, methods, enums, and for C++ classes, nested classes, member methods, member fields, member constants, member static functions, variables and constants.
The overloaded functions support currently is not so great, since I am currently using the arguments name to build the selector. I have to still decide a better strategy for the overloaded methods.
WARNING: automatic garbage collection of the C++ classes is not yet ready, but it will be soon. Downcasting of C++ objects is also not supported yet.
Greetings Ronie Salgado
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
Marvelous news. --- Philippe Back Dramatic Performance Improvements Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 Mail:phil@highoctane.be | Web: http://philippeback.eu Blog: http://philippeback.be | Twitter: @philippeback Youtube: http://www.youtube.com/user/philippeback/videos High Octane SPRL rue cour Boisacq 101 | 1301 Bierges | Belgium Pharo Consortium Member - http://consortium.pharo.org/ Featured on the Software Process and Measurement Cast - http://spamcast.libsyn.com Sparx Systems Enterprise Architect and Ability Engineering EADocX Value Added Reseller On Wed, Jan 22, 2014 at 10:49 PM, Alexandre Bergel <alexandre.bergel@me.com>wrote:
Wow! I guess this complement well NativeBoost right?
Alexandre
On Jan 22, 2014, at 5:25 PM, Ronie Salgado <roniesalg@gmail.com> wrote:
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
It works by creating a C side code and Smalltalk that can be filed-in in a image, which contains a class with all the NativeBoost-FFI callouts, a class for the global module, classes for each C++ class and SWIGTYPE_* classes for opaque types or types that SWIG couldn't wrap.
It currently can generate bindings for C/C++ functions, constants, define based constants, opaque structures, methods, enums, and for C++ classes, nested classes, member methods, member fields, member constants, member static functions, variables and constants.
The overloaded functions support currently is not so great, since I am currently using the arguments name to build the selector. I have to still decide a better strategy for the overloaded methods.
WARNING: automatic garbage collection of the C++ classes is not yet ready, but it will be soon. Downcasting of C++ objects is also not supported yet.
Greetings Ronie Salgado
-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
Hello Ronie, Could you show us a sample script? Cheers, Hernán 2014/1/22 Ronie Salgado <roniesalg@gmail.com>
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
It works by creating a C side code and Smalltalk that can be filed-in in a image, which contains a class with all the NativeBoost-FFI callouts, a class for the global module, classes for each C++ class and SWIGTYPE_* classes for opaque types or types that SWIG couldn't wrap.
It currently can generate bindings for C/C++ functions, constants, define based constants, opaque structures, methods, enums, and for C++ classes, nested classes, member methods, member fields, member constants, member static functions, variables and constants.
The overloaded functions support currently is not so great, since I am currently using the arguments name to build the selector. I have to still decide a better strategy for the overloaded methods.
WARNING: automatic garbage collection of the C++ classes is not yet ready, but it will be soon. Downcasting of C++ objects is also not supported yet.
Greetings Ronie Salgado
Hi Ronie: On 22 Jan 2014, at 21:25, Ronie Salgado <roniesalg@gmail.com> wrote:
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
Just out of curiosity, how does it compare to TalkFFI? (http://lists.gforge.inria.fr/pipermail/pharo-project/2013-January/072847.htm..., http://vimeo.com/63472765) Best regards Stefan -- Stefan Marr INRIA Lille - Nord Europe http://stefan-marr.de/research/
Its completely different approach. SWIG is a tool designed to create bindings, for which you provide an interface descripitions, which usually includes your C/C++ headers, and you provide some SWIG specific directives for improving the generated bindings. It produces C side module and a language specific module. The C side is required, since some C stuff can only be detected by an actual C compiler. In these matter, maybe TalkFFI is better, because is based in an actual compiler(clang), but maybe you lose some bindings fine tuning. The problem is when you add C++ to the mix. In C++ there's not a standard defined ABI, so a module made in one C++ compiler is almost for sure going to be incompatible with a module made by another C++ compiler. For making bindings to a C++ module, you have to first make C bindings to that module, compile those bindings with the same compiler that was used for the original module and then you can start making NativeBoost bindings. This is the only portable way for C++. For example, gcc and Microsoft Visual C++ compilers are completely incompatible for C++, but for C they can interact. Clang is apparently designed to use the same ABI as gcc, so maybe it could be used in Linux, Mac but not for a closed source library in Window. What SWIG does is generating the C bindings and Smalltalk code that calls the C bindings ready to be filed in. In addition, you can add extra code to the bindings to make things that hard to do using directly NativeBoost but can be easy to do with an intermediate function in C, for example messing with some unions and things that required lot of pointer arithmetic, that could be used maybe for a bad written library that some project needs. P.S I have yet to make an example. When I add some documentation, which is required to be able to integrate into the main SWIG distribution, I will post a new message to the list. Greetings Ronie Salgado 2014/1/23 Stefan Marr <smalltalk@stefan-marr.de>
Hi Ronie:
On 22 Jan 2014, at 21:25, Ronie Salgado <roniesalg@gmail.com> wrote:
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
Just out of curiosity, how does it compare to TalkFFI? ( http://lists.gforge.inria.fr/pipermail/pharo-project/2013-January/072847.htm..., http://vimeo.com/63472765)
Best regards Stefan
-- Stefan Marr INRIA Lille - Nord Europe http://stefan-marr.de/research/
Hi all! On 01/22/2014 09:25 PM, Ronie Salgado wrote:
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
Just wanted to report that this stuff WORKS :) * I cloned down Ronie's SWIG. Did the autogen.sh, ./configure, make, make install dance. * I went into Examples/ruby/value and: * Changed module name to "Example" (instead of "example") * Ran: swig -pharo example.i (produces C code and Example.st) * Compile: gcc -m32 -c -fpic -fPIC example_wrap.c example.c * Link: gcc -m32 -shared example.o example_wrap.o -o Example.so * Then I filed in Example.st into Pharo 2.0, and put Example.so in the directory where I started the pharo VM. And well, after fixing some buglets (if you want to recreate this, email me or ask here) it works! I added a little test class for fun. Fantastic stuff! Sure, not complete by any means, but hey, it worked and not a single VM crash while playing with it. Ronie - good work! If you are on a Linux, I attached these two files. File into Pharo 2.0, open TestRunner and run ExampleTest. regards, Göran
Excellent!!!! That's really a missing block coming to life! I'll try it this weekend on my Linux system. Phil On Fri, Feb 21, 2014 at 10:24 AM, Göran Krampe <goran@krampe.se> wrote:
Hi all!
On 01/22/2014 09:25 PM, Ronie Salgado wrote:
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
Just wanted to report that this stuff WORKS :)
* I cloned down Ronie's SWIG. Did the autogen.sh, ./configure, make, make install dance. * I went into Examples/ruby/value and: * Changed module name to "Example" (instead of "example") * Ran: swig -pharo example.i (produces C code and Example.st) * Compile: gcc -m32 -c -fpic -fPIC example_wrap.c example.c * Link: gcc -m32 -shared example.o example_wrap.o -o Example.so * Then I filed in Example.st into Pharo 2.0, and put Example.so in the directory where I started the pharo VM.
And well, after fixing some buglets (if you want to recreate this, email me or ask here) it works! I added a little test class for fun.
Fantastic stuff! Sure, not complete by any means, but hey, it worked and not a single VM crash while playing with it. Ronie - good work!
If you are on a Linux, I attached these two files. File into Pharo 2.0, open TestRunner and run ExampleTest.
regards, Göran
Hi there!, It is great to hear that it is being useful. Currently I am taking some vacations, so I will probably be without Internet access for some days. Currently the most missing parts are documentation, which I am going to write when having some time. Here are some early details, before I start properly documenting them: ·This is heavily based in the C# bindings. There are additional typemaps such as nbtype, nbin and nbout. They are similar to the cstype, csin and csout typemaps presents the C# generator. ·They are two Smalltalk class and some common superclasses generated. One Smalltalk class contains all the NB-FFI callout methods, another contains wrappers for all the global C/C++ stuff. ·For each C++ a Smalltalk class is generated. ·The common superclass provides a destroy method for destroying an object. ·The extra overhead in new are to register properly with the GC. ·The C pointer is wrapped in a reference object that contains things for properly interact with the GC and destroying C++ object avoiding crashing the VM. Swig usually makes some kind of opaque type for some pointers. If they are intended to be an Array, maybe some extra typemaps has to be added. More information about typemaps is on the SWIG manual. I took that kind of approach because it gives more flexibility, but it provides probably a reduced performance in comparison with placing the callouts directly in the wrapped classes. Some people that saw this when developing this did not like this approach. I will document all of this more properly later, but I have to go now because I am in a hurry. Best regards, Ronie Salgado 2014-02-21 7:16 GMT-03:00 phil@highoctane.be <phil@highoctane.be>:
Excellent!!!!
That's really a missing block coming to life!
I'll try it this weekend on my Linux system.
Phil
On Fri, Feb 21, 2014 at 10:24 AM, Göran Krampe <goran@krampe.se> wrote:
Hi all!
On 01/22/2014 09:25 PM, Ronie Salgado wrote:
I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
Just wanted to report that this stuff WORKS :)
* I cloned down Ronie's SWIG. Did the autogen.sh, ./configure, make, make install dance. * I went into Examples/ruby/value and: * Changed module name to "Example" (instead of "example") * Ran: swig -pharo example.i (produces C code and Example.st) * Compile: gcc -m32 -c -fpic -fPIC example_wrap.c example.c * Link: gcc -m32 -shared example.o example_wrap.o -o Example.so * Then I filed in Example.st into Pharo 2.0, and put Example.so in the directory where I started the pharo VM.
And well, after fixing some buglets (if you want to recreate this, email me or ask here) it works! I added a little test class for fun.
Fantastic stuff! Sure, not complete by any means, but hey, it worked and not a single VM crash while playing with it. Ronie - good work!
If you are on a Linux, I attached these two files. File into Pharo 2.0, open TestRunner and run ExampleTest.
regards, Göran
Really impressive Ronie!! Alexandre
Le 21-02-2014 à 12:50, Ronie Salgado <roniesalg@gmail.com> a écrit :
Hi there!,
It is great to hear that it is being useful. Currently I am taking some vacations, so I will probably be without Internet access for some days. Currently the most missing parts are documentation, which I am going to write when having some time.
Here are some early details, before I start properly documenting them: ·This is heavily based in the C# bindings. There are additional typemaps such as nbtype, nbin and nbout. They are similar to the cstype, csin and csout typemaps presents the C# generator. ·They are two Smalltalk class and some common superclasses generated. One Smalltalk class contains all the NB-FFI callout methods, another contains wrappers for all the global C/C++ stuff. ·For each C++ a Smalltalk class is generated. ·The common superclass provides a destroy method for destroying an object. ·The extra overhead in new are to register properly with the GC. ·The C pointer is wrapped in a reference object that contains things for properly interact with the GC and destroying C++ object avoiding crashing the VM.
Swig usually makes some kind of opaque type for some pointers. If they are intended to be an Array, maybe some extra typemaps has to be added. More information about typemaps is on the SWIG manual.
I took that kind of approach because it gives more flexibility, but it provides probably a reduced performance in comparison with placing the callouts directly in the wrapped classes. Some people that saw this when developing this did not like this approach.
I will document all of this more properly later, but I have to go now because I am in a hurry.
Best regards, Ronie Salgado
2014-02-21 7:16 GMT-03:00 phil@highoctane.be <phil@highoctane.be>:
Excellent!!!!
That's really a missing block coming to life!
I'll try it this weekend on my Linux system.
Phil
On Fri, Feb 21, 2014 at 10:24 AM, Göran Krampe <goran@krampe.se> wrote: Hi all!
On 01/22/2014 09:25 PM, Ronie Salgado wrote: I started working on extending SWIG to generate bindings for NativeBoost FFI, using as a reference the C# binding generator. I am committing my work on this into https://github.com/ronsaldo/swig/ .
Just wanted to report that this stuff WORKS :)
* I cloned down Ronie's SWIG. Did the autogen.sh, ./configure, make, make install dance. * I went into Examples/ruby/value and: * Changed module name to "Example" (instead of "example") * Ran: swig -pharo example.i (produces C code and Example.st) * Compile: gcc -m32 -c -fpic -fPIC example_wrap.c example.c * Link: gcc -m32 -shared example.o example_wrap.o -o Example.so * Then I filed in Example.st into Pharo 2.0, and put Example.so in the directory where I started the pharo VM.
And well, after fixing some buglets (if you want to recreate this, email me or ask here) it works! I added a little test class for fun.
Fantastic stuff! Sure, not complete by any means, but hey, it worked and not a single VM crash while playing with it. Ronie - good work!
If you are on a Linux, I attached these two files. File into Pharo 2.0, open TestRunner and run ExampleTest.
regards, Göran
Hey! On 02/21/2014 06:50 PM, Ronie Salgado wrote:
Hi there!,
It is great to hear that it is being useful. Currently I am taking some vacations, so I will probably be without Internet access for some days. [SNIP]
I took that kind of approach because it gives more flexibility, but it provides probably a reduced performance in comparison with placing the callouts directly in the wrapped classes. Some people that saw this when developing this did not like this approach.
Yeah, I can see what you mean - and I think that's just fine. One bit tricky problem: there are classes generated starting with lower case. That doesn't fly :) Also, LF instead of CRs after signatures makes NB barf when parsing. That was easily solved with a global tr '\n' '\r' of course, the class naming is a bit harder - need to go to the source. I can see if I can find that one :) regards, Göran PS. Tried wrapping Nanomsg.org today, got so far as to start running test code - but then it fails creating a socket - not sure why, might not be your fault at all actually :) When it works I will blog about it.
Hi! Now I have Internet access for a pair of days again. Oops, I had forgotten to commit code, which added warnings for classes starting with lowercase. Unfortunately, there is not a general automated way for dealing with classes starting, that also avoids names clashes. The only solution is to issue a warning telling the problematic class, and allow the programmer to rename it using the %rename(NewName) oldName; directive, by placing it in the interface definition file. About the newlines, I took the opportunity to add code for replacing all of them with CR, in the generated Smalltalk code. Now I remembered a hard to fix bug, with overloaded functions. Currently this uses the arguments names to generate their selector, but it does not take into account their types. The following illustrates the problem: void overloadedFunctionA(int a, int b); -> Module class>>overloadedFunctionA: a b: b void overloadedFunctionA(float a, double b); -> Module class>>overloadedFunctionA: a b: b Different signatures, but the same selector is generated. A method implementation is generated for both of them, but because they have the same selector and belong to the same class, only the last implementation can be seem, because it overwrites the previous ones. Another problem is with C++exception handling, is not implemented yet, but it is possible to support them. Best Regards, Ronie 2014-02-21 15:43 GMT-03:00 Göran Krampe <goran@krampe.se>:
Hey!
On 02/21/2014 06:50 PM, Ronie Salgado wrote:
Hi there!,
It is great to hear that it is being useful. Currently I am taking some vacations, so I will probably be without Internet access for some days.
[SNIP]
I took that kind of approach because it gives more flexibility, but it
provides probably a reduced performance in comparison with placing the callouts directly in the wrapped classes. Some people that saw this when developing this did not like this approach.
Yeah, I can see what you mean - and I think that's just fine.
One bit tricky problem: there are classes generated starting with lower case. That doesn't fly :)
Also, LF instead of CRs after signatures makes NB barf when parsing. That was easily solved with a global tr '\n' '\r' of course, the class naming is a bit harder - need to go to the source.
I can see if I can find that one :)
regards, Göran
PS. Tried wrapping Nanomsg.org today, got so far as to start running test code - but then it fails creating a socket - not sure why, might not be your fault at all actually :) When it works I will blog about it.
participants (6)
-
Alexandre Bergel -
Göran Krampe -
Hernán Morales Durand -
phil＠highoctane.be -
Ronie Salgado -
Stefan Marr