[Pharo-project] NativeBoost FFI
Hi guys, Over the weekend I had a look at NativeBoost FFI and I have enjoyed it. I really like this idea of taming the beast from high-level languages. However I ran into some problems with the number of arguments one can pass to a smaltalk method (it turns out to be 15 at max)... That bad for FFI especially when you want to interface with legacy c code without touching the C-stuff (header, object files, etc). In consequence I've hacked my way around this limitation by introduction of a new type of arguments that we can pass to callouts. I call these arguments "in object arguments". The idea isn't new and is very simple, instead of passing 20 arguments to your function just pass one an array with all the arguments you need. Where it turns interesting is that I think we can pass more than just an array we can pass real objects (think about ParameterObject) that hold in their instance variables the arguments we want. lets say that we want to call a function " int function(int x, int y)", now with the NativeBoost wrapper we will get a call something like function_x: x y: y <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int x, int y)) the idea is to replace x: y: with only one argument function: anArgument <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int anArgument@1, int anArgument@2)) or even function: aPoint <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int aPoint@x, int aPoint@y)) Yes this last example breaks the encapsulation a little bit by acceding directly to the instance variable of aPoint... but... well another example will be: function_arg1: a arg2: b <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int a@1, int a@2, int b@3, int b@1, int a@6)) Well that's all that I did this weekend. Attached to this mail you can find a file-out of these developments... they include: 1. the idea presented above 2. refactoring of NBFFICallout argument loader creation 3. a small "extension/fix" of the skipSpace method (in spec parser) to skip all separators not just spaces (think about a header definition with tabs) Well that's it folks, enjoy Cheers -- Dr. Ciprian TEODOROV Ingénieur Développement CAO tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
Hi Ciprian, I ran into same problem when I tried to interface Lapack with FFI a few years ago. So I used this simple idea to let the Compiler automatically pass an array of arguments, with two variants - hacked the compiler - subclassed the compiler See http://code.google.com/p/smallapack/wiki/SmallapackPortingOnSqueak http://bugs.squeak.org/view.php?id=2918 http://www.squeaksource.com/MetacelloRepository/ConfigurationOfSmallapack-ni... http://www.squeaksource.com/Smallapack.html http://www.squeaksource.com/Smallapack/Smallapack-Compiler.trunk-nice.6.mcz Nicolas 2012/10/21 Ciprian Teodorov <ciprian.teodorov@gmail.com>:
Hi guys,
Over the weekend I had a look at NativeBoost FFI and I have enjoyed it. I really like this idea of taming the beast from high-level languages. However I ran into some problems with the number of arguments one can pass to a smaltalk method (it turns out to be 15 at max)... That bad for FFI especially when you want to interface with legacy c code without touching the C-stuff (header, object files, etc).
In consequence I've hacked my way around this limitation by introduction of a new type of arguments that we can pass to callouts. I call these arguments "in object arguments". The idea isn't new and is very simple, instead of passing 20 arguments to your function just pass one an array with all the arguments you need. Where it turns interesting is that I think we can pass more than just an array we can pass real objects (think about ParameterObject) that hold in their instance variables the arguments we want.
lets say that we want to call a function " int function(int x, int y)", now with the NativeBoost wrapper we will get a call something like
function_x: x y: y <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int x, int y))
the idea is to replace x: y: with only one argument
function: anArgument <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int anArgument@1, int anArgument@2))
or even
function: aPoint <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int aPoint@x, int aPoint@y))
Yes this last example breaks the encapsulation a little bit by acceding directly to the instance variable of aPoint... but... well
another example will be:
function_arg1: a arg2: b <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int a@1, int a@2, int b@3, int b@1, int a@6))
Well that's all that I did this weekend.
Attached to this mail you can find a file-out of these developments... they include: 1. the idea presented above 2. refactoring of NBFFICallout argument loader creation 3. a small "extension/fix" of the skipSpace method (in spec parser) to skip all separators not just spaces (think about a header definition with tabs)
Well that's it folks, enjoy Cheers -- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
On 21 October 2012 23:51, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Hi Ciprian,
I ran into same problem when I tried to interface Lapack with FFI a few years ago. So I used this simple idea to let the Compiler automatically pass an array of arguments, with two variants - hacked the compiler - subclassed the compiler
Just as a side note, today i think that 15 args is too much. I would even put less, say 5. To my thinking, methods which taking more than 5 arguments need to be refactored to /dev/null, and developers who think that it is good idea to make so much arguments be sent to correction facility :)
See http://code.google.com/p/smallapack/wiki/SmallapackPortingOnSqueak http://bugs.squeak.org/view.php?id=2918 http://www.squeaksource.com/MetacelloRepository/ConfigurationOfSmallapack-ni... http://www.squeaksource.com/Smallapack.html http://www.squeaksource.com/Smallapack/Smallapack-Compiler.trunk-nice.6.mcz
Nicolas
-- Best regards, Igor Stasenko.
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity. And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;) Nicolas 2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 21 October 2012 23:51, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Hi Ciprian,
I ran into same problem when I tried to interface Lapack with FFI a few years ago. So I used this simple idea to let the Compiler automatically pass an array of arguments, with two variants - hacked the compiler - subclassed the compiler
Just as a side note, today i think that 15 args is too much. I would even put less, say 5. To my thinking, methods which taking more than 5 arguments need to be refactored to /dev/null, and developers who think that it is good idea to make so much arguments be sent to correction facility :)
See http://code.google.com/p/smallapack/wiki/SmallapackPortingOnSqueak http://bugs.squeak.org/view.php?id=2918 http://www.squeaksource.com/MetacelloRepository/ConfigurationOfSmallapack-ni... http://www.squeaksource.com/Smallapack.html http://www.squeaksource.com/Smallapack/Smallapack-Compiler.trunk-nice.6.mcz
Nicolas
-- Best regards, Igor Stasenko.
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay. Because first, you introduce it because you need it to deal with external world (only for 1% of cases, and in 0% of cases for good written smalltalk code). But that's only at beginning. Once people discover that they have no limits, i bet they will start abusing it in most horrible manner you ever seen. Copy-paste is worst enemy of programmer. But usually not to one who using it, but one who needs to maintain the code after :)
Nicolas
-- Best regards, Igor Stasenko.
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all. Otherwise, the alternative would be to write all interfacing code in C (Slang would not be an option with 5 parameters) and create new primitives/plugins. The exact opposite of FFI. Or even worse in terms of costs/benefits, rewrite LAPACK+BLAS.
Because first, you introduce it because you need it to deal with external world (only for 1% of cases, and in 0% of cases for good written smalltalk code). But that's only at beginning. Once people discover that they have no limits, i bet they will start abusing it in most horrible manner you ever seen.
With such arguments you will soon want to protect us against some evil super powers like become: thisContext etc... ;) Don't castrate us and let us manage the risk of having 15+ parameters. Nicolas
Copy-paste is worst enemy of programmer. But usually not to one who using it, but one who needs to maintain the code after :)
Nicolas
-- Best regards, Igor Stasenko.
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like: 1. passing as array args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork self callFn: args. 2. passing as instance of class, or external structure: args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args. admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more. Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
Otherwise, the alternative would be to write all interfacing code in C (Slang would not be an option with 5 parameters) and create new primitives/plugins. The exact opposite of FFI.
Or even worse in terms of costs/benefits, rewrite LAPACK+BLAS.
Because first, you introduce it because you need it to deal with external world (only for 1% of cases, and in 0% of cases for good written smalltalk code). But that's only at beginning. Once people discover that they have no limits, i bet they will start abusing it in most horrible manner you ever seen.
With such arguments you will soon want to protect us against some evil super powers like become: thisContext etc... ;) Don't castrate us and let us manage the risk of having 15+ parameters.
Yes, you are right. Another philosophy of mine is that system should not put artificial barriers for developer. He should be able to choose what is good for him, not system nor its authors. So i have two contradicting philosophies :) I think we can easily have more than 15 args with Opal compiler. And i will gladly add support for that in NativeBoost. Fixing existing compiler is a waste of time as to me.
Nicolas
-- Best regards, Igor Stasenko.
Ciprian, i took a look at your code. And I like it, except some naming choices, but that is not a big deal. :) There's one thing is missing in NBSTInObjectArgument: bounds checking. I would also add an option to control generating code for bounds checking for indirect arguments, turned on by default. Mainly this is all about following method: NBSTInObjectArgument>>emitLoad: gen | baseAddress | gen proxy stackValue: stackIndex. + gen optCheckIndirectArgBounds ifTrue: [ + "generate code here to check that index is valid for passed argument, otherwise fail the primitive" + ] gen proxy fetchPointer: elementIndex - 1 ofObject: gen asm EAX. ^ gen asm EAX May i ask you to give me your nickname on squeaksource, so i will add you as developer, so you can upload code there? -- Best regards, Igor Stasenko.
Also, to generalize the concept i would allow @<index> syntax for all arguments (except from constants of course). (Btw, is everyone ok with '@' syntax?) - first change parser's #parseArgument to pass additional argument to requestor, i.e. send #argName: indirectIndex: type: ptrArity: instead of #argName:type:ptrArity: Like that , you won't need to parse it again in NBFFICallout but just check index for being not-nil. Second, refactor NBSTInObjectArgument to be a wrapper around another loader, i.e: NBFnArgument subclass: #NBSTInObjectArgument instanceVariableNames: 'argumentLoader elementIndex' classVariableNames: '' poolDictionaries: '' category: 'NativeBoost-Core-FFI' NBSTInObjectArgument>>emitLoad: gen | baseAddress | argumentLoader emitLoad: gen to: gen asm EAX. gen proxy fetchPointer: elementIndex - 1 ofObject: gen asm EAX. ^ gen asm EAX Like that , a @<index> syntax can be used not just for method arguments but for any other, e.g. self@<index> , ivar@<index> -- Best regards, Igor Stasenko.
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args. self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down. Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration. Cheers, Henry
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
Hi guys, First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;) Now it will be a difficult for me to address all the issues raised ... however I will try First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit. The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz <http://www.graphviz.org/>, metis<http://www.cs.umn.edu/~metis>, abc <http://www.eecs.berkeley.edu/~alanmi/abc/>, vpr<http://www.eecg.toronto.edu/~vaughn/vpr/vpr.html>to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances... Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml<http://www.sdml.info/projects/srcml/>, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work. Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation. Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;) As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea. However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;) Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords! As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments? Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list). Cheers, Ciprian Teodorov On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
HI, Ciprian. I added you to developers. Just single comment about auto-generated wrapper(s)/bindings: You need to do it only once. Then it will not be needed. Yes, you can do it automated or manually (automated is preferred), but since you do it only once, even if manually, this is not going to be too big bottleneck. Sure things, having a tool(s) which can automate that would be nice. But according to my experience, you can automate it only partially, but not fully. There , of course, exceptions like OpenGL library which API design allows to automate wrapping. And sure thing it would be nice to have some code generators to import from C header(s). On 22 October 2012 19:52, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi guys,
First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;)
Now it will be a difficult for me to address all the issues raised ... however I will try
First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit.
The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz, metis, abc, vpr to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances...
Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work.
Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation.
Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;)
As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea.
However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;)
Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments?
Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list).
Cheers, Ciprian Teodorov
On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
Hi Igor, There seems to be an issue with the squeaksource repo... I don't see myself listed amongst the developers nor can I commit I'm using this connection string: MCHttpRepository location: 'http://www.squeaksource.com/NativeBoost' user: 'cip.t' password: '*******' Cheers Ciprian On Tue, Oct 23, 2012 at 10:20 AM, Igor Stasenko <siguctua@gmail.com> wrote:
HI, Ciprian. I added you to developers.
Just single comment about auto-generated wrapper(s)/bindings: You need to do it only once. Then it will not be needed.
Yes, you can do it automated or manually (automated is preferred), but since you do it only once, even if manually, this is not going to be too big bottleneck. Sure things, having a tool(s) which can automate that would be nice. But according to my experience, you can automate it only partially, but not fully. There , of course, exceptions like OpenGL library which API design allows to automate wrapping.
And sure thing it would be nice to have some code generators to import from C header(s).
On 22 October 2012 19:52, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi guys,
First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;)
Now it will be a difficult for me to address all the issues raised ... however I will try
First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit.
The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz, metis, abc, vpr to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances...
Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work.
Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation.
Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;)
As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea.
However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;)
Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments?
Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list).
Cheers, Ciprian Teodorov
On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables
;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader
(
in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
sorry, it looks like i forgot to hit 'save' button after adding you to project. On 23 October 2012 19:35, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi Igor,
There seems to be an issue with the squeaksource repo... I don't see myself listed amongst the developers nor can I commit
I'm using this connection string:
MCHttpRepository location: 'http://www.squeaksource.com/NativeBoost'
user: 'cip.t' password: '*******'
Cheers Ciprian
On Tue, Oct 23, 2012 at 10:20 AM, Igor Stasenko <siguctua@gmail.com> wrote:
HI, Ciprian. I added you to developers.
Just single comment about auto-generated wrapper(s)/bindings: You need to do it only once. Then it will not be needed.
Yes, you can do it automated or manually (automated is preferred), but since you do it only once, even if manually, this is not going to be too big bottleneck. Sure things, having a tool(s) which can automate that would be nice. But according to my experience, you can automate it only partially, but not fully. There , of course, exceptions like OpenGL library which API design allows to automate wrapping.
And sure thing it would be nice to have some code generators to import from C header(s).
On 22 October 2012 19:52, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi guys,
First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;)
Now it will be a difficult for me to address all the issues raised ... however I will try
First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit.
The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz, metis, abc, vpr to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances...
Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work.
Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation.
Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;)
As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea.
However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;)
Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments?
Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list).
Cheers, Ciprian Teodorov
On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
May be we should ask dave mason because he wrote a C header parser or something like that. On Oct 23, 2012, at 7:35 PM, Ciprian Teodorov wrote:
Hi Igor,
There seems to be an issue with the squeaksource repo... I don't see myself listed amongst the developers nor can I commit
I'm using this connection string: MCHttpRepository location: 'http://www.squeaksource.com/NativeBoost'
user: 'cip.t'
password: '*******'
Cheers Ciprian
On Tue, Oct 23, 2012 at 10:20 AM, Igor Stasenko <siguctua@gmail.com> wrote: HI, Ciprian. I added you to developers.
Just single comment about auto-generated wrapper(s)/bindings: You need to do it only once. Then it will not be needed.
Yes, you can do it automated or manually (automated is preferred), but since you do it only once, even if manually, this is not going to be too big bottleneck. Sure things, having a tool(s) which can automate that would be nice. But according to my experience, you can automate it only partially, but not fully. There , of course, exceptions like OpenGL library which API design allows to automate wrapping.
And sure thing it would be nice to have some code generators to import from C header(s).
On 22 October 2012 19:52, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi guys,
First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;)
Now it will be a difficult for me to address all the issues raised ... however I will try
First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit.
The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz, metis, abc, vpr to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances...
Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work.
Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation.
Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;)
As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea.
However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;)
Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments?
Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list).
Cheers, Ciprian Teodorov
On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
Hi Igor, I finally managed to commit the stuff. Now I looked a little bit into the idea of generalizing it to work with any loader, which seems to work pretty much. Now I have two questions: First of all, what about #argName:type:arity? Should we remove it and replace it everywhere with #argName:indirectIndex:type:arity ? Or we should keep it? I don't quite grasp the impact such a change will have on the whole machinery (Ex. NBNativeFunctionGen, and NativeArgument). Secondly, i was thinking about ways to check the indirect bounds... and I'm quite dry of ideas... Is there any generic way to check the bounds of an object, or we should do that on a case by case basis? Typically I would think that we need something that gives us the size of an object in bytes, however I'm not sure that all our objects are contiguous in memory (or if they are that they will be like that forever), moreover what about smallintegers? Should we restrict the approach to Arrays only? well that's it for now Cheers, Ciprian On Wed, Oct 24, 2012 at 8:47 PM, Stéphane Ducasse <stephane.ducasse@inria.fr
wrote:
May be we should ask dave mason because he wrote a C header parser or something like that.
On Oct 23, 2012, at 7:35 PM, Ciprian Teodorov wrote:
Hi Igor,
There seems to be an issue with the squeaksource repo... I don't see myself listed amongst the developers nor can I commit
I'm using this connection string: MCHttpRepository location: 'http://www.squeaksource.com/NativeBoost'
user: 'cip.t'
password: '*******'
Cheers Ciprian
On Tue, Oct 23, 2012 at 10:20 AM, Igor Stasenko <siguctua@gmail.com> wrote: HI, Ciprian. I added you to developers.
Just single comment about auto-generated wrapper(s)/bindings: You need to do it only once. Then it will not be needed.
Yes, you can do it automated or manually (automated is preferred), but since you do it only once, even if manually, this is not going to be too big bottleneck. Sure things, having a tool(s) which can automate that would be nice. But according to my experience, you can automate it only partially, but not fully. There , of course, exceptions like OpenGL library which API design allows to automate wrapping.
And sure thing it would be nice to have some code generators to import from C header(s).
On 22 October 2012 19:52, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi guys,
First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;)
Now it will be a difficult for me to address all the issues raised ... however I will try
First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit.
The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz, metis, abc, vpr to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances...
Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work.
Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation.
Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;)
As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea.
However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;)
Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments?
Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list).
Cheers, Ciprian Teodorov
On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for
the
sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument
loader (
in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
On 24 October 2012 21:55, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi Igor,
I finally managed to commit the stuff. Now I looked a little bit into the idea of generalizing it to work with any loader, which seems to work pretty much. Now I have two questions: First of all, what about #argName:type:arity? Should we remove it and replace it everywhere with #argName:indirectIndex:type:arity ? Or we should keep it? I don't quite grasp the impact such a change will have on the whole machinery (Ex. NBNativeFunctionGen, and NativeArgument).
This is a contract between parser and its requestor. Since we changing contract, all requestors should use new protocol. NBNativeFunctionGen can simply ignore that, because they don't handle smalltalk objects. NativeArgument is encapsulation of parsed argument. But since index transformed into different loader, there no need to change anything for it.
Secondly, i was thinking about ways to check the indirect bounds... and I'm quite dry of ideas... Is there any generic way to check the bounds of an object, or we should do that on a case by case basis? Typically I would think that we need something that gives us the size of an object in bytes, however I'm not sure that all our objects are contiguous in memory (or if they are that they will be like that forever), moreover what about smallintegers? Should we restrict the approach to Arrays only?
Look at InterpreterProxy, it has such functionality. And sure thing smallints should no be allowed. But if you find it hard to implement, just leave 'self halt' there, and meanwhile use callouts with option to check bounds turned off. I will implement that later.
well that's it for now Cheers, Ciprian
On Wed, Oct 24, 2012 at 8:47 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
May be we should ask dave mason because he wrote a C header parser or something like that.
On Oct 23, 2012, at 7:35 PM, Ciprian Teodorov wrote:
Hi Igor,
There seems to be an issue with the squeaksource repo... I don't see myself listed amongst the developers nor can I commit
I'm using this connection string: MCHttpRepository location: 'http://www.squeaksource.com/NativeBoost'
user: 'cip.t'
password: '*******'
Cheers Ciprian
On Tue, Oct 23, 2012 at 10:20 AM, Igor Stasenko <siguctua@gmail.com> wrote: HI, Ciprian. I added you to developers.
Just single comment about auto-generated wrapper(s)/bindings: You need to do it only once. Then it will not be needed.
Yes, you can do it automated or manually (automated is preferred), but since you do it only once, even if manually, this is not going to be too big bottleneck. Sure things, having a tool(s) which can automate that would be nice. But according to my experience, you can automate it only partially, but not fully. There , of course, exceptions like OpenGL library which API design allows to automate wrapping.
And sure thing it would be nice to have some code generators to import from C header(s).
On 22 October 2012 19:52, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi guys,
First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;)
Now it will be a difficult for me to address all the issues raised ... however I will try
First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit.
The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz, metis, abc, vpr to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances...
Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work.
Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation.
Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;)
As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea.
However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;)
Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments?
Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list).
Cheers, Ciprian Teodorov
On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
Hi Igor, I have commited the refactoring that you have suggested. Namely, using the indirect loader as a wrapper of another loader. I have renamed the class to NBSTIndirectArgument (I think it better reflects its purpose).... I would appreciate though if you can have a look at the "bound checking" that I have implemented for the #emitLoad: since I'm not really an expert user of nativeboost ;). Cheers, Ciprian On Thu, Oct 25, 2012 at 4:15 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 24 October 2012 21:55, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi Igor,
I finally managed to commit the stuff. Now I looked a little bit into the idea of generalizing it to work with any loader, which seems to work pretty much. Now I have two questions: First of all, what about #argName:type:arity? Should we remove it and replace it everywhere with #argName:indirectIndex:type:arity ? Or we should keep it? I don't quite grasp the impact such a change will have on the whole machinery (Ex. NBNativeFunctionGen, and NativeArgument).
This is a contract between parser and its requestor. Since we changing contract, all requestors should use new protocol. NBNativeFunctionGen can simply ignore that, because they don't handle smalltalk objects. NativeArgument is encapsulation of parsed argument. But since index transformed into different loader, there no need to change anything for it.
Secondly, i was thinking about ways to check the indirect bounds... and I'm quite dry of ideas... Is there any generic way to check the bounds of an object, or we should do that on a case by case basis? Typically I would think that we need something that gives us the size of an object in bytes, however I'm not sure that all our objects are contiguous in memory (or if they are that they will be like that forever), moreover what about smallintegers? Should we restrict the approach to Arrays only?
Look at InterpreterProxy, it has such functionality. And sure thing smallints should no be allowed. But if you find it hard to implement, just leave 'self halt' there, and meanwhile use callouts with option to check bounds turned off. I will implement that later.
well that's it for now Cheers, Ciprian
On Wed, Oct 24, 2012 at 8:47 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
May be we should ask dave mason because he wrote a C header parser or something like that.
On Oct 23, 2012, at 7:35 PM, Ciprian Teodorov wrote:
Hi Igor,
There seems to be an issue with the squeaksource repo... I don't see myself listed amongst the developers nor can I commit
I'm using this connection string: MCHttpRepository location: 'http://www.squeaksource.com/NativeBoost'
user: 'cip.t'
password: '*******'
Cheers Ciprian
On Tue, Oct 23, 2012 at 10:20 AM, Igor Stasenko <siguctua@gmail.com> wrote: HI, Ciprian. I added you to developers.
Just single comment about auto-generated wrapper(s)/bindings: You need to do it only once. Then it will not be needed.
Yes, you can do it automated or manually (automated is preferred), but since you do it only once, even if manually, this is not going to be too big bottleneck. Sure things, having a tool(s) which can automate that would be nice. But according to my experience, you can automate it only partially, but not fully. There , of course, exceptions like OpenGL library which API design allows to automate wrapping.
And sure thing it would be nice to have some code generators to import from C header(s).
On 22 October 2012 19:52, Ciprian Teodorov <
ciprian.teodorov@gmail.com>
wrote:
Hi guys,
First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;)
Now it will be a difficult for me to address all the issues raised ... however I will try
First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit.
The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz, metis, abc, vpr to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances...
Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work.
Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation.
Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;)
As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea.
However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;)
Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments?
Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list).
Cheers, Ciprian Teodorov
On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS
for
the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much
less
space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
On 26 October 2012 19:49, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi Igor,
I have commited the refactoring that you have suggested. Namely, using the indirect loader as a wrapper of another loader. I have renamed the class to NBSTIndirectArgument (I think it better reflects its purpose)....
I would appreciate though if you can have a look at the "bound checking" that I have implemented for the #emitLoad: since I'm not really an expert user of nativeboost ;).
sure, i will review the code. Thanks for contribution!
Cheers, Ciprian
On Thu, Oct 25, 2012 at 4:15 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 24 October 2012 21:55, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi Igor,
I finally managed to commit the stuff. Now I looked a little bit into the idea of generalizing it to work with any loader, which seems to work pretty much. Now I have two questions: First of all, what about #argName:type:arity? Should we remove it and replace it everywhere with #argName:indirectIndex:type:arity ? Or we should keep it? I don't quite grasp the impact such a change will have on the whole machinery (Ex. NBNativeFunctionGen, and NativeArgument).
This is a contract between parser and its requestor. Since we changing contract, all requestors should use new protocol. NBNativeFunctionGen can simply ignore that, because they don't handle smalltalk objects. NativeArgument is encapsulation of parsed argument. But since index transformed into different loader, there no need to change anything for it.
Secondly, i was thinking about ways to check the indirect bounds... and I'm quite dry of ideas... Is there any generic way to check the bounds of an object, or we should do that on a case by case basis? Typically I would think that we need something that gives us the size of an object in bytes, however I'm not sure that all our objects are contiguous in memory (or if they are that they will be like that forever), moreover what about smallintegers? Should we restrict the approach to Arrays only?
Look at InterpreterProxy, it has such functionality. And sure thing smallints should no be allowed. But if you find it hard to implement, just leave 'self halt' there, and meanwhile use callouts with option to check bounds turned off. I will implement that later.
well that's it for now Cheers, Ciprian
On Wed, Oct 24, 2012 at 8:47 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
May be we should ask dave mason because he wrote a C header parser or something like that.
On Oct 23, 2012, at 7:35 PM, Ciprian Teodorov wrote:
Hi Igor,
There seems to be an issue with the squeaksource repo... I don't see myself listed amongst the developers nor can I commit
I'm using this connection string: MCHttpRepository location: 'http://www.squeaksource.com/NativeBoost'
user: 'cip.t'
password: '*******'
Cheers Ciprian
On Tue, Oct 23, 2012 at 10:20 AM, Igor Stasenko <siguctua@gmail.com> wrote: HI, Ciprian. I added you to developers.
Just single comment about auto-generated wrapper(s)/bindings: You need to do it only once. Then it will not be needed.
Yes, you can do it automated or manually (automated is preferred), but since you do it only once, even if manually, this is not going to be too big bottleneck. Sure things, having a tool(s) which can automate that would be nice. But according to my experience, you can automate it only partially, but not fully. There , of course, exceptions like OpenGL library which API design allows to automate wrapping.
And sure thing it would be nice to have some code generators to import from C header(s).
On 22 October 2012 19:52, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi guys,
First of all sorry for not saying anything the whole day ... but I don't have internet connection at work. I hope I'll be excused though ;)
Now it will be a difficult for me to address all the issues raised ... however I will try
First of all, IMHO any FFI should facilitate the task of the person writing the library wrapper. This comes the issue of (hopefully automatically) parsing C headers and generating the calls. Why?... well because if I take for instance the BLAS/LAPACK case (which was cited today) have over 1000 functions exported. Even if it is not 1000 lets say that you have a smaller library from which you want to use only one function X but in order to use that precise function you will maybe have to initialize some C context data, or a specific data structure. Of course you can do that by hand, but what if you really want to benefit from the library without diving into the details... well in that case you are a little bit stuck, either you write by hand a bunch of wrappers or you quit.
The problem is that nice libraries that you want to use they usually have horrible data structures behind that you cannot set up easily. This happened to me several times in the future while trying to use different external library calls (graphviz, metis, abc, vpr to name a few) in Pharo (with FFI, with Alien, etc). In almost all cases I ended up generating text files and calling some bloody c program using these files as input. The problem is that has a huge impact on performances...
Now with nativeboost I found it pretty easy to automatically generate some usable binding... I parsed the C headers with srcml, the I have parsed the XML looking for function definitions... For these definitions I've generated the wrappers. Only that for over 15 argument functions that did not work.
Now, I do not like the idea of hacking the compiler, or even subclassing it like Nicolas did for Smallapack. And luckily we do not need to do that with the arguments in an array trick... Moreover, I completely agree with Igor on the fact that 15 arguments are to many, however Pharo really needs to have an automatable FFI generation.
Igor, I have seen that you have mentioned some work-arounds using instance variables, and/or NBExternalStructures. Both these ideas are great, especially the use of the NBExternalStructure, however in my opinion if you want to generate them automatically you've got yourself an even bigger problem. Will you generate a new class for each function call that you have? What about the FFI being a wormhole to an ugly and mean world? I think we should try to reduce the size of that hole ;) Though, you have a point with using named arguments, and I think that is a cool solution too. That is why I was speaking about directly accessing the instance variables by name. I simply didn't know enough about the hidden powers of NBExternalStructure. ;)
As for extending the nativeboost signatures and accessing object fields by name, I completely agree with you that it is not a good idea.
However, using the indices of arrays I think is only a small addition that can have a huge impact on the way people use NativeBoost FFI without adding to much extra-overhead. By the way, I think we definitely have to do the bound-checking trick by default. Moreover, having the NBSTInObjectArgument as a wrapper over another loader is a great idea, giving us another degree of "controlled" freedom. ;)
Thanks Henrik for the good joke (I didn't see it coming :)), It makes me hate this idea now. :P
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
As for supporting or not more than 15 arguments for typical method calls... I don't know... maybe is good maybe is bad, but I think there are other things that need to be done before making a fuss about it. By the way is it an arbitrary limit, or it is imposed by the use of 4 bytes for storing the number of arguments?
Now let me come back to practical issues. My squeaksource id is cip.t (you can find me searching "ciprian" in the members list).
Cheers, Ciprian Teodorov
On Mon, Oct 22, 2012 at 1:09 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 22 October 2012 12:00, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 22.10.2012 02:37, Igor Stasenko wrote:
On 22 October 2012 01:59, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2012/10/22 Igor Stasenko <siguctua@gmail.com>:
On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity.
And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;)
Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay.
That's not exactly the philosophy behind FFI. FFI is here to let the user manage the external chaotic world. OK, external peels of the onion should have 5 parameters or less. Near the sprout, you can't raise such barriers, or there is no onion at all.
Well, that's part of developer's responsibility, how to prevent chaos. Needless to say, nobody wants to deal with so many arguments at once (too much space for mistakes). As for my workaround: this mainly, how you tame the complexity in case it is inevitable? Look how code to call that function will look like:
1. passing as array
args := Array new: 100. args at:1 put: x; at:2 put: y; ... at: 100 put: zork
self callFn: args.
self callFn: {x . y} Looks kinda familiar doesnt it? (hint: swap { for ( and . for , ) I for one welcome our new syntactic overlords!
Yes, it looks familiar, but cannot tell where i seen it. Gah.. how i could forget about it?
2. passing as instance of class, or external structure:
args := MyFunctionArgs new. args firstArgumentName: x; secondArgumentName: y; ... hundrethArgumentName: zork. self callFn: args.
admit that dealing with names instead of numbers leaves much less space for mistakes and serves for better clarity at same time. So, even if it is more cumbersome because requires defining extra class, at the end you win much more.
Anyways, if people think it is worth adding indirect argument loader ( in form of param@<index>, but not param@ivar), we can introduce that.
While I often find this a good idea for maintainability, it sorta flies in the face of another of ST's strengths, iterative/exporatory programming. If you are forced to think up front about which parameter classes you need due to a small limit, rather than introduce them ad-hoc when the code really needs the refactoring to remain legible, it slows you down.
Not thinking of FFI specifically, but I have seen lots of evolved mathematical models where a 5-parameter limit upfront would probably lead to either: a) *Really* bad code, ie. making the calculation object stateful by storing in instvars instead. (and in the process, make it really hard to know which instvars are actually part of object state and not temp vars of some calculation) b) Switching to another programming language out of frustration.
keyword message syntax is bad for many arguments.. for such cases i find a positional argument notation more appeal because it is more compact. In any case, a complex math formulas smell equally bad in any programming language (sometime even if written by hand on paper using math notation(s) ;)
Cheers, Henry
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
-- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
Hi, Ciprian. Yes, the number of arguments limit is a problem (which occurs 1% of the time), but if it occurs, it beating you hard :) But NativeBoost allows you to use instance variable names in function signature, (the max number of inst vars is 255, which is more than enough i think ;) ), and what is most important (as you noted) it doesn't breaking encapsulation, since methods having full right for accessing receiver's instance variables in their class scope. So, for making a binding to function which takes too many arguments, what you can do is to create a class: Object subclass: #MyBigFunctionBinding instanceVariableNames: 'var1 var2 .... var100' .. And then implement a method for FFI callout in it: MyBigFunctionBinding>>callThatFatThing <primitive: > ^ self call: #( int function (int var1, int var2 , ....... var100 )) Another approach is to create an NBExternalStructure which will have same memory layout as arguments of function (so if you pass it by value, it will push exactly same number of values in right order on stack): NBExternalStructure subclass: #BloatedArgs .. BloatedFnArgs class>>fieldsDesc #( int arg1; int arg2; . ... int arg200; ) then you can just do: MyBigFunctionBinding>>callThatFatThing: args <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (BloatedFnArgs args )) So, as you can see from above, you can deal with this problem one way or another. And given that it is just 1% of all C functions (or even less), personally, i don't think that it is worth extending a signature syntax. On 21 October 2012 21:36, Ciprian Teodorov <ciprian.teodorov@gmail.com> wrote:
Hi guys,
Over the weekend I had a look at NativeBoost FFI and I have enjoyed it. I really like this idea of taming the beast from high-level languages. However I ran into some problems with the number of arguments one can pass to a smaltalk method (it turns out to be 15 at max)... That bad for FFI especially when you want to interface with legacy c code without touching the C-stuff (header, object files, etc).
In consequence I've hacked my way around this limitation by introduction of a new type of arguments that we can pass to callouts. I call these arguments "in object arguments". The idea isn't new and is very simple, instead of passing 20 arguments to your function just pass one an array with all the arguments you need. Where it turns interesting is that I think we can pass more than just an array we can pass real objects (think about ParameterObject) that hold in their instance variables the arguments we want.
lets say that we want to call a function " int function(int x, int y)", now with the NativeBoost wrapper we will get a call something like
function_x: x y: y <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int x, int y))
the idea is to replace x: y: with only one argument
function: anArgument <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int anArgument@1, int anArgument@2))
or even
function: aPoint <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int aPoint@x, int aPoint@y))
Yes this last example breaks the encapsulation a little bit by acceding directly to the instance variable of aPoint... but... well
this is not going to work unless you specify the class to use (which contains such instance variables), which will make signatures even more uglier, something like: function: aPoint <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int aPoint@Point:x, int aPoint@Point:y)) and that amount of mess beyond acceptance levels, to my taste :)
another example will be:
function_arg1: a arg2: b <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int a@1, int a@2, int b@3, int b@1, int a@6))
Well that's all that I did this weekend.
Attached to this mail you can find a file-out of these developments... they include: 1. the idea presented above 2. refactoring of NBFFICallout argument loader creation 3. a small "extension/fix" of the skipSpace method (in spec parser) to skip all separators not just spaces (think about a header definition with tabs)
Thanks , i will look at it.
Well that's it folks, enjoy Cheers -- Dr. Ciprian TEODOROV Ingénieur Développement CAO
tél : 06 08 54 73 48 mail : ciprian.teodorov@gmail.com www.teodorov.ro
-- Best regards, Igor Stasenko.
On 22 October 2012 00:31, Igor Stasenko <siguctua@gmail.com> wrote:
Hi, Ciprian.
this is not going to work unless you specify the class to use (which contains such instance variables), which will make signatures even more uglier, something like:
function: aPoint <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int aPoint@Point:x, int aPoint@Point:y))
and that amount of mess beyond acceptance levels, to my taste :)
and even more than that, if you change the shape of a class (like remove instance variable or change their order), this method won't recompile automatically, means that next time you attempt to use it, you will crash the system, or get weird behavior. While if method sits in same class and accessing its instance variables, it is completely safe, because when you changing shape of a class, it will automatically recompile all its methods and therefore wipe generated code out. -- Best regards, Igor Stasenko.
participants (5)
-
Ciprian Teodorov -
Henrik Sperre Johansen -
Igor Stasenko -
Nicolas Cellier -
Stéphane Ducasse