[Pharo-project] NativeBoost examples (was: Re: DoubleArray)
On 23 September 2010 02:00, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
Fair enough :) Â The most recent thing I recall on NativeBoost is Stef asking for what I assume must be a Mac plugin; I thought Linux would be left out too, but it appears not, though it is not in my current vm. Â Beyond some list traffic, I found this:
The work for supporting Linux and Mac is in progress.
 http://code.google.com/p/nativeboost/wiki/NativeBoost
Is there other documentation somewhere? Â Build instructions?
You can find all existing documentation, installation instructions on wiki pages at the above site. There is no other place for it. http://code.google.com/p/nativeboost/wiki/Installation - installation instructions.
You have a block copy, which is good. Â What about indexing, such as getting floats or doubles out of external memory?
This is just a piece of cake. ;) readDoubleFrom: address " This method loads the double from given external address. an address can be an instance of NBExternalAddress, or simple ByteArray with at least 8 bytes long, which holds a double floating value" <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'> "We are using a pseudo-function prototype and supplying own assembler code, instead of making a call to external function. In given example, an address argument , after coercion is pushed on stack. " ^ NBFFICallout cdecl: #( double ( void * address) ) " A pseudo-function takes 1 argument, and returns double value. Under cdecl call convention, all floating point return types is returned in fp(0) CPU register" emitCall: [:gen | | asm | asm := gen asm. "Here , we expecting that an address value is already pushed on stack" asm pop: asm EAX; "load an address value into EAX register by popping a stack" fld: (asm EAX ptr64). "load a floating point value from memory, at base address, held in EAX register into fp(0) register, we are using #ptr64, to indicate that memory operand size is 64bits long" " return value set, we are done. A code generator will take care for emitting code, which converts a double floating point value into smalltalk object. " ] I just uploaded this example and also how to write doubles from/to memory. See at SqS/NativeBoost - NativeBoost-Examples package, class NBBasicExamples.
I am very comfortable adding functions to a library for integer and float/double calculations on "large" arrays and calling them via FFI; plugins are another story. Â Are there examples to follow?
Tell me, what examples you would like to see. I will gladly provide them.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Wednesday, September 22, 2010 6:20 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] DoubleArray
On 23 September 2010 00:37, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Henry,
I have been getting away with #doubleAt: and #doubleAt:put:, and otherwise doing pretty much what you said, though I just use a byte array to hold the data. Â It seems to work; for FFI, I end up passing void pointers instead of double pointers, which I don't like doing.
Your point about using ordinary arrays makes sense to a point, though even w/o external interfacing, there is still a use for DoubleArray just as there is a use for String vs. an array of characters.
The main reason, why they are not here, that VM needs an additional set of primitives, which working with them.
<sellsman hat on> I can only say that with NativeBoost, you could be able to implement it quite easily in image and ship it in image, without a need to change VM or use external plugin. <sellsman hat off>
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Henrik Sperre Johansen [henrik.s.johansen@veloxit.no] Sent: Wednesday, September 22, 2010 5:23 PM To: pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] DoubleArray
 On 22.09.2010 23:09, Schwab,Wilhelm K wrote:
Squeak, deliberately, does not have a double array class. Â Is that because there is some horrible reason it can't work, or because it won't work on certain platforms? If the goal is to pass it as a parameter to an external function, one way would be to subclass WordArray, storing(and reading) each Float in two elements, and have the endianness decided by Smalltalk endianness. Probably want a startUp method for the class to swap instances endianness if the platform has changed as well. Put another way, should we add such a class, if only via an external heap or whatever would be needed to make it reliable?
Bill More likely because no one has actually had a need for one. It's definitely not _needed_ in base, where normal arrays of floats would do the same job just as well, if not better. IMHO, if someone makes one, it so should probably be part of a package related to external interfacing.
Cheers, Henry
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
Sig, The examples that make sense depend on what NativeBoost wants to be when it grows up. Is it a full competitor to FFI and/or Alien? If so, then it would be nice to see a rich set of calls to functions with different parameter types; ideally it would be in the form of something useful that gets added to Pharo using NativeBoost. That does a couple of things, providing both proof of concept and examples in one shot. With tongue firmly in cheek, looking at your indexing example, my stack might pop before the machine's :) However, it looks like it should settle down quickly enough, and things like that go on in the vm all the time; you are arguably just brining it into the image. Where would #readDoubleFrom: be in the class hiearchy? One of the reasons I asked about indexing an external array is that FFI is not good at such things. If you can provide ways to copy, fill, etc. blocks of memory and do it quickly, you gain ground, assuming you want to compete with FFI. Do callbacks, and you gain a lot of ground. Make calls on separate threads, and you gain a LOT of ground. That assumes that you can make it simple enough for a mildly above-average C programmer to understand how to use and ideally extend it. I will look at your example package on Squeak Source. One thing that drives me nuts with the Linux vm is that it looks for things where it wants to look, and gives no feedback about what it tried to do or even that it failed (except that handle is still zero). Your callout pragmas take a module name; is there any way to see inside of that or to provide a full path to the library? It is nice to be able to confirm loading of a library w/o having to call something, since there won't always be something "safe" (free of side effects) to call as a test. At the risk of seeming difficult, everything is academic until we have a Linux plugin or at least can build same. Alien has that problem. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Wednesday, September 22, 2010 10:05 PM To: Pharo-project@lists.gforge.inria.fr Subject: [Pharo-project] NativeBoost examples (was: Re: DoubleArray) On 23 September 2010 02:00, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
Fair enough :) The most recent thing I recall on NativeBoost is Stef asking for what I assume must be a Mac plugin; I thought Linux would be left out too, but it appears not, though it is not in my current vm. Beyond some list traffic, I found this:
The work for supporting Linux and Mac is in progress.
http://code.google.com/p/nativeboost/wiki/NativeBoost
Is there other documentation somewhere? Build instructions?
You can find all existing documentation, installation instructions on wiki pages at the above site. There is no other place for it. http://code.google.com/p/nativeboost/wiki/Installation - installation instructions.
You have a block copy, which is good. What about indexing, such as getting floats or doubles out of external memory?
This is just a piece of cake. ;) readDoubleFrom: address " This method loads the double from given external address. an address can be an instance of NBExternalAddress, or simple ByteArray with at least 8 bytes long, which holds a double floating value" <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'> "We are using a pseudo-function prototype and supplying own assembler code, instead of making a call to external function. In given example, an address argument , after coercion is pushed on stack. " ^ NBFFICallout cdecl: #( double ( void * address) ) " A pseudo-function takes 1 argument, and returns double value. Under cdecl call convention, all floating point return types is returned in fp(0) CPU register" emitCall: [:gen | | asm | asm := gen asm. "Here , we expecting that an address value is already pushed on stack" asm pop: asm EAX; "load an address value into EAX register by popping a stack" fld: (asm EAX ptr64). "load a floating point value from memory, at base address, held in EAX register into fp(0) register, we are using #ptr64, to indicate that memory operand size is 64bits long" " return value set, we are done. A code generator will take care for emitting code, which converts a double floating point value into smalltalk object. " ] I just uploaded this example and also how to write doubles from/to memory. See at SqS/NativeBoost - NativeBoost-Examples package, class NBBasicExamples.
I am very comfortable adding functions to a library for integer and float/double calculations on "large" arrays and calling them via FFI; plugins are another story. Are there examples to follow?
Tell me, what examples you would like to see. I will gladly provide them.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Wednesday, September 22, 2010 6:20 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] DoubleArray
On 23 September 2010 00:37, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Henry,
I have been getting away with #doubleAt: and #doubleAt:put:, and otherwise doing pretty much what you said, though I just use a byte array to hold the data. It seems to work; for FFI, I end up passing void pointers instead of double pointers, which I don't like doing.
Your point about using ordinary arrays makes sense to a point, though even w/o external interfacing, there is still a use for DoubleArray just as there is a use for String vs. an array of characters.
The main reason, why they are not here, that VM needs an additional set of primitives, which working with them.
<sellsman hat on> I can only say that with NativeBoost, you could be able to implement it quite easily in image and ship it in image, without a need to change VM or use external plugin. <sellsman hat off>
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Henrik Sperre Johansen [henrik.s.johansen@veloxit.no] Sent: Wednesday, September 22, 2010 5:23 PM To: pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] DoubleArray
On 22.09.2010 23:09, Schwab,Wilhelm K wrote:
Squeak, deliberately, does not have a double array class. Is that because there is some horrible reason it can't work, or because it won't work on certain platforms? If the goal is to pass it as a parameter to an external function, one way would be to subclass WordArray, storing(and reading) each Float in two elements, and have the endianness decided by Smalltalk endianness. Probably want a startUp method for the class to swap instances endianness if the platform has changed as well. Put another way, should we add such a class, if only via an external heap or whatever would be needed to make it reliable?
Bill More likely because no one has actually had a need for one. It's definitely not _needed_ in base, where normal arrays of floats would do the same job just as well, if not better. IMHO, if someone makes one, it so should probably be part of a package related to external interfacing.
Cheers, Henry
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 23 September 2010 06:11, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The examples that make sense depend on what NativeBoost wants to be when it grows up. Â Is it a full competitor to FFI and/or Alien? Â If so, then it would be nice to see a rich set of calls to functions with different parameter types; ideally it would be in the form of something useful that gets added to Pharo using NativeBoost. Â That does a couple of things, providing both proof of concept and examples in one shot.
One major 'proof of concept' kinda, is NBOpenGL package, which provides a full OpenGL bindings by using NativeBoost :) But sure thing, basic examples is a must. That's why i asked you , whant you wanna see. I have no idea, what can be userful to pharoers , since i can do anything and its completely trivial for me to implement, as an author of this project ;) So, without some feedback, i have no idea, if something in my code looks confusing, or hard to use and needs deeper explanation(s) and examples.
With tongue firmly in cheek, looking at your indexing example, my stack might pop before the machine's  :)  However, it looks like it should settle down quickly enough, and things like that go on in the vm all the time; you are arguably just brining it into the image.
Indeed. Arguably. You putting your code into an image, instead of extra plugin. So, once your code is debugged and works well, you are done, and don't have to worry with shipping a custom VM/plugin with your product. The development and deployment of new primitives is much faster if you using NB.
Where would #readDoubleFrom: be in the class hiearchy? Â One of the reasons I asked about indexing an external array is that FFI is not good at such things. Â If you can provide ways to copy, fill, etc. blocks of memory and do it quickly, you gain ground, assuming you want to compete with FFI.
In this particular example, you can do a usual way - make a DoubleArray class, which is a subclass of ByteArray , then implement #at: and #at:put: using nativeboost. Implement various stuff to fully cover Collection protocol, implement any extra stuff like blob operations (multiplication, addition etc) if you want. And you don't have to compete with FFI, since you can implement all primitives in NativeBoost without using any external functions. You can use any kind of optimizations in native code, which are not available in C, like using SSE/SIMD instructions etc. Even more, you can detect a CPU type, and depending on that, generate a most efficient code for your little class/library.
Do callbacks, and you gain a lot of ground. Â Make calls on separate threads, and you gain a LOT of ground. Â That assumes that you can make it simple enough for a mildly above-average C programmer to understand how to use and ideally extend it. Â I will look at your example package on Squeak Source.
Callbacks is there. See NBFFICallbackTests>>benchQSort for example.
One thing that drives me nuts with the Linux vm is that it looks for things where it wants to look, and gives no feedback about what it tried to do or even that it failed (except that handle is still zero). Â Your callout pragmas take a module name; is there any way to see inside of that or to provide a full path to the library? Â It is nice to be able to confirm loading of a library w/o having to call something, since there won't always be something "safe" (free of side effects) to call as a test.
Wait, there is no callout pragmas.. All stuff which starts from ^ NBFFICallout cdecl: #( double ( void * address) ) ... is smalltalk code, and you can debug it and see what it does, and how it loading a module and retrieving an external function pointer from it. You can customize this procedure in any way you want (by subclassing NBFFICallout), you can even customize the way how you making a call. For instance, in NBOpenGL, a generated code does not making a direct call to previously discovered function address. It using an sub-instance of NBOpenGLAPIBase, which is a variable-sized object, holding a function pointers, and it reads a corresponding pointer from this object, and only then making a call. So, there are plenty various tricks which can be done without much hassle. You just need to step away from rigid patterns, imposed by C compler, linker and all stuff around that, and you will see, that you can do miracles in smalltalk + native code :)
At the risk of seeming difficult, everything is academic until we have a Linux plugin or at least can build same. Â Alien has that problem.
I'm working on that.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Wednesday, September 22, 2010 10:05 PM To: Pharo-project@lists.gforge.inria.fr Subject: [Pharo-project] NativeBoost examples (was: Re: Â DoubleArray)
On 23 September 2010 02:00, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
Fair enough :) Â The most recent thing I recall on NativeBoost is Stef asking for what I assume must be a Mac plugin; I thought Linux would be left out too, but it appears not, though it is not in my current vm. Â Beyond some list traffic, I found this:
The work for supporting Linux and Mac is in progress.
 http://code.google.com/p/nativeboost/wiki/NativeBoost
Is there other documentation somewhere? Â Build instructions?
You can find all existing documentation, installation instructions on wiki pages at the above site. There is no other place for it. http://code.google.com/p/nativeboost/wiki/Installation - installation instructions.
You have a block copy, which is good. Â What about indexing, such as getting floats or doubles out of external memory?
This is just a piece of cake. Â ;)
readDoubleFrom: address     " This method loads the double from given external address.     an address can be an instance of NBExternalAddress, or     simple ByteArray with at least 8 bytes long, which holds a double floating value"
    <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'>
    "We are using a pseudo-function prototype and supplying own assembler code,     instead of making a call to external function.     In given example, an address argument , after coercion is pushed on stack. "
    ^ NBFFICallout
        cdecl: #( double ( void * address) )         " A pseudo-function takes 1 argument, and returns double value.          Under cdecl call convention, all floating point return types is returned in fp(0) CPU register"
        emitCall: [:gen |  | asm |             asm := gen asm.
            "Here , we expecting that an address value is already pushed on stack"
            asm pop: asm EAX;  "load an address value into EAX register by popping a stack"             fld: (asm EAX ptr64). "load a floating point value from memory, at base address, held in EAX register into fp(0) register,                 we are using #ptr64, to indicate that memory operand size is 64bits long"
            " return value set, we are done. A code generator will take care for emitting code, which converts             a double floating point value into smalltalk object. "         ]
I just uploaded this example and also how to write doubles from/to memory. See at SqS/NativeBoost - NativeBoost-Examples package, class NBBasicExamples.
I am very comfortable adding functions to a library for integer and float/double calculations on "large" arrays and calling them via FFI; plugins are another story. Â Are there examples to follow?
Tell me, what examples you would like to see. I will gladly provide them.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Wednesday, September 22, 2010 6:20 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] DoubleArray
On 23 September 2010 00:37, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Henry,
I have been getting away with #doubleAt: and #doubleAt:put:, and otherwise doing pretty much what you said, though I just use a byte array to hold the data. Â It seems to work; for FFI, I end up passing void pointers instead of double pointers, which I don't like doing.
Your point about using ordinary arrays makes sense to a point, though even w/o external interfacing, there is still a use for DoubleArray just as there is a use for String vs. an array of characters.
The main reason, why they are not here, that VM needs an additional set of primitives, which working with them.
<sellsman hat on> I can only say that with NativeBoost, you could be able to implement it quite easily in image and ship it in image, without a need to change VM or use external plugin. <sellsman hat off>
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Henrik Sperre Johansen [henrik.s.johansen@veloxit.no] Sent: Wednesday, September 22, 2010 5:23 PM To: pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] DoubleArray
 On 22.09.2010 23:09, Schwab,Wilhelm K wrote:
Squeak, deliberately, does not have a double array class. Â Is that because there is some horrible reason it can't work, or because it won't work on certain platforms? If the goal is to pass it as a parameter to an external function, one way would be to subclass WordArray, storing(and reading) each Float in two elements, and have the endianness decided by Smalltalk endianness. Probably want a startUp method for the class to swap instances endianness if the platform has changed as well. Put another way, should we add such a class, if only via an external heap or whatever would be needed to make it reliable?
Bill More likely because no one has actually had a need for one. It's definitely not _needed_ in base, where normal arrays of floats would do the same job just as well, if not better. IMHO, if someone makes one, it so should probably be part of a package related to external interfacing.
Cheers, Henry
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
Sig, I certainly don't want to stir up a war you might be trying to avoid, but it does strike me that you offer an alternative to FFI. That's ok, because Alien clearly does that, so we might as well have a third choice. Please make a lot of noise when you have a Linux plugin. I will give some thought to what might make a useful project/demo, but OpenGL should certainly have its points in both directions. If it does something cool, so much the better. Long term, it would be nice to have a retained mode 3D system. Croquet's TObject hierarchy seemed to be heading in the right direction, but they resisted the idea of putting a 3D world in a 2D application; I'm thinking of 3D modeling, games, etc. A way to create a window, add some objects, lights and a camera and rotate, move them would be wonderful to have. Alice is/was close, but it required a lot of turning off Wonderland (end user) features that I felt should have been added if desired rather than taken away. It also required the mathematically aware to think like layman, which was a problem. Prior to finding QCad, I designed a few electronics enclosures by putting "boxes on shelves" and flying around it with the help of Alice. There is a grant application I need to be working on (should be doing so right now, in fact, right Stef?<g>). That has me thinking about Fourier Transforms, graphs and statistics. I have a fairly pathetic DOUBLEArray that seems to work. Having a complete set of double, float, [s]dword, [s]word arrays would be nice for many uses. When 1.1.1 comes along, I need to redo some FFI code to benefit from the selector fixes. I might be able to stall that long enough to try NB, but it really just needs to have some things renamed. My PLplot interface could be redone or not, but so far I see no reason to bother. If it ends up needing callbacks, it would be a good candidate. A different interface really needs work because it screams for callbacks and would be a wonderful showpiece for NB. That's a big job, but a good goal. There is little to no time pressure (what I have works as far as it goes), but a big payoff if I can make it work with callbacks.
From a marketing perspective, if you don't yet have a cool 3D demo, that might be your best use of time. Either way, I need to slowly study what you have and begin to experiement toward the above mentiond rewrite once you have Linux locked down.
If you do have a cool 3D demo, why isn't it linked from your NativeBoost web site? :) Seriously, add a page of links to projects, or just one page that links to the SqueakSource pages of interest. The easier you make it for someone to see that you can do all of this stuff, the more likley you are to create converts. Have you looked at Ocean? It is being built with Alien. When you can do Windows, Linux and Mac, there might be an argument to switch. I don't control the project, but it seems reasonble to at least consider NB for it. I hope that's of some help. I look forward to trying it. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Thursday, September 23, 2010 9:07 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] NativeBoost examples (was: Re: DoubleArray) On 23 September 2010 06:11, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The examples that make sense depend on what NativeBoost wants to be when it grows up. Is it a full competitor to FFI and/or Alien? If so, then it would be nice to see a rich set of calls to functions with different parameter types; ideally it would be in the form of something useful that gets added to Pharo using NativeBoost. That does a couple of things, providing both proof of concept and examples in one shot.
One major 'proof of concept' kinda, is NBOpenGL package, which provides a full OpenGL bindings by using NativeBoost :) But sure thing, basic examples is a must. That's why i asked you , whant you wanna see. I have no idea, what can be userful to pharoers , since i can do anything and its completely trivial for me to implement, as an author of this project ;) So, without some feedback, i have no idea, if something in my code looks confusing, or hard to use and needs deeper explanation(s) and examples.
With tongue firmly in cheek, looking at your indexing example, my stack might pop before the machine's :) However, it looks like it should settle down quickly enough, and things like that go on in the vm all the time; you are arguably just brining it into the image.
Indeed. Arguably. You putting your code into an image, instead of extra plugin. So, once your code is debugged and works well, you are done, and don't have to worry with shipping a custom VM/plugin with your product. The development and deployment of new primitives is much faster if you using NB.
Where would #readDoubleFrom: be in the class hiearchy? One of the reasons I asked about indexing an external array is that FFI is not good at such things. If you can provide ways to copy, fill, etc. blocks of memory and do it quickly, you gain ground, assuming you want to compete with FFI.
In this particular example, you can do a usual way - make a DoubleArray class, which is a subclass of ByteArray , then implement #at: and #at:put: using nativeboost. Implement various stuff to fully cover Collection protocol, implement any extra stuff like blob operations (multiplication, addition etc) if you want. And you don't have to compete with FFI, since you can implement all primitives in NativeBoost without using any external functions. You can use any kind of optimizations in native code, which are not available in C, like using SSE/SIMD instructions etc. Even more, you can detect a CPU type, and depending on that, generate a most efficient code for your little class/library.
Do callbacks, and you gain a lot of ground. Make calls on separate threads, and you gain a LOT of ground. That assumes that you can make it simple enough for a mildly above-average C programmer to understand how to use and ideally extend it. I will look at your example package on Squeak Source.
Callbacks is there. See NBFFICallbackTests>>benchQSort for example.
One thing that drives me nuts with the Linux vm is that it looks for things where it wants to look, and gives no feedback about what it tried to do or even that it failed (except that handle is still zero). Your callout pragmas take a module name; is there any way to see inside of that or to provide a full path to the library? It is nice to be able to confirm loading of a library w/o having to call something, since there won't always be something "safe" (free of side effects) to call as a test.
Wait, there is no callout pragmas.. All stuff which starts from ^ NBFFICallout cdecl: #( double ( void * address) ) ... is smalltalk code, and you can debug it and see what it does, and how it loading a module and retrieving an external function pointer from it. You can customize this procedure in any way you want (by subclassing NBFFICallout), you can even customize the way how you making a call. For instance, in NBOpenGL, a generated code does not making a direct call to previously discovered function address. It using an sub-instance of NBOpenGLAPIBase, which is a variable-sized object, holding a function pointers, and it reads a corresponding pointer from this object, and only then making a call. So, there are plenty various tricks which can be done without much hassle. You just need to step away from rigid patterns, imposed by C compler, linker and all stuff around that, and you will see, that you can do miracles in smalltalk + native code :)
At the risk of seeming difficult, everything is academic until we have a Linux plugin or at least can build same. Alien has that problem.
I'm working on that.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Wednesday, September 22, 2010 10:05 PM To: Pharo-project@lists.gforge.inria.fr Subject: [Pharo-project] NativeBoost examples (was: Re: DoubleArray)
On 23 September 2010 02:00, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
Fair enough :) The most recent thing I recall on NativeBoost is Stef asking for what I assume must be a Mac plugin; I thought Linux would be left out too, but it appears not, though it is not in my current vm. Beyond some list traffic, I found this:
The work for supporting Linux and Mac is in progress.
http://code.google.com/p/nativeboost/wiki/NativeBoost
Is there other documentation somewhere? Build instructions?
You can find all existing documentation, installation instructions on wiki pages at the above site. There is no other place for it. http://code.google.com/p/nativeboost/wiki/Installation - installation instructions.
You have a block copy, which is good. What about indexing, such as getting floats or doubles out of external memory?
This is just a piece of cake. ;)
readDoubleFrom: address " This method loads the double from given external address. an address can be an instance of NBExternalAddress, or simple ByteArray with at least 8 bytes long, which holds a double floating value"
<primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'>
"We are using a pseudo-function prototype and supplying own assembler code, instead of making a call to external function. In given example, an address argument , after coercion is pushed on stack. "
^ NBFFICallout
cdecl: #( double ( void * address) ) " A pseudo-function takes 1 argument, and returns double value. Under cdecl call convention, all floating point return types is returned in fp(0) CPU register"
emitCall: [:gen | | asm | asm := gen asm.
"Here , we expecting that an address value is already pushed on stack"
asm pop: asm EAX; "load an address value into EAX register by popping a stack" fld: (asm EAX ptr64). "load a floating point value from memory, at base address, held in EAX register into fp(0) register, we are using #ptr64, to indicate that memory operand size is 64bits long"
" return value set, we are done. A code generator will take care for emitting code, which converts a double floating point value into smalltalk object. " ]
I just uploaded this example and also how to write doubles from/to memory. See at SqS/NativeBoost - NativeBoost-Examples package, class NBBasicExamples.
I am very comfortable adding functions to a library for integer and float/double calculations on "large" arrays and calling them via FFI; plugins are another story. Are there examples to follow?
Tell me, what examples you would like to see. I will gladly provide them.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Wednesday, September 22, 2010 6:20 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] DoubleArray
On 23 September 2010 00:37, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Henry,
I have been getting away with #doubleAt: and #doubleAt:put:, and otherwise doing pretty much what you said, though I just use a byte array to hold the data. It seems to work; for FFI, I end up passing void pointers instead of double pointers, which I don't like doing.
Your point about using ordinary arrays makes sense to a point, though even w/o external interfacing, there is still a use for DoubleArray just as there is a use for String vs. an array of characters.
The main reason, why they are not here, that VM needs an additional set of primitives, which working with them.
<sellsman hat on> I can only say that with NativeBoost, you could be able to implement it quite easily in image and ship it in image, without a need to change VM or use external plugin. <sellsman hat off>
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Henrik Sperre Johansen [henrik.s.johansen@veloxit.no] Sent: Wednesday, September 22, 2010 5:23 PM To: pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] DoubleArray
On 22.09.2010 23:09, Schwab,Wilhelm K wrote:
Squeak, deliberately, does not have a double array class. Is that because there is some horrible reason it can't work, or because it won't work on certain platforms? If the goal is to pass it as a parameter to an external function, one way would be to subclass WordArray, storing(and reading) each Float in two elements, and have the endianness decided by Smalltalk endianness. Probably want a startUp method for the class to swap instances endianness if the platform has changed as well. Put another way, should we add such a class, if only via an external heap or whatever would be needed to make it reliable?
Bill More likely because no one has actually had a need for one. It's definitely not _needed_ in base, where normal arrays of floats would do the same job just as well, if not better. IMHO, if someone makes one, it so should probably be part of a package related to external interfacing.
Cheers, Henry
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
participants (2)
-
Igor Stasenko -
Schwab,Wilhelm K