Accessing DLLs in Smalltalk/V
I found an old Smalltalk/V imagine from 2000 and tracked down how I was able add calls to Windows DLLs and to add my own. It was in the class DynamicLinkLibrary and its subclasses, e. g., a call to the GDIDLL to draw a Bezier spline (which I believe I added) was: bezierTo: hDC lpPoints: anLPPOINT count: anInt <api: PolyBezierTo ulong struct ulong boolean> ^self invalidArgument I added by own DLL to do matrix transformations that was written in ASSEMBLER to optimized the floating point processor pipelining (a start operation every cycle while it took 3 cycles to complete) and it performed about 15% better. I have not found anything similar in Pharo, but it must if external libraries are used. NativeBoost seems to come close, but I cannot find any documentation. Can someone point me to some documentation, code, anything? It is greatly appreciated. bw
The PharoForTheEnterprise book has a chapter on Nativeboost. You can find the book here in pdf format https://ci.inria.fr/pharo-contribution/job/PharoForTheEnterprise/lastSuccess... also you can click on the folder to download only the Nativeboost chapter, but I advice to download the whole book since it has tons of useful info. On Thu, May 29, 2014 at 5:15 PM, Bob Williams <rwilliams19@cox.net> wrote:
I found an old Smalltalk/V imagine from 2000 and tracked down how I was able add calls to Windows DLLs and to add my own. It was in the class DynamicLinkLibrary and its subclasses, e. g., a call to the GDIDLL to draw a Bezier spline (which I believe I added) was:
bezierTo: hDC lpPoints: anLPPOINT count: anInt
<api: PolyBezierTo ulong struct ulong boolean>
^self invalidArgument
I added by own DLL to do matrix transformations that was written in ASSEMBLER to optimized the floating point processor pipelining (a start operation every cycle while it took 3 cycles to complete) and it performed about 15% better.
I have not found anything similar in Pharo, but it must if external libraries are used. NativeBoost seems to come close, but I cannot find any documentation. Can someone point me to some documentation, code, anything? It is greatly appreciated.
bw
For nativeBoost have a look at the tutorial to use NB to bind X11 https://ci.inria.fr/pharo-contribution/job/PharoForTheEnterprise/lastSuccess... I'm sorry but if igor does not write documentation, it takes far too much energy from me to understand and write something. It would be so much better if people with the knowledge would write. Stef On 29/5/14 16:15, Bob Williams wrote:
I found an old Smalltalk/V imagine from 2000 and tracked down how I was able add calls to Windows DLLs and to add my own. It was in the class DynamicLinkLibrary and its subclasses, e. g., a call to the GDIDLL to draw a Bezier spline (which I believe I added) was:
bezierTo: hDC lpPoints: anLPPOINT count: anInt
<api: PolyBezierTo ulong struct ulong boolean>
^self invalidArgument
I added by own DLL to do matrix transformations that was written in ASSEMBLER to optimized the floating point processor pipelining (a start operation every cycle while it took 3 cycles to complete) and it performed about 15% better.
I have not found anything similar in Pharo, but it must if external libraries are used. NativeBoost seems to come close, but I cannot find any documentation. Can someone point me to some documentation, code, anything? It is greatly appreciated.
bw
Hello, To do what you want, there are 3 options: - using NativeBoost - using FFI from the VM - compiling your dll as part of the vm (VM plugin) NativeBoost is clearly the best option because it is the fastest, it is mostly implemented in the image so you can look at the code and it is by default in the image. FFI from the VM typically manages much better callbacks, so if you plan to extensively use callbacks you should look at it, by loading the package of Aliens (It does not rely any more on the old implementation of aliens but still have the same image side API and library). Alternatively you can improve NativeBoost so it manages better call back, this is "easy" as it is mostly implemented in the image. Compiling your dll as a VM plugin is usually not the best option, but it can be useful in some situation (uncommon OS or uncommon hardware for example). I explained how to compile and bind a dll with NativeBoost here (subsection Implementing the FFI calls) : http://clementbera.wordpress.com/2013/06/19/optimizing-pharo-to-c-speed-with... It is a very basic but very simple example. Additional examples can be found in the class NBBasicExamples in the package NativeBoost-Examples. There are also tests (For example NBFFICallbackTests>>#benchQSort) that you can look at, they work and show how to use NativeBoost. Else there is the chapter in Pharo for the entrerprise. Other people will give you a link. 2014-05-29 16:15 GMT+02:00 Bob Williams <rwilliams19@cox.net>:
I found an old Smalltalk/V imagine from 2000 and tracked down how I was able add calls to Windows DLLs and to add my own. It was in the class DynamicLinkLibrary and its subclasses, e. g., a call to the GDIDLL to draw a Bezier spline (which I believe I added) was:
bezierTo: hDC lpPoints: anLPPOINT count: anInt
<api: PolyBezierTo ulong struct ulong boolean>
^self invalidArgument
I added by own DLL to do matrix transformations that was written in ASSEMBLER to optimized the floating point processor pipelining (a start operation every cycle while it took 3 cycles to complete) and it performed about 15% better.
I have not found anything similar in Pharo, but it must if external libraries are used. NativeBoost seems to come close, but I cannot find any documentation. Can someone point me to some documentation, code, anything? It is greatly appreciated.
bw
Hello Clément, Am 29.05.2014 um 17:23 schrieb Clément Bera <bera.clement@gmail.com>:
NativeBoost is clearly the best option because it is the fastest, it is mostly implemented in the image so you can look at the code and it is by default in the image.
FFI from the VM typically manages much better callbacks, so if you plan to extensively use callbacks you should look at it, by loading the package of Aliens (It does not rely any more on the old implementation of aliens but still have the same image side API and library). Alternatively you can improve NativeBoost so it manages better call back, this is "easy" as it is mostly implemented in the image.
Could you explain why FFI is better at managing callbacks? I have used NativeBoost or implementing JNIPort, and callbacks were one of the easier parts of the work I had to do. As far as I know, both FFI and NativeBoost donât support callbacks from threads other than the thread in which the Smalltalk code is executed, but thatâs a limitation of both of them. As to improving NativeBoost: Thatâs not âeasyâ at all. Some parts of it are quite cryptic if you are not familiar with x86 assembler code, the documentation is still sparse, and I am under the impression that there are some unfinished parts. It is practically impossible to add something to NativeBoost if you are not Igor, and just want to use it for your own project as opposed to spending many hours trying to figure out how NativeBoost works. I was somewhat lucky that I found everything I needed in the archives of the pharo-dev and pharo-users mailing list., but i wouldnât call this documentation.
I explained how to compile and bind a dll with NativeBoost here (subsection Implementing the FFI calls) : http://clementbera.wordpress.com/2013/06/19/optimizing-pharo-to-c-speed-with... It is a very basic but very simple example.
Yes, this page helped, but as you wrote: basic and simple example - the more involved stuff remains undocumented. When working on JNIPort, I struggled with simple things like de-referencing a pointer to a pointer to a struct etc. - I didnât find documentation for this, and this is a basic feature in my opinion. Best regards, Joachim Geidel
Additional examples can be found in the class NBBasicExamples in the package NativeBoost-Examples. There are also tests (For example NBFFICallbackTests>>#benchQSort) that you can look at, they work and show how to use NativeBoost.
Else there is the chapter in Pharo for the entrerprise. Other people will give you a link.
Hi joachim could you spend some time and helping us to write a documentation on nativeboost? Else it will always stay the same. I cannot systematically try to write documentation on things I do not have expertise because it is killing me. For igor everything is obvious :) but we spent some afternoons improving the x11 tutorial. Stef On 30/5/14 08:50, Joachim Geidel wrote:
Hello Clément,
Am 29.05.2014 um 17:23 schrieb Clément Bera <bera.clement@gmail.com <mailto:bera.clement@gmail.com>>:
NativeBoost is clearly the best option because it is the fastest, it is mostly implemented in the image so you can look at the code and it is by default in the image.
FFI from the VM typically manages much better callbacks, so if you plan to extensively use callbacks you should look at it, by loading the package of Aliens (It does not rely any more on the old implementation of aliens but still have the same image side API and library). Alternatively you can improve NativeBoost so it manages better call back, this is "easy" as it is mostly implemented in the image.
Could you explain why FFI is better at managing callbacks? I have used NativeBoost or implementing JNIPort, and callbacks were one of the easier parts of the work I had to do. As far as I know, both FFI and NativeBoost donât support callbacks from threads other than the thread in which the Smalltalk code is executed, but thatâs a limitation of both of them.
As to improving NativeBoost: Thatâs not âeasyâ at all. Some parts of it are quite cryptic if you are not familiar with x86 assembler code, the documentation is still sparse, and I am under the impression that there are some unfinished parts. It is practically impossible to add something to NativeBoost if you are not Igor, and just want to use it for your own project as opposed to spending many hours trying to figure out how NativeBoost works. I was somewhat lucky that I found everything I needed in the archives of the pharo-dev and pharo-users mailing list., but i wouldnât call this documentation.
I explained how to compile and bind a dll with NativeBoost here (subsection Implementing the FFI calls) : http://clementbera.wordpress.com/2013/06/19/optimizing-pharo-to-c-speed-with... It is a very basic but very simple example.
Yes, this page helped, but as you wrote: basic and simple example - the more involved stuff remains undocumented. When working on JNIPort, I struggled with simple things like de-referencing a pointer to a pointer to a struct etc. - I didnât find documentation for this, and this is a basic feature in my opinion.
Best regards, Joachim Geidel
Additional examples can be found in the class NBBasicExamples in the package NativeBoost-Examples. There are also tests (For example NBFFICallbackTests>>#benchQSort) that you can look at, they work and show how to use NativeBoost.
Else there is the chapter in Pharo for the entrerprise. Other people will give you a link.
he has not provided documentation for Nativeboost that's true, but he never left a question unanswered either ;) I think I will come around eventually to adding documentation to Nativeboost. Imagine every Pharoer adding 10 lines of documentation every day. Say 100 people . That's 33 pages per day , 12167 pages per year. Thats 30 books of the size of PBE per year. An insane amount of documentation. On Fri, May 30, 2014 at 10:57 AM, stepharo <stepharo@free.fr> wrote:
Hi joachim
could you spend some time and helping us to write a documentation on nativeboost? Else it will always stay the same. I cannot systematically try to write documentation on things I do not have expertise because it is killing me.
For igor everything is obvious :) but we spent some afternoons improving the x11 tutorial.
Stef
On 30/5/14 08:50, Joachim Geidel wrote:
Hello Clément,
Am 29.05.2014 um 17:23 schrieb Clément Bera <bera.clement@gmail.com>:
NativeBoost is clearly the best option because it is the fastest, it is mostly implemented in the image so you can look at the code and it is by default in the image.
FFI from the VM typically manages much better callbacks, so if you plan to extensively use callbacks you should look at it, by loading the package of Aliens (It does not rely any more on the old implementation of aliens but still have the same image side API and library). Alternatively you can improve NativeBoost so it manages better call back, this is "easy" as it is mostly implemented in the image.
Could you explain why FFI is better at managing callbacks? I have used NativeBoost or implementing JNIPort, and callbacks were one of the easier parts of the work I had to do. As far as I know, both FFI and NativeBoost donât support callbacks from threads other than the thread in which the Smalltalk code is executed, but thatâs a limitation of both of them.
As to improving NativeBoost: Thatâs not âeasyâ at all. Some parts of it are quite cryptic if you are not familiar with x86 assembler code, the documentation is still sparse, and I am under the impression that there are some unfinished parts. It is practically impossible to add something to NativeBoost if you are not Igor, and just want to use it for your own project as opposed to spending many hours trying to figure out how NativeBoost works. I was somewhat lucky that I found everything I needed in the archives of the pharo-dev and pharo-users mailing list., but i wouldnât call this documentation.
I explained how to compile and bind a dll with NativeBoost here (subsection Implementing the FFI calls) :
http://clementbera.wordpress.com/2013/06/19/optimizing-pharo-to-c-speed-with... It is a very basic but very simple example.
Yes, this page helped, but as you wrote: basic and simple example - the more involved stuff remains undocumented. When working on JNIPort, I struggled with simple things like de-referencing a pointer to a pointer to a struct etc. - I didnât find documentation for this, and this is a basic feature in my opinion.
Best regards, Joachim Geidel
Additional examples can be found in the class NBBasicExamples in the package NativeBoost-Examples. There are also tests (For example NBFFICallbackTests>>#benchQSort) that you can look at, they work and show how to use NativeBoost.
Else there is the chapter in Pharo for the entrerprise. Other people will give you a link.
Hi all, Am 30.05.2014 um 10:30 schrieb kilon alios <kilon.alios@gmail.com>:
he has not provided documentation for Nativeboost that's true, but he never left a question unanswered either ;)
just to make sure that my message isnât misunderstood: I am not blaming Igor for not having written a book about NativeBoost (yet), and I am very thankful that he wrote NativeBoost in the first place. I know how hard it is to maintain a piece of software over an extended period of time. You have to work on a full-time job, earn money, have some remainders of a life etc. Itâs sometimes easier for academics as they may have some more freedom what they work on (but not necessarily so). If Chris Uppal had not written the excellent documentation of JNIPort for Dolphin before I took over maintenance, the JNIPort documentation would be in the same state as the NativeBoost docs. I simply donât have enough spare time for it.
I think I will come around eventually to adding documentation to Nativeboost.
Excellent, thanks in advance! I can help in proof reading, but donât expect too much - see above.
Imagine every Pharoer adding 10 lines of documentation every day. Say 100 people . That's 33 pages per day , 12167 pages per year. Thats 30 books of the size of PBE per year. An insane amount of documentation.
If it were so easy⦠To write NativeBoost docs, you have to understand what you are writing about in the first place, and you have to have a plan - chapter structure, consistent examples, ideally everything explained in the context of a typical problem. The current tutorial in already very good in this respect. But thatâs hard and takes time, and it canât be split up easily among more than three people - which accidentally seems to be the approximate number of knowledgeable NativeBoost experts on this planet who actually could actually do it.
On Fri, May 30, 2014 at 10:57 AM, stepharo <stepharo@free.fr> wrote: Hi joachim
could you spend some time and helping us to write a documentation on nativeboost? Else it will always stay the same. I cannot systematically try to write documentation on things I do not have expertise because it is killing me.
Same for me, I am just a NativeBoost amateur, and unfortunately my time budget is severely limited. I would prefer spending my time on enhancing JNIPort (callbacks are currently too complicated). But I can help in proofreading. Joachim
"just to make sure that my message isnât misunderstood: I am not blaming Igor for not having written a book about NativeBoost (yet), and I am very thankful that he wrote NativeBoost in the first place. I know how hard it is to maintain a piece of software over an extended period of time. You have to work on a full-time job, earn money, have some remainders of a life etc. Itâs sometimes easier for academics as they may have some more freedom what they work on (but not necessarily so). If Chris Uppal had not written the excellent documentation of JNIPort for Dolphin before I took over maintenance, the JNIPort documentation would be in the same state as the NativeBoost docs. I simply donât have enough spare time for it. " good but I was not replying to you, I was replying to Stephan. "Excellent, thanks in advance! I can help in proof reading, but donât expect too much - see above." I never expect anything from anyone, even when I complain about it ;). I am already porting PBE to Pharo 3 and doing video tutorials on Pharo 3 so my hands are full too. "If it were so easy⦠To write NativeBoost docs, you have to understand what you are writing about in the first place, and you have to have a plan - chapter structure, consistent examples, ideally everything explained in the context of a typical problem. The current tutorial in already very good in this respect. But thatâs hard and takes time, and it canât be split up easily among more than three people - which accidentally seems to be the approximate number of knowledgeable NativeBoost experts on this planet who actually could actually do it." yeah its not that easy, that's true, its easier, write a line a day, Your philosophy about well structured documentation is of course ideal but this comes down to Well Structure but little documentation vs loads but messy documentation. Which in that case I pick the second option with my eyes closed. Which you think most people will pick ? Plus most documentation is very messy anyway. I am a lawyer as professional and I deal with messy documentation from day to day basis, its not that I don't hate i, I do, but the alternative of no documentation is simply not an option to me. And if you think about it messy documentation still beats google search hands down. So never underestimate the immense value of messy documentation. The idea that Nativeboost itself is a tricky subject, well sure you need to understand how C works and be able to tolerate all the manual management mess, but I can't say beyond that is that hard. I suspect however that the problem is that not that many people use Nativeboost in the first place. On Fri, May 30, 2014 at 12:40 PM, Joachim Geidel < joachim.geidel@onlinehome.de> wrote:
Hi all,
Am 30.05.2014 um 10:30 schrieb kilon alios <kilon.alios@gmail.com>:
he has not provided documentation for Nativeboost that's true, but he never left a question unanswered either ;)
just to make sure that my message isnât misunderstood: I am not blaming Igor for not having written a book about NativeBoost (yet), and I am very thankful that he wrote NativeBoost in the first place. I know how hard it is to maintain a piece of software over an extended period of time. You have to work on a full-time job, earn money, have some remainders of a life etc. Itâs sometimes easier for academics as they may have some more freedom what they work on (but not necessarily so). If Chris Uppal had not written the excellent documentation of JNIPort for Dolphin before I took over maintenance, the JNIPort documentation would be in the same state as the NativeBoost docs. I simply donât have enough spare time for it.
I think I will come around eventually to adding documentation to Nativeboost.
Excellent, thanks in advance! I can help in proof reading, but donât expect too much - see above.
Imagine every Pharoer adding 10 lines of documentation every day. Say 100 people . That's 33 pages per day , 12167 pages per year. Thats 30 books of the size of PBE per year. An insane amount of documentation.
If it were so easy⦠To write NativeBoost docs, you have to understand what you are writing about in the first place, and you have to have a plan - chapter structure, consistent examples, ideally everything explained in the context of a typical problem. The current tutorial in already very good in this respect. But thatâs hard and takes time, and it canât be split up easily among more than three people - which accidentally seems to be the approximate number of knowledgeable NativeBoost experts on this planet who actually could actually do it.
On Fri, May 30, 2014 at 10:57 AM, stepharo <stepharo@free.fr> wrote:
Hi joachim
could you spend some time and helping us to write a documentation on nativeboost? Else it will always stay the same. I cannot systematically try to write documentation on things I do not have expertise because it is killing me.
Same for me, I am just a NativeBoost amateur, and unfortunately my time budget is severely limited. I would prefer spending my time on enhancing JNIPort (callbacks are currently too complicated). But I can help in proofreading.
Joachim
kilon alios wrote
An insane amount of documentation
<rant> "insane" may be the key term here. We could be inventing executable active essays, which is the kind of magic our live environment makes possible, instead of writing and rewriting paper docs that quickly go stale. Non-executable docs are a duplication smell and maintenance hassle. Text-only is one of the least-effective ways to communicate. I'd love to see more lively kernel style docs</rant> ----- Cheers, Sean -- View this message in context: http://forum.world.st/Accessing-DLLs-in-Smalltalk-V-tp4760856p4761006.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
great point , I think that Pillar already does this but its not exactly hassle free. I have to say I really like Pillar, very simple syntax, it generates pdf and hmtl ouput as well latex. Still will be an extra effort to bring it inside Pharo in a form of an editor. But frankly if you want to make your own documentation with Pillar is already very easy and the final result looks very good. On Fri, May 30, 2014 at 6:02 PM, Sean P. DeNigris <sean@clipperadams.com> wrote:
kilon alios wrote
An insane amount of documentation
<rant> "insane" may be the key term here. We could be inventing executable active essays, which is the kind of magic our live environment makes possible, instead of writing and rewriting paper docs that quickly go stale. Non-executable docs are a duplication smell and maintenance hassle. Text-only is one of the least-effective ways to communicate. I'd love to see more lively kernel style docs</rant>
----- Cheers, Sean -- View this message in context: http://forum.world.st/Accessing-DLLs-in-Smalltalk-V-tp4760856p4761006.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Hi, About active essays and writing documentation inside Pharo, I'm trying to bring some experience on writing using a tree/outliner like interface for it. Questions are in another thread, but I hope to help with some kind of "dynabooklets", for the moment using pandoc's markdown, wich seems better known and suited for academical writing, but may be at some point it can share some Pillar insights. Cheers, Offray On 05/30/2014 11:12 AM, kilon alios wrote:
great point , I think that Pillar already does this but its not exactly hassle free. I have to say I really like Pillar, very simple syntax, it generates pdf and hmtl ouput as well latex. Still will be an extra effort to bring it inside Pharo in a form of an editor. But frankly if you want to make your own documentation with Pillar is already very easy and the final result looks very good.
On Fri, May 30, 2014 at 6:02 PM, Sean P. DeNigris <sean@clipperadams.com <mailto:sean@clipperadams.com>> wrote:
kilon alios wrote > An insane amount of documentation
<rant> "insane" may be the key term here. We could be inventing executable active essays, which is the kind of magic our live environment makes possible, instead of writing and rewriting paper docs that quickly go stale. Non-executable docs are a duplication smell and maintenance hassle. Text-only is one of the least-effective ways to communicate. I'd love to see more lively kernel style docs</rant>
----- Cheers, Sean -- View this message in context: http://forum.world.st/Accessing-DLLs-in-Smalltalk-V-tp4760856p4761006.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
On 30.05.2014 08:50, Joachim Geidel wrote:
Yes, this page helped, but as you wrote: basic and simple example - the more involved stuff remains undocumented. When working on JNIPort, I struggled with simple things like de-referencing a pointer to a pointer to a struct etc. - I didnât find documentation for this, and this is a basic feature in my opinion. Looking through blog posts and the mailing list archive, I didn't find it so hard to work with (yet), but I am also just dealing with opaque types. One thing I'd wish to learn more about is proper resource management - the library I am using is constructing a tree of connected structs, and I haven't found a recipe how to deal with that (integrating with the pharo vm garbage collector).
<thread_hijack>Silly question: where can I find information about finalization et al in Pharo/ Squeak? My google searches didn't yield any results that where easy to understand.</thread_hijack>
participants (8)
-
Bob Williams -
Clément Bera -
Joachim Geidel -
kilon alios -
Markus Fritsche -
Offray Vladimir Luna Cárdenas -
Sean P. DeNigris -
stepharo