January 2012 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Small improvement for WriteStream >> nextPutAll:

by Eliot Miranda

On Sun, Jan 8, 2012 at 5:46 AM, Mariano Martinez Peck <marianopeck(a)gmail.com > wrote: > Hi guys. Some time ago Henry spot to us a small improvement for > WriteStream >> nextPutAll: that we are using in Fuel, but I think it can be > general. Henry comment was exactly: > > *Also: For variableBytes classes, if you rewrite: > > WriteStream > nextPutAll: aCollection > > | newEnd | > collection class instSpec == aCollection class instSpec ifFalse: > [^ super nextPutAll: aCollection ]. > > newEnd := position + aCollection size. > newEnd > writeLimit ifTrue: > [self growTo: newEnd + 10]. > > collection replaceFrom: position+1 to: newEnd with: aCollectionstartingAt: 1. > position := newEnd. > > ^ aCollection > > You can now pass all variableByte classes (Like ByteString) directly to a > stream with a ByteArray collection, and the replaceFrom:to:with:* > *startingAt: primitive will work correctly, just like the file primitive > does > This means you don't need special Serializers for these either, using > f.ex. clunky nextStringPutAll: methods with manual asByteArray conversions. > * > > > So...if you agree, I can commit the patch. > Surely this will break special encoded strings where the encode/decode is done in at:/at:put:. Moving the bytes isn't safe in general. However, if the programmer knows they themselves can use next:putAll:startingAt: or some such. I don't think your suggestion is safe in general for WriteStream. Instead if next:putAll:startingAt: doesn't do the job invent some new protocol that will. > Let me know. > > -- > Mariano > http://marianopeck.wordpress.com > > -- best, Eliot

Jan. 8, 2012

Re: [Pharo-project] Best way for FFI in Pharo

by Mariano Martinez Peck

On Sun, Jan 8, 2012 at 11:53 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>wrote: > I am just the messenger: the library names are different on the various > platforms, > Yes, but that's not a problem with the solution I have just mentioned. No need to change anything. > and the names can change over time (especially on Linux). > > yes, but that's why I meant this is not that frequent at all. How many times it change? once a year? And that's why I recommended having a small script to change it. So again, I don't see this like a critical problem of FFI. > > ------------------------------ > *From:* pharo-project-bounces(a)lists.gforge.inria.fr [ > pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Mariano > Martinez Peck [marianopeck(a)gmail.com] > *Sent:* Sunday, January 08, 2012 5:33 PM > > *To:* Pharo-project(a)lists.gforge.inria.fr > *Subject:* Re: [Pharo-project] Best way for FFI in Pharo > > > > On Sun, Jan 8, 2012 at 11:26 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>wrote: > >> Why do it (automated or not) 2000 times when it can be done once? >> > > I don't understand. Why should I need to change the module name so > frequently? I have been working with squeakDBX since 4 years and we never > needed to change it. > > >> Think of the wasted time and change log bloat. Would you like to do it >> every time you move from Windows to Linux and back? >> > > I don't understand why you say you should change that when moving from one > OS to another one. I think you may have design issue, not FFI. With DBXTalk > we have the superclass PharoOpenDBX, with has all the openDBX api with > methods ^ self subclassResponsability. That's to mark the API. Then we > have 2 subclasses, OpenDBXUnix and OpenDBXWindows. And then OpenDBXMac > which is subclass from the Unix one but doesn't change or add anything so > far. > Then we have a Current (singleton) stored in a class variable and a class > side initialize that does > > ffiImplementationForOS > | platformName | > platformName := Smalltalk os platformName. > platformName = 'Win32' ifTrue:[ ^OpenDBXWin32 basicNew initialize]. > platformName = 'unix' ifTrue:[ ^OpenDBXUnix basicNew initialize]. > platformName = 'Mac OS' ifTrue:[ ^OpenDBXMacOSX basicNew initialize]. > self error: 'Cannot identify platform'. > > And that's all. We never need to change anything. The only thing you > should care is if moving the same image from one OS to another one then you > have to reinitialize this class variable. Or...you could add it to startUp > list if you want. > > Cheers > > >> Say it once. The superiority of #moduleName is clear. >> >> >> >> ------------------------------ >> *From:* pharo-project-bounces(a)lists.gforge.inria.fr [ >> pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Mariano >> Martinez Peck [marianopeck(a)gmail.com] >> *Sent:* Sunday, January 08, 2012 4:29 PM >> >> *To:* Pharo-project(a)lists.gforge.inria.fr >> *Subject:* Re: [Pharo-project] Best way for FFI in Pharo >> >> >> >> On Sun, Jan 8, 2012 at 10:13 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>wrote: >> >>> Agreed, except for thinking that changing all of the module names is >>> not a big deal :) >>> >> >> Why not? how many smalltalk lines of code do you think it could take to >> automate that? ;) >> >> >> >>> An external library subclass is the way to go. I know this already >>> exists in FFI, but the evolving chapter on Spock does not mention it - it >>> should not only mention it, but recommend it, IMHO. >>> >>> Stef, does it make sense yet? >>> >>> >>> >>> >>> ------------------------------ >>> *From:* pharo-project-bounces(a)lists.gforge.inria.fr [ >>> pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Mariano >>> Martinez Peck [marianopeck(a)gmail.com] >>> *Sent:* Sunday, January 08, 2012 3:52 PM >>> >>> *To:* Pharo-project(a)lists.gforge.inria.fr >>> *Subject:* Re: [Pharo-project] Best way for FFI in Pharo >>> >>> >>> >>> On Sun, Jan 8, 2012 at 9:25 PM, StÃ©phane Ducasse < >>> stephane.ducasse(a)inria.fr> wrote: >>> >>>> can you give an example that I understand >>>> "One thing I strongly recommend is to favor #moduleName over pragmas >>>> listing the library - imagine changing 2000+ GSL calls if the library name >>>> changes =:0 " >>>> >>>> >>> Bill: that already exists in FFI. >>> Stef: what Bill says is that if you bind to a specific library you have >>> to put its library name in each method that calls a ffi function. Example >>> of DBX: >>> >>> apiErrorType: handle number: err >>> "int odbx_error_type( odbx_t*, int )" >>> <cdecl: long 'odbx_error_type' (ulong long) module: 'opendbx'> >>> ^ self externalCallFailed >>> >>> ---- >>> >>> apiInitialize: handle backend: backend host: host port: port >>> "long odbx_init(odbx_t**, char*, char*, char*)" >>> <cdecl: long 'odbx_init' (ulong* char* char* char*) module: 'opendbx >>> '> >>> ^self externalCallFailed >>> >>> --- >>> >>> xxx >>> >>> --- >>> >>> Notice the "module: 'opendbx" >>> So...if now the library is renamed or whatever, you have to change all >>> methods. But I don't think this is a real big deal. There are much worst >>> things. >>> >>> Finaly, I copy paste an answer from Andreas from a previous thread: >>> >>> >>> *The Right Way to do this is to have a subclass of ExternalLibrary and >>> implement the class-side method #moduleName along the lines of: >>> >>> MyLibrary class>>moduleName >>> "Answer the module name to use for this library" >>> Smalltalk platformName = 'Win32' ifTrue:[^'MyLibrary32.dll']. >>> Smalltalk platformName = 'unix' ifTrue:[ >>> "check various locations and versions" >>> #('/usr/lib/libMyLibrary.so' >>> '/usr/lib/libMyLibrary.so.1' >>> '/usr/lib/libMyLibrary.so.2' >>> '/usr/share/libMyLibrary.so.1' >>> '/usr/share/libMyLibrary.so.2'* >>> *) do:[:location| >>> (FileDirectory fileExists: location) ifTrue:[^location]. >>> ]. >>> ^self error:'MyLibrary is not installed' >>> ]. >>> * >>> * >>> * >>> >>>> Tx >>>> >>>> On Jan 8, 2012, at 9:18 PM, Schwab,Wilhelm K wrote: >>>> >>>> > Stef, >>>> > >>>> > Absent the NB experience, +1 to the comments below. FFI has pretty >>>> much "just worked" for me. We need double arrays. Callbacks would be >>>> great. One thing I strongly recommend is to favor #moduleName over pragmas >>>> listing the library - imagine changing 2000+ GSL calls if the library name >>>> changes =:0 It makes a LOT more sense to have a class per library, and >>>> that class knows the name to use. That's all the more true when one >>>> considers code such as ODBC that can run on multiple platforms with >>>> different names. #moduleName can test the OS and answer the correct name. >>>> > >>>> > Bill >>>> > >>>> > >>>> > ________________________________________ >>>> > From: pharo-project-bounces(a)lists.gforge.inria.fr [ >>>> pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of StÃ©phane >>>> Ducasse [stephane.ducasse(a)inria.fr] >>>> > Sent: Sunday, January 08, 2012 2:16 PM >>>> > To: Pharo-project(a)lists.gforge.inria.fr >>>> > Subject: Re: [Pharo-project] Best way for FFI in Pharo >>>> > >>>> > thanks for the feedback. >>>> > We will come back you soon :) >>>> > Because we should get FFI and NativeBoost fully working :). >>>> > >>>> > Stef >>>> > >>>> > On Jan 8, 2012, at 7:29 PM, ncalexan wrote: >>>> > >>>> >> >>>> >> fstephany wrote >>>> >>> >>>> >>> I'm also a bit lost between all the different options (plugins, >>>> >>> NativeBoost, FFI, Alien). >>>> >>> >>>> >> >>>> >> I also found this confusing, so let me tell my recent experience and >>>> my >>>> >> conclusion. >>>> >> >>>> >> I have written bindings to the SDL game programming library for use >>>> with >>>> >> Pharo. SDL is cross-platform, and I want to support Mac (most of >>>> all), >>>> >> Windows (one must), and Linux (if I must). As far as I can tell, >>>> Alien is >>>> >> not cross-platform and not maintained, so I have not spent time >>>> >> investigating it. >>>> >> >>>> >> Following Igor Stasenko's OpenGLSpecs package, I wrote an SDLSpecs >>>> package >>>> >> which parses the relevant C header files and then writes either an >>>> NB or FFI >>>> >> callout class tree. I need to define a few structures, make a class >>>> pool >>>> >> with some constants, and then define lots of callouts, many of which >>>> take >>>> >> references to structures (like 'foo(struct x*)'). >>>> >> >>>> >> I was able to make both work, more or less, on my Macbook Pro, using >>>> stock >>>> >> Pharo 1.3 images and Igor's NBCog.app. >>>> >> >>>> >> I found NB to be a nicer programmer experience but I found that the >>>> FFI just >>>> >> works. There are a lot of gratuitous differences, but: >>>> >> >>>> >> * NB requires a VM compiled with a separate plugin (that I built >>>> from source >>>> >> with no trouble -- Igor has done splendid work with CMakeVMMaker). >>>> >> * NB should be faster than the FFI at pretty much everything, but I >>>> can't >>>> >> say since I haven't measured. >>>> >> * NB has useful primitives for determining if your external objects >>>> are >>>> >> still valid and for determining the current operating system (I >>>> intend to >>>> >> use these even if I don't use NB). >>>> >> * NB has very few tests and examples and essentially no >>>> documentation. >>>> >> * NB has a simple conceptual model, but it is still not as simple as >>>> the FFI >>>> >> model. >>>> >> * NB uses variableByteClasses to great advantage -- this is very >>>> cool. >>>> >> >>>> >> * NB makes it easy to crash your image, since you are evaluating >>>> native code >>>> >> in image. >>>> >> * NB is still, to my eye, raw and untested on Mac. I had problems >>>> with >>>> >> stack alignments in bizarre places, and NB's interaction with garbage >>>> >> collection basically ended my attempts to use it. >>>> >> * NB is basically impossible to debug if you aren't very strong with >>>> x86 >>>> >> assembly (I'm not even strong with x86 assembly), and the stack >>>> >> manipulations rendered GDB pretty much useless for me. >>>> >> * NB does not integrate well with the Pharo tools -- I think the >>>> >> MethodTrailers are screwing up references, implementors, and source >>>> views, >>>> >> but I haven't investigated. >>>> >> * NB does not seem to support indirections very easily -- I >>>> struggled to >>>> >> understand how to interface to foreign C functions with specs like >>>> >> 'foo(struct x)', 'foo(struct x *)' and 'foo(struct x **)', and >>>> eventually >>>> >> gave up. >>>> >> >>>> >> I wanted to help Igor with the NB project, but the NB-OpenGL Mac >>>> bindings >>>> >> crash horribly for me, and although I was able to fix them, it I >>>> just don't >>>> >> hack enough x86 assembly to figure out all the tricks Igor is doing. >>>> The >>>> >> thought of making it all work on 3 platforms, and fix all the tool >>>> >> integration, was just too much for me. >>>> >> >>>> >> In conclusion, I prefer the FFI. It is old, cross platform, well >>>> tested, >>>> >> somewhat documented, and reliable. I think Igor has done some >>>> amazing work >>>> >> and I do not want to bash his project (and certainly not his code -- >>>> the >>>> >> man's a magician) but the fancy features are not necessary for my >>>> project. >>>> >> >>>> >> >>>> >> fstephany wrote >>>> >>> >>>> >>> Will this interface handle callbacks ? >>>> >>> >>>> >> >>>> >> I do not need callbacks so cannot speak to this issue. >>>> >> >>>> >> Yours, >>>> >> Nick Alexander >>>> >> >>>> >> -- >>>> >> View this message in context: >>>> http://forum.world.st/Best-way-for-FFI-in-Pharo-tp4275467p4276356.html >>>> >> Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >>>> >> >>>> > >>>> > >>>> > >>>> >>>> >>>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> >>> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

Jan. 8, 2012

Re: [Pharo-project] Best way for FFI in Pharo

by Schwab,Wilhelm K

I am just the messenger: the library names are different on the various platforms, and the names can change over time (especially on Linux). ________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Mariano Martinez Peck [marianopeck(a)gmail.com] Sent: Sunday, January 08, 2012 5:33 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] Best way for FFI in Pharo On Sun, Jan 8, 2012 at 11:26 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu<mailto:bschwab@anest.ufl.edu>> wrote: Why do it (automated or not) 2000 times when it can be done once? I don't understand. Why should I need to change the module name so frequently? I have been working with squeakDBX since 4 years and we never needed to change it. Think of the wasted time and change log bloat. Would you like to do it every time you move from Windows to Linux and back? I don't understand why you say you should change that when moving from one OS to another one. I think you may have design issue, not FFI. With DBXTalk we have the superclass PharoOpenDBX, with has all the openDBX api with methods ^ self subclassResponsability. That's to mark the API. Then we have 2 subclasses, OpenDBXUnix and OpenDBXWindows. And then OpenDBXMac which is subclass from the Unix one but doesn't change or add anything so far. Then we have a Current (singleton) stored in a class variable and a class side initialize that does ffiImplementationForOS | platformName | platformName := Smalltalk os platformName. platformName = 'Win32' ifTrue:[ ^OpenDBXWin32 basicNew initialize]. platformName = 'unix' ifTrue:[ ^OpenDBXUnix basicNew initialize]. platformName = 'Mac OS' ifTrue:[ ^OpenDBXMacOSX basicNew initialize]. self error: 'Cannot identify platform'. And that's all. We never need to change anything. The only thing you should care is if moving the same image from one OS to another one then you have to reinitialize this class variable. Or...you could add it to startUp list if you want. Cheers Say it once. The superiority of #moduleName is clear. ________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr> [pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr>] on behalf of Mariano Martinez Peck [marianopeck(a)gmail.com<mailto:marianopeck@gmail.com>] Sent: Sunday, January 08, 2012 4:29 PM To: Pharo-project(a)lists.gforge.inria.fr<mailto:Pharo-project@lists.gforge.inria.fr> Subject: Re: [Pharo-project] Best way for FFI in Pharo On Sun, Jan 8, 2012 at 10:13 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu<mailto:bschwab@anest.ufl.edu>> wrote: Agreed, except for thinking that changing all of the module names is not a big deal :) Why not? how many smalltalk lines of code do you think it could take to automate that? ;) An external library subclass is the way to go. I know this already exists in FFI, but the evolving chapter on Spock does not mention it - it should not only mention it, but recommend it, IMHO. Stef, does it make sense yet? ________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr> [pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr>] on behalf of Mariano Martinez Peck [marianopeck(a)gmail.com<mailto:marianopeck@gmail.com>] Sent: Sunday, January 08, 2012 3:52 PM To: Pharo-project(a)lists.gforge.inria.fr<mailto:Pharo-project@lists.gforge.inria.fr> Subject: Re: [Pharo-project] Best way for FFI in Pharo On Sun, Jan 8, 2012 at 9:25 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr<mailto:stephane.ducasse@inria.fr>> wrote: can you give an example that I understand "One thing I strongly recommend is to favor #moduleName over pragmas listing the library - imagine changing 2000+ GSL calls if the library name changes =:0 " Bill: that already exists in FFI. Stef: what Bill says is that if you bind to a specific library you have to put its library name in each method that calls a ffi function. Example of DBX: apiErrorType: handle number: err "int odbx_error_type( odbx_t*, int )" <cdecl: long 'odbx_error_type' (ulong long) module: 'opendbx'> ^ self externalCallFailed ---- apiInitialize: handle backend: backend host: host port: port "long odbx_init(odbx_t**, char*, char*, char*)" <cdecl: long 'odbx_init' (ulong* char* char* char*) module: 'opendbx'> ^self externalCallFailed --- xxx --- Notice the "module: 'opendbx" So...if now the library is renamed or whatever, you have to change all methods. But I don't think this is a real big deal. There are much worst things. Finaly, I copy paste an answer from Andreas from a previous thread: The Right Way to do this is to have a subclass of ExternalLibrary and implement the class-side method #moduleName along the lines of: MyLibrary class>>moduleName "Answer the module name to use for this library" Smalltalk platformName = 'Win32' ifTrue:[^'MyLibrary32.dll']. Smalltalk platformName = 'unix' ifTrue:[ "check various locations and versions" #('/usr/lib/libMyLibrary.so' '/usr/lib/libMyLibrary.so.1' '/usr/lib/libMyLibrary.so.2' '/usr/share/libMyLibrary.so.1' '/usr/share/libMyLibrary.so.2' ) do:[:location| (FileDirectory fileExists: location) ifTrue:[^location]. ]. ^self error:'MyLibrary is not installed' ]. Tx On Jan 8, 2012, at 9:18 PM, Schwab,Wilhelm K wrote: > Stef, > > Absent the NB experience, +1 to the comments below. FFI has pretty much "just worked" for me. We need double arrays. Callbacks would be great. One thing I strongly recommend is to favor #moduleName over pragmas listing the library - imagine changing 2000+ GSL calls if the library name changes =:0 It makes a LOT more sense to have a class per library, and that class knows the name to use. That's all the more true when one considers code such as ODBC that can run on multiple platforms with different names. #moduleName can test the OS and answer the correct name. > > Bill > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr> [pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr>] on behalf of StÃ©phane Ducasse [stephane.ducasse(a)inria.fr<mailto:stephane.ducasse@inria.fr>] > Sent: Sunday, January 08, 2012 2:16 PM > To: Pharo-project(a)lists.gforge.inria.fr<mailto:Pharo-project@lists.gforge.inria.fr> > Subject: Re: [Pharo-project] Best way for FFI in Pharo > > thanks for the feedback. > We will come back you soon :) > Because we should get FFI and NativeBoost fully working :). > > Stef > > On Jan 8, 2012, at 7:29 PM, ncalexan wrote: > >> >> fstephany wrote >>> >>> I'm also a bit lost between all the different options (plugins, >>> NativeBoost, FFI, Alien). >>> >> >> I also found this confusing, so let me tell my recent experience and my >> conclusion. >> >> I have written bindings to the SDL game programming library for use with >> Pharo. SDL is cross-platform, and I want to support Mac (most of all), >> Windows (one must), and Linux (if I must). As far as I can tell, Alien is >> not cross-platform and not maintained, so I have not spent time >> investigating it. >> >> Following Igor Stasenko's OpenGLSpecs package, I wrote an SDLSpecs package >> which parses the relevant C header files and then writes either an NB or FFI >> callout class tree. I need to define a few structures, make a class pool >> with some constants, and then define lots of callouts, many of which take >> references to structures (like 'foo(struct x*)'). >> >> I was able to make both work, more or less, on my Macbook Pro, using stock >> Pharo 1.3 images and Igor's NBCog.app. >> >> I found NB to be a nicer programmer experience but I found that the FFI just >> works. There are a lot of gratuitous differences, but: >> >> * NB requires a VM compiled with a separate plugin (that I built from source >> with no trouble -- Igor has done splendid work with CMakeVMMaker). >> * NB should be faster than the FFI at pretty much everything, but I can't >> say since I haven't measured. >> * NB has useful primitives for determining if your external objects are >> still valid and for determining the current operating system (I intend to >> use these even if I don't use NB). >> * NB has very few tests and examples and essentially no documentation. >> * NB has a simple conceptual model, but it is still not as simple as the FFI >> model. >> * NB uses variableByteClasses to great advantage -- this is very cool. >> >> * NB makes it easy to crash your image, since you are evaluating native code >> in image. >> * NB is still, to my eye, raw and untested on Mac. I had problems with >> stack alignments in bizarre places, and NB's interaction with garbage >> collection basically ended my attempts to use it. >> * NB is basically impossible to debug if you aren't very strong with x86 >> assembly (I'm not even strong with x86 assembly), and the stack >> manipulations rendered GDB pretty much useless for me. >> * NB does not integrate well with the Pharo tools -- I think the >> MethodTrailers are screwing up references, implementors, and source views, >> but I haven't investigated. >> * NB does not seem to support indirections very easily -- I struggled to >> understand how to interface to foreign C functions with specs like >> 'foo(struct x)', 'foo(struct x *)' and 'foo(struct x **)', and eventually >> gave up. >> >> I wanted to help Igor with the NB project, but the NB-OpenGL Mac bindings >> crash horribly for me, and although I was able to fix them, it I just don't >> hack enough x86 assembly to figure out all the tricks Igor is doing. The >> thought of making it all work on 3 platforms, and fix all the tool >> integration, was just too much for me. >> >> In conclusion, I prefer the FFI. It is old, cross platform, well tested, >> somewhat documented, and reliable. I think Igor has done some amazing work >> and I do not want to bash his project (and certainly not his code -- the >> man's a magician) but the fancy features are not necessary for my project. >> >> >> fstephany wrote >>> >>> Will this interface handle callbacks ? >>> >> >> I do not need callbacks so cannot speak to this issue. >> >> Yours, >> Nick Alexander >> >> -- >> View this message in context: http://forum.world.st/Best-way-for-FFI-in-Pharo-tp4275467p4276356.html >> Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >> > > > -- Mariano http://marianopeck.wordpress.com -- Mariano http://marianopeck.wordpress.com -- Mariano http://marianopeck.wordpress.com

Jan. 8, 2012

Re: [Pharo-project] Best way for FFI in Pharo

by Mariano Martinez Peck

On Sun, Jan 8, 2012 at 11:26 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>wrote: > Why do it (automated or not) 2000 times when it can be done once? > I don't understand. Why should I need to change the module name so frequently? I have been working with squeakDBX since 4 years and we never needed to change it. > Think of the wasted time and change log bloat. Would you like to do it > every time you move from Windows to Linux and back? > I don't understand why you say you should change that when moving from one OS to another one. I think you may have design issue, not FFI. With DBXTalk we have the superclass PharoOpenDBX, with has all the openDBX api with methods ^ self subclassResponsability. That's to mark the API. Then we have 2 subclasses, OpenDBXUnix and OpenDBXWindows. And then OpenDBXMac which is subclass from the Unix one but doesn't change or add anything so far. Then we have a Current (singleton) stored in a class variable and a class side initialize that does ffiImplementationForOS | platformName | platformName := Smalltalk os platformName. platformName = 'Win32' ifTrue:[ ^OpenDBXWin32 basicNew initialize]. platformName = 'unix' ifTrue:[ ^OpenDBXUnix basicNew initialize]. platformName = 'Mac OS' ifTrue:[ ^OpenDBXMacOSX basicNew initialize]. self error: 'Cannot identify platform'. And that's all. We never need to change anything. The only thing you should care is if moving the same image from one OS to another one then you have to reinitialize this class variable. Or...you could add it to startUp list if you want. Cheers > Say it once. The superiority of #moduleName is clear. > > > > ------------------------------ > *From:* pharo-project-bounces(a)lists.gforge.inria.fr [ > pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Mariano > Martinez Peck [marianopeck(a)gmail.com] > *Sent:* Sunday, January 08, 2012 4:29 PM > > *To:* Pharo-project(a)lists.gforge.inria.fr > *Subject:* Re: [Pharo-project] Best way for FFI in Pharo > > > > On Sun, Jan 8, 2012 at 10:13 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>wrote: > >> Agreed, except for thinking that changing all of the module names is >> not a big deal :) >> > > Why not? how many smalltalk lines of code do you think it could take to > automate that? ;) > > > >> An external library subclass is the way to go. I know this already >> exists in FFI, but the evolving chapter on Spock does not mention it - it >> should not only mention it, but recommend it, IMHO. >> >> Stef, does it make sense yet? >> >> >> >> >> ------------------------------ >> *From:* pharo-project-bounces(a)lists.gforge.inria.fr [ >> pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Mariano >> Martinez Peck [marianopeck(a)gmail.com] >> *Sent:* Sunday, January 08, 2012 3:52 PM >> >> *To:* Pharo-project(a)lists.gforge.inria.fr >> *Subject:* Re: [Pharo-project] Best way for FFI in Pharo >> >> >> >> On Sun, Jan 8, 2012 at 9:25 PM, StÃ©phane Ducasse < >> stephane.ducasse(a)inria.fr> wrote: >> >>> can you give an example that I understand >>> "One thing I strongly recommend is to favor #moduleName over pragmas >>> listing the library - imagine changing 2000+ GSL calls if the library name >>> changes =:0 " >>> >>> >> Bill: that already exists in FFI. >> Stef: what Bill says is that if you bind to a specific library you have >> to put its library name in each method that calls a ffi function. Example >> of DBX: >> >> apiErrorType: handle number: err >> "int odbx_error_type( odbx_t*, int )" >> <cdecl: long 'odbx_error_type' (ulong long) module: 'opendbx'> >> ^ self externalCallFailed >> >> ---- >> >> apiInitialize: handle backend: backend host: host port: port >> "long odbx_init(odbx_t**, char*, char*, char*)" >> <cdecl: long 'odbx_init' (ulong* char* char* char*) module: 'opendbx >> '> >> ^self externalCallFailed >> >> --- >> >> xxx >> >> --- >> >> Notice the "module: 'opendbx" >> So...if now the library is renamed or whatever, you have to change all >> methods. But I don't think this is a real big deal. There are much worst >> things. >> >> Finaly, I copy paste an answer from Andreas from a previous thread: >> >> >> *The Right Way to do this is to have a subclass of ExternalLibrary and >> implement the class-side method #moduleName along the lines of: >> >> MyLibrary class>>moduleName >> "Answer the module name to use for this library" >> Smalltalk platformName = 'Win32' ifTrue:[^'MyLibrary32.dll']. >> Smalltalk platformName = 'unix' ifTrue:[ >> "check various locations and versions" >> #('/usr/lib/libMyLibrary.so' >> '/usr/lib/libMyLibrary.so.1' >> '/usr/lib/libMyLibrary.so.2' >> '/usr/share/libMyLibrary.so.1' >> '/usr/share/libMyLibrary.so.2'* >> *) do:[:location| >> (FileDirectory fileExists: location) ifTrue:[^location]. >> ]. >> ^self error:'MyLibrary is not installed' >> ]. >> * >> * >> * >> >>> Tx >>> >>> On Jan 8, 2012, at 9:18 PM, Schwab,Wilhelm K wrote: >>> >>> > Stef, >>> > >>> > Absent the NB experience, +1 to the comments below. FFI has pretty >>> much "just worked" for me. We need double arrays. Callbacks would be >>> great. One thing I strongly recommend is to favor #moduleName over pragmas >>> listing the library - imagine changing 2000+ GSL calls if the library name >>> changes =:0 It makes a LOT more sense to have a class per library, and >>> that class knows the name to use. That's all the more true when one >>> considers code such as ODBC that can run on multiple platforms with >>> different names. #moduleName can test the OS and answer the correct name. >>> > >>> > Bill >>> > >>> > >>> > ________________________________________ >>> > From: pharo-project-bounces(a)lists.gforge.inria.fr [ >>> pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of StÃ©phane >>> Ducasse [stephane.ducasse(a)inria.fr] >>> > Sent: Sunday, January 08, 2012 2:16 PM >>> > To: Pharo-project(a)lists.gforge.inria.fr >>> > Subject: Re: [Pharo-project] Best way for FFI in Pharo >>> > >>> > thanks for the feedback. >>> > We will come back you soon :) >>> > Because we should get FFI and NativeBoost fully working :). >>> > >>> > Stef >>> > >>> > On Jan 8, 2012, at 7:29 PM, ncalexan wrote: >>> > >>> >> >>> >> fstephany wrote >>> >>> >>> >>> I'm also a bit lost between all the different options (plugins, >>> >>> NativeBoost, FFI, Alien). >>> >>> >>> >> >>> >> I also found this confusing, so let me tell my recent experience and >>> my >>> >> conclusion. >>> >> >>> >> I have written bindings to the SDL game programming library for use >>> with >>> >> Pharo. SDL is cross-platform, and I want to support Mac (most of >>> all), >>> >> Windows (one must), and Linux (if I must). As far as I can tell, >>> Alien is >>> >> not cross-platform and not maintained, so I have not spent time >>> >> investigating it. >>> >> >>> >> Following Igor Stasenko's OpenGLSpecs package, I wrote an SDLSpecs >>> package >>> >> which parses the relevant C header files and then writes either an NB >>> or FFI >>> >> callout class tree. I need to define a few structures, make a class >>> pool >>> >> with some constants, and then define lots of callouts, many of which >>> take >>> >> references to structures (like 'foo(struct x*)'). >>> >> >>> >> I was able to make both work, more or less, on my Macbook Pro, using >>> stock >>> >> Pharo 1.3 images and Igor's NBCog.app. >>> >> >>> >> I found NB to be a nicer programmer experience but I found that the >>> FFI just >>> >> works. There are a lot of gratuitous differences, but: >>> >> >>> >> * NB requires a VM compiled with a separate plugin (that I built from >>> source >>> >> with no trouble -- Igor has done splendid work with CMakeVMMaker). >>> >> * NB should be faster than the FFI at pretty much everything, but I >>> can't >>> >> say since I haven't measured. >>> >> * NB has useful primitives for determining if your external objects >>> are >>> >> still valid and for determining the current operating system (I >>> intend to >>> >> use these even if I don't use NB). >>> >> * NB has very few tests and examples and essentially no documentation. >>> >> * NB has a simple conceptual model, but it is still not as simple as >>> the FFI >>> >> model. >>> >> * NB uses variableByteClasses to great advantage -- this is very cool. >>> >> >>> >> * NB makes it easy to crash your image, since you are evaluating >>> native code >>> >> in image. >>> >> * NB is still, to my eye, raw and untested on Mac. I had problems >>> with >>> >> stack alignments in bizarre places, and NB's interaction with garbage >>> >> collection basically ended my attempts to use it. >>> >> * NB is basically impossible to debug if you aren't very strong with >>> x86 >>> >> assembly (I'm not even strong with x86 assembly), and the stack >>> >> manipulations rendered GDB pretty much useless for me. >>> >> * NB does not integrate well with the Pharo tools -- I think the >>> >> MethodTrailers are screwing up references, implementors, and source >>> views, >>> >> but I haven't investigated. >>> >> * NB does not seem to support indirections very easily -- I struggled >>> to >>> >> understand how to interface to foreign C functions with specs like >>> >> 'foo(struct x)', 'foo(struct x *)' and 'foo(struct x **)', and >>> eventually >>> >> gave up. >>> >> >>> >> I wanted to help Igor with the NB project, but the NB-OpenGL Mac >>> bindings >>> >> crash horribly for me, and although I was able to fix them, it I just >>> don't >>> >> hack enough x86 assembly to figure out all the tricks Igor is doing. >>> The >>> >> thought of making it all work on 3 platforms, and fix all the tool >>> >> integration, was just too much for me. >>> >> >>> >> In conclusion, I prefer the FFI. It is old, cross platform, well >>> tested, >>> >> somewhat documented, and reliable. I think Igor has done some >>> amazing work >>> >> and I do not want to bash his project (and certainly not his code -- >>> the >>> >> man's a magician) but the fancy features are not necessary for my >>> project. >>> >> >>> >> >>> >> fstephany wrote >>> >>> >>> >>> Will this interface handle callbacks ? >>> >>> >>> >> >>> >> I do not need callbacks so cannot speak to this issue. >>> >> >>> >> Yours, >>> >> Nick Alexander >>> >> >>> >> -- >>> >> View this message in context: >>> http://forum.world.st/Best-way-for-FFI-in-Pharo-tp4275467p4276356.html >>> >> Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >>> >> >>> > >>> > >>> > >>> >>> >>> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

Jan. 8, 2012

Re: [Pharo-project] Best way for FFI in Pharo

by Schwab,Wilhelm K

Why do it (automated or not) 2000 times when it can be done once? Think of the wasted time and change log bloat. Would you like to do it every time you move from Windows to Linux and back? Say it once. The superiority of #moduleName is clear. ________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Mariano Martinez Peck [marianopeck(a)gmail.com] Sent: Sunday, January 08, 2012 4:29 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] Best way for FFI in Pharo On Sun, Jan 8, 2012 at 10:13 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu<mailto:bschwab@anest.ufl.edu>> wrote: Agreed, except for thinking that changing all of the module names is not a big deal :) Why not? how many smalltalk lines of code do you think it could take to automate that? ;) An external library subclass is the way to go. I know this already exists in FFI, but the evolving chapter on Spock does not mention it - it should not only mention it, but recommend it, IMHO. Stef, does it make sense yet? ________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr> [pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr>] on behalf of Mariano Martinez Peck [marianopeck(a)gmail.com<mailto:marianopeck@gmail.com>] Sent: Sunday, January 08, 2012 3:52 PM To: Pharo-project(a)lists.gforge.inria.fr<mailto:Pharo-project@lists.gforge.inria.fr> Subject: Re: [Pharo-project] Best way for FFI in Pharo On Sun, Jan 8, 2012 at 9:25 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr<mailto:stephane.ducasse@inria.fr>> wrote: can you give an example that I understand "One thing I strongly recommend is to favor #moduleName over pragmas listing the library - imagine changing 2000+ GSL calls if the library name changes =:0 " Bill: that already exists in FFI. Stef: what Bill says is that if you bind to a specific library you have to put its library name in each method that calls a ffi function. Example of DBX: apiErrorType: handle number: err "int odbx_error_type( odbx_t*, int )" <cdecl: long 'odbx_error_type' (ulong long) module: 'opendbx'> ^ self externalCallFailed ---- apiInitialize: handle backend: backend host: host port: port "long odbx_init(odbx_t**, char*, char*, char*)" <cdecl: long 'odbx_init' (ulong* char* char* char*) module: 'opendbx'> ^self externalCallFailed --- xxx --- Notice the "module: 'opendbx" So...if now the library is renamed or whatever, you have to change all methods. But I don't think this is a real big deal. There are much worst things. Finaly, I copy paste an answer from Andreas from a previous thread: The Right Way to do this is to have a subclass of ExternalLibrary and implement the class-side method #moduleName along the lines of: MyLibrary class>>moduleName "Answer the module name to use for this library" Smalltalk platformName = 'Win32' ifTrue:[^'MyLibrary32.dll']. Smalltalk platformName = 'unix' ifTrue:[ "check various locations and versions" #('/usr/lib/libMyLibrary.so' '/usr/lib/libMyLibrary.so.1' '/usr/lib/libMyLibrary.so.2' '/usr/share/libMyLibrary.so.1' '/usr/share/libMyLibrary.so.2' ) do:[:location| (FileDirectory fileExists: location) ifTrue:[^location]. ]. ^self error:'MyLibrary is not installed' ]. Tx On Jan 8, 2012, at 9:18 PM, Schwab,Wilhelm K wrote: > Stef, > > Absent the NB experience, +1 to the comments below. FFI has pretty much "just worked" for me. We need double arrays. Callbacks would be great. One thing I strongly recommend is to favor #moduleName over pragmas listing the library - imagine changing 2000+ GSL calls if the library name changes =:0 It makes a LOT more sense to have a class per library, and that class knows the name to use. That's all the more true when one considers code such as ODBC that can run on multiple platforms with different names. #moduleName can test the OS and answer the correct name. > > Bill > > > ________________________________________ > From: pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr> [pharo-project-bounces(a)lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr>] on behalf of StÃ©phane Ducasse [stephane.ducasse(a)inria.fr<mailto:stephane.ducasse@inria.fr>] > Sent: Sunday, January 08, 2012 2:16 PM > To: Pharo-project(a)lists.gforge.inria.fr<mailto:Pharo-project@lists.gforge.inria.fr> > Subject: Re: [Pharo-project] Best way for FFI in Pharo > > thanks for the feedback. > We will come back you soon :) > Because we should get FFI and NativeBoost fully working :). > > Stef > > On Jan 8, 2012, at 7:29 PM, ncalexan wrote: > >> >> fstephany wrote >>> >>> I'm also a bit lost between all the different options (plugins, >>> NativeBoost, FFI, Alien). >>> >> >> I also found this confusing, so let me tell my recent experience and my >> conclusion. >> >> I have written bindings to the SDL game programming library for use with >> Pharo. SDL is cross-platform, and I want to support Mac (most of all), >> Windows (one must), and Linux (if I must). As far as I can tell, Alien is >> not cross-platform and not maintained, so I have not spent time >> investigating it. >> >> Following Igor Stasenko's OpenGLSpecs package, I wrote an SDLSpecs package >> which parses the relevant C header files and then writes either an NB or FFI >> callout class tree. I need to define a few structures, make a class pool >> with some constants, and then define lots of callouts, many of which take >> references to structures (like 'foo(struct x*)'). >> >> I was able to make both work, more or less, on my Macbook Pro, using stock >> Pharo 1.3 images and Igor's NBCog.app. >> >> I found NB to be a nicer programmer experience but I found that the FFI just >> works. There are a lot of gratuitous differences, but: >> >> * NB requires a VM compiled with a separate plugin (that I built from source >> with no trouble -- Igor has done splendid work with CMakeVMMaker). >> * NB should be faster than the FFI at pretty much everything, but I can't >> say since I haven't measured. >> * NB has useful primitives for determining if your external objects are >> still valid and for determining the current operating system (I intend to >> use these even if I don't use NB). >> * NB has very few tests and examples and essentially no documentation. >> * NB has a simple conceptual model, but it is still not as simple as the FFI >> model. >> * NB uses variableByteClasses to great advantage -- this is very cool. >> >> * NB makes it easy to crash your image, since you are evaluating native code >> in image. >> * NB is still, to my eye, raw and untested on Mac. I had problems with >> stack alignments in bizarre places, and NB's interaction with garbage >> collection basically ended my attempts to use it. >> * NB is basically impossible to debug if you aren't very strong with x86 >> assembly (I'm not even strong with x86 assembly), and the stack >> manipulations rendered GDB pretty much useless for me. >> * NB does not integrate well with the Pharo tools -- I think the >> MethodTrailers are screwing up references, implementors, and source views, >> but I haven't investigated. >> * NB does not seem to support indirections very easily -- I struggled to >> understand how to interface to foreign C functions with specs like >> 'foo(struct x)', 'foo(struct x *)' and 'foo(struct x **)', and eventually >> gave up. >> >> I wanted to help Igor with the NB project, but the NB-OpenGL Mac bindings >> crash horribly for me, and although I was able to fix them, it I just don't >> hack enough x86 assembly to figure out all the tricks Igor is doing. The >> thought of making it all work on 3 platforms, and fix all the tool >> integration, was just too much for me. >> >> In conclusion, I prefer the FFI. It is old, cross platform, well tested, >> somewhat documented, and reliable. I think Igor has done some amazing work >> and I do not want to bash his project (and certainly not his code -- the >> man's a magician) but the fancy features are not necessary for my project. >> >> >> fstephany wrote >>> >>> Will this interface handle callbacks ? >>> >> >> I do not need callbacks so cannot speak to this issue. >> >> Yours, >> Nick Alexander >> >> -- >> View this message in context: http://forum.world.st/Best-way-for-FFI-in-Pharo-tp4275467p4276356.html >> Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >> > > > -- Mariano http://marianopeck.wordpress.com -- Mariano http://marianopeck.wordpress.com

Jan. 8, 2012

Re: [Pharo-project] Process questions about configurationOfâ¦.

by Mariano Martinez Peck

On Sun, Jan 8, 2012 at 10:51 PM, Francois Stephany < tulipe.moutarde(a)gmail.com> wrote: > Isn't there a pragma or something to mark an expected failure ? > > Indeed :) thanks for the reminder. Well, forget about that item. Still the question of what happens if 80% of the tests are green. Ok or broken? > > On 08/01/12 13:20, Mariano Martinez Peck wrote: > >> >> 3) What happens with tests? do we only accept 100% green tests as ok? >> In Fuel for example, we like to create tests before the real code. And >> moreover, we create tests to demonstrate things we don't support yet. So >> sometimes we have red tests even if there is no bug, but just a missing >> thing we will do in the near future. >> maybe we can have a treshlod ? say, 80% of green tests? >> > > -- > http://tulipemoutarde.be > BE: +32 (0)65 709 131 > CA: +1 778 558 3225 > > -- Mariano http://marianopeck.wordpress.com

Jan. 8, 2012

Re: [Pharo-project] Process questions about configurationOfâ¦.

by Francois Stephany

Isn't there a pragma or something to mark an expected failure ? On 08/01/12 13:20, Mariano Martinez Peck wrote: > > 3) What happens with tests? do we only accept 100% green tests as ok? > In Fuel for example, we like to create tests before the real code. And > moreover, we create tests to demonstrate things we don't support yet. So > sometimes we have red tests even if there is no bug, but just a missing > thing we will do in the near future. > maybe we can have a treshlod ? say, 80% of green tests? -- http://tulipemoutarde.be BE: +32 (0)65 709 131 CA: +1 778 558 3225

Jan. 8, 2012

Re: [Pharo-project] Best way for FFI in Pharo

by Mariano Martinez Peck

On Sun, Jan 8, 2012 at 10:13 PM, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu>wrote: > Agreed, except for thinking that changing all of the module names is not > a big deal :) > Why not? how many smalltalk lines of code do you think it could take to automate that? ;) > An external library subclass is the way to go. I know this already > exists in FFI, but the evolving chapter on Spock does not mention it - it > should not only mention it, but recommend it, IMHO. > > Stef, does it make sense yet? > > > > > ------------------------------ > *From:* pharo-project-bounces(a)lists.gforge.inria.fr [ > pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of Mariano > Martinez Peck [marianopeck(a)gmail.com] > *Sent:* Sunday, January 08, 2012 3:52 PM > > *To:* Pharo-project(a)lists.gforge.inria.fr > *Subject:* Re: [Pharo-project] Best way for FFI in Pharo > > > > On Sun, Jan 8, 2012 at 9:25 PM, StÃ©phane Ducasse < > stephane.ducasse(a)inria.fr> wrote: > >> can you give an example that I understand >> "One thing I strongly recommend is to favor #moduleName over pragmas >> listing the library - imagine changing 2000+ GSL calls if the library name >> changes =:0 " >> >> > Bill: that already exists in FFI. > Stef: what Bill says is that if you bind to a specific library you have to > put its library name in each method that calls a ffi function. Example of > DBX: > > apiErrorType: handle number: err > "int odbx_error_type( odbx_t*, int )" > <cdecl: long 'odbx_error_type' (ulong long) module: 'opendbx'> > ^ self externalCallFailed > > ---- > > apiInitialize: handle backend: backend host: host port: port > "long odbx_init(odbx_t**, char*, char*, char*)" > <cdecl: long 'odbx_init' (ulong* char* char* char*) module: 'opendbx'> > ^self externalCallFailed > > --- > > xxx > > --- > > Notice the "module: 'opendbx" > So...if now the library is renamed or whatever, you have to change all > methods. But I don't think this is a real big deal. There are much worst > things. > > Finaly, I copy paste an answer from Andreas from a previous thread: > > > *The Right Way to do this is to have a subclass of ExternalLibrary and > implement the class-side method #moduleName along the lines of: > > MyLibrary class>>moduleName > "Answer the module name to use for this library" > Smalltalk platformName = 'Win32' ifTrue:[^'MyLibrary32.dll']. > Smalltalk platformName = 'unix' ifTrue:[ > "check various locations and versions" > #('/usr/lib/libMyLibrary.so' > '/usr/lib/libMyLibrary.so.1' > '/usr/lib/libMyLibrary.so.2' > '/usr/share/libMyLibrary.so.1' > '/usr/share/libMyLibrary.so.2'* > *) do:[:location| > (FileDirectory fileExists: location) ifTrue:[^location]. > ]. > ^self error:'MyLibrary is not installed' > ]. > * > * > * > >> Tx >> >> On Jan 8, 2012, at 9:18 PM, Schwab,Wilhelm K wrote: >> >> > Stef, >> > >> > Absent the NB experience, +1 to the comments below. FFI has pretty >> much "just worked" for me. We need double arrays. Callbacks would be >> great. One thing I strongly recommend is to favor #moduleName over pragmas >> listing the library - imagine changing 2000+ GSL calls if the library name >> changes =:0 It makes a LOT more sense to have a class per library, and >> that class knows the name to use. That's all the more true when one >> considers code such as ODBC that can run on multiple platforms with >> different names. #moduleName can test the OS and answer the correct name. >> > >> > Bill >> > >> > >> > ________________________________________ >> > From: pharo-project-bounces(a)lists.gforge.inria.fr [ >> pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of StÃ©phane >> Ducasse [stephane.ducasse(a)inria.fr] >> > Sent: Sunday, January 08, 2012 2:16 PM >> > To: Pharo-project(a)lists.gforge.inria.fr >> > Subject: Re: [Pharo-project] Best way for FFI in Pharo >> > >> > thanks for the feedback. >> > We will come back you soon :) >> > Because we should get FFI and NativeBoost fully working :). >> > >> > Stef >> > >> > On Jan 8, 2012, at 7:29 PM, ncalexan wrote: >> > >> >> >> >> fstephany wrote >> >>> >> >>> I'm also a bit lost between all the different options (plugins, >> >>> NativeBoost, FFI, Alien). >> >>> >> >> >> >> I also found this confusing, so let me tell my recent experience and my >> >> conclusion. >> >> >> >> I have written bindings to the SDL game programming library for use >> with >> >> Pharo. SDL is cross-platform, and I want to support Mac (most of all), >> >> Windows (one must), and Linux (if I must). As far as I can tell, >> Alien is >> >> not cross-platform and not maintained, so I have not spent time >> >> investigating it. >> >> >> >> Following Igor Stasenko's OpenGLSpecs package, I wrote an SDLSpecs >> package >> >> which parses the relevant C header files and then writes either an NB >> or FFI >> >> callout class tree. I need to define a few structures, make a class >> pool >> >> with some constants, and then define lots of callouts, many of which >> take >> >> references to structures (like 'foo(struct x*)'). >> >> >> >> I was able to make both work, more or less, on my Macbook Pro, using >> stock >> >> Pharo 1.3 images and Igor's NBCog.app. >> >> >> >> I found NB to be a nicer programmer experience but I found that the >> FFI just >> >> works. There are a lot of gratuitous differences, but: >> >> >> >> * NB requires a VM compiled with a separate plugin (that I built from >> source >> >> with no trouble -- Igor has done splendid work with CMakeVMMaker). >> >> * NB should be faster than the FFI at pretty much everything, but I >> can't >> >> say since I haven't measured. >> >> * NB has useful primitives for determining if your external objects are >> >> still valid and for determining the current operating system (I intend >> to >> >> use these even if I don't use NB). >> >> * NB has very few tests and examples and essentially no documentation. >> >> * NB has a simple conceptual model, but it is still not as simple as >> the FFI >> >> model. >> >> * NB uses variableByteClasses to great advantage -- this is very cool. >> >> >> >> * NB makes it easy to crash your image, since you are evaluating >> native code >> >> in image. >> >> * NB is still, to my eye, raw and untested on Mac. I had problems with >> >> stack alignments in bizarre places, and NB's interaction with garbage >> >> collection basically ended my attempts to use it. >> >> * NB is basically impossible to debug if you aren't very strong with >> x86 >> >> assembly (I'm not even strong with x86 assembly), and the stack >> >> manipulations rendered GDB pretty much useless for me. >> >> * NB does not integrate well with the Pharo tools -- I think the >> >> MethodTrailers are screwing up references, implementors, and source >> views, >> >> but I haven't investigated. >> >> * NB does not seem to support indirections very easily -- I struggled >> to >> >> understand how to interface to foreign C functions with specs like >> >> 'foo(struct x)', 'foo(struct x *)' and 'foo(struct x **)', and >> eventually >> >> gave up. >> >> >> >> I wanted to help Igor with the NB project, but the NB-OpenGL Mac >> bindings >> >> crash horribly for me, and although I was able to fix them, it I just >> don't >> >> hack enough x86 assembly to figure out all the tricks Igor is doing. >> The >> >> thought of making it all work on 3 platforms, and fix all the tool >> >> integration, was just too much for me. >> >> >> >> In conclusion, I prefer the FFI. It is old, cross platform, well >> tested, >> >> somewhat documented, and reliable. I think Igor has done some amazing >> work >> >> and I do not want to bash his project (and certainly not his code -- >> the >> >> man's a magician) but the fancy features are not necessary for my >> project. >> >> >> >> >> >> fstephany wrote >> >>> >> >>> Will this interface handle callbacks ? >> >>> >> >> >> >> I do not need callbacks so cannot speak to this issue. >> >> >> >> Yours, >> >> Nick Alexander >> >> >> >> -- >> >> View this message in context: >> http://forum.world.st/Best-way-for-FFI-in-Pharo-tp4275467p4276356.html >> >> Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >> >> >> > >> > >> > >> >> >> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

Jan. 8, 2012

Re: [Pharo-project] loading bytecodes into the image?

by Mariano Martinez Peck

On Sun, Jan 8, 2012 at 10:03 PM, Philippe Marschall <kustos(a)gmx.net> wrote: > Hi > > Ok, I'm not pretending this isn't a stupid idea, I'm just curios. > > Suppose I want to build something that would allow me to load "compiled > classes" (without sources) into the VM. How would you approach something > like this? What things need special care (bindings, symbols, load order, â¦)? > > We do that in Fuel. This is part of an external package called FuelMetalevel, based on Fuel. This package lets you serialize in binary way classes, traits, method dictionaries, compiled methods, etc. But such package just takes care about serialization. It doesn't notify, it doesn't validate, it doesn't add anything in Smalltalk globals, it doesn't rebind compiled methods, doesn't send class side #initialize, etc. In Marea ( http://rmod.lille.inria.fr/web/pier/software/Marea) I am able to swap out classes and traits to disk, let proxies in the image, and the swap them back if they are needed. For this I am using FuelMetalevel for example. Then we have another external package called FuelPackageLoader which is based in FuelMetalevel, which adds all the mentioned missing things. It adds semantics to the plain serialization of FuelMetalevel. It notify about class creation, it validates classes, rebinds compiled methods, sends class side #initialize, notifies, etc etc etc. As an example of this last one, we were able to export all seaside and materialize them in a different image. No sources. http://marianopeck.wordpress.com/2011/09/24/importing-and-exporting-package… > Cheers > Philippe > > > -- Mariano http://marianopeck.wordpress.com

Jan. 8, 2012

Re: [Pharo-project] Best way for FFI in Pharo

by StÃ©phane Ducasse

tx! Now I understand what you mean. On Jan 8, 2012, at 9:52 PM, Mariano Martinez Peck wrote: > > > On Sun, Jan 8, 2012 at 9:25 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > can you give an example that I understand > "One thing I strongly recommend is to favor #moduleName over pragmas listing the library - imagine changing 2000+ GSL calls if the library name changes =:0 " > > > Bill: that already exists in FFI. > Stef: what Bill says is that if you bind to a specific library you have to put its library name in each method that calls a ffi function. Example of DBX: > > apiErrorType: handle number: err > "int odbx_error_type( odbx_t*, int )" > <cdecl: long 'odbx_error_type' (ulong long) module: 'opendbx'> > ^ self externalCallFailed > > ---- > > apiInitialize: handle backend: backend host: host port: port > "long odbx_init(odbx_t**, char*, char*, char*)" > <cdecl: long 'odbx_init' (ulong* char* char* char*) module: 'opendbx'> > ^self externalCallFailed > > --- > > xxx > > --- > > Notice the "module: 'opendbx" > So...if now the library is renamed or whatever, you have to change all methods. But I don't think this is a real big deal. There are much worst things. > > Finaly, I copy paste an answer from Andreas from a previous thread: > > > The Right Way to do this is to have a subclass of ExternalLibrary and implement the class-side method #moduleName along the lines of: > > MyLibrary class>>moduleName > "Answer the module name to use for this library" > Smalltalk platformName = 'Win32' ifTrue:[^'MyLibrary32.dll']. > Smalltalk platformName = 'unix' ifTrue:[ > "check various locations and versions" > #('/usr/lib/libMyLibrary.so' > '/usr/lib/libMyLibrary.so.1' > '/usr/lib/libMyLibrary.so.2' > '/usr/share/libMyLibrary.so.1' > '/usr/share/libMyLibrary.so.2' > ) do:[:location| > (FileDirectory fileExists: location) ifTrue:[^location]. > ]. > ^self error:'MyLibrary is not installed' > ]. > > > Tx > > On Jan 8, 2012, at 9:18 PM, Schwab,Wilhelm K wrote: > > > Stef, > > > > Absent the NB experience, +1 to the comments below. FFI has pretty much "just worked" for me. We need double arrays. Callbacks would be great. One thing I strongly recommend is to favor #moduleName over pragmas listing the library - imagine changing 2000+ GSL calls if the library name changes =:0 It makes a LOT more sense to have a class per library, and that class knows the name to use. That's all the more true when one considers code such as ODBC that can run on multiple platforms with different names. #moduleName can test the OS and answer the correct name. > > > > Bill > > > > > > ________________________________________ > > From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] on behalf of StÃ©phane Ducasse [stephane.ducasse(a)inria.fr] > > Sent: Sunday, January 08, 2012 2:16 PM > > To: Pharo-project(a)lists.gforge.inria.fr > > Subject: Re: [Pharo-project] Best way for FFI in Pharo > > > > thanks for the feedback. > > We will come back you soon :) > > Because we should get FFI and NativeBoost fully working :). > > > > Stef > > > > On Jan 8, 2012, at 7:29 PM, ncalexan wrote: > > > >> > >> fstephany wrote > >>> > >>> I'm also a bit lost between all the different options (plugins, > >>> NativeBoost, FFI, Alien). > >>> > >> > >> I also found this confusing, so let me tell my recent experience and my > >> conclusion. > >> > >> I have written bindings to the SDL game programming library for use with > >> Pharo. SDL is cross-platform, and I want to support Mac (most of all), > >> Windows (one must), and Linux (if I must). As far as I can tell, Alien is > >> not cross-platform and not maintained, so I have not spent time > >> investigating it. > >> > >> Following Igor Stasenko's OpenGLSpecs package, I wrote an SDLSpecs package > >> which parses the relevant C header files and then writes either an NB or FFI > >> callout class tree. I need to define a few structures, make a class pool > >> with some constants, and then define lots of callouts, many of which take > >> references to structures (like 'foo(struct x*)'). > >> > >> I was able to make both work, more or less, on my Macbook Pro, using stock > >> Pharo 1.3 images and Igor's NBCog.app. > >> > >> I found NB to be a nicer programmer experience but I found that the FFI just > >> works. There are a lot of gratuitous differences, but: > >> > >> * NB requires a VM compiled with a separate plugin (that I built from source > >> with no trouble -- Igor has done splendid work with CMakeVMMaker). > >> * NB should be faster than the FFI at pretty much everything, but I can't > >> say since I haven't measured. > >> * NB has useful primitives for determining if your external objects are > >> still valid and for determining the current operating system (I intend to > >> use these even if I don't use NB). > >> * NB has very few tests and examples and essentially no documentation. > >> * NB has a simple conceptual model, but it is still not as simple as the FFI > >> model. > >> * NB uses variableByteClasses to great advantage -- this is very cool. > >> > >> * NB makes it easy to crash your image, since you are evaluating native code > >> in image. > >> * NB is still, to my eye, raw and untested on Mac. I had problems with > >> stack alignments in bizarre places, and NB's interaction with garbage > >> collection basically ended my attempts to use it. > >> * NB is basically impossible to debug if you aren't very strong with x86 > >> assembly (I'm not even strong with x86 assembly), and the stack > >> manipulations rendered GDB pretty much useless for me. > >> * NB does not integrate well with the Pharo tools -- I think the > >> MethodTrailers are screwing up references, implementors, and source views, > >> but I haven't investigated. > >> * NB does not seem to support indirections very easily -- I struggled to > >> understand how to interface to foreign C functions with specs like > >> 'foo(struct x)', 'foo(struct x *)' and 'foo(struct x **)', and eventually > >> gave up. > >> > >> I wanted to help Igor with the NB project, but the NB-OpenGL Mac bindings > >> crash horribly for me, and although I was able to fix them, it I just don't > >> hack enough x86 assembly to figure out all the tricks Igor is doing. The > >> thought of making it all work on 3 platforms, and fix all the tool > >> integration, was just too much for me. > >> > >> In conclusion, I prefer the FFI. It is old, cross platform, well tested, > >> somewhat documented, and reliable. I think Igor has done some amazing work > >> and I do not want to bash his project (and certainly not his code -- the > >> man's a magician) but the fancy features are not necessary for my project. > >> > >> > >> fstephany wrote > >>> > >>> Will this interface handle callbacks ? > >>> > >> > >> I do not need callbacks so cannot speak to this issue. > >> > >> Yours, > >> Nick Alexander > >> > >> -- > >> View this message in context: http://forum.world.st/Best-way-for-FFI-in-Pharo-tp4275467p4276356.html > >> Sent from the Pharo Smalltalk mailing list archive at Nabble.com. > >> > > > > > > > > > > > > -- > Mariano > http://marianopeck.wordpress.com >

Jan. 8, 2012