October 2012 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] NativeBoost FFI

by Igor Stasenko

On 22 October 2012 01:20, Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> wrote: > 5 parameters???? Igor, you're a dictator ;) > Those theories are nice, but unhelpfull when applied to FFI > Pragmatically there's not any chance I rewrite LAPACK+BLAS for the > sake of purity. > > And your workaround (creating a class) is very poor, because maybe > classes themselves should not have more than 5 instance variables ;) > Hehe. This is same thing like increasing number of literals for methods (and max distances between jumps). It just makes sense where you deal with external chaotic world. My ideology is simple: prevent that chaos from entering our little peaceful bay. Because first, you introduce it because you need it to deal with external world (only for 1% of cases, and in 0% of cases for good written smalltalk code). But that's only at beginning. Once people discover that they have no limits, i bet they will start abusing it in most horrible manner you ever seen. Copy-paste is worst enemy of programmer. But usually not to one who using it, but one who needs to maintain the code after :) > Nicolas > -- Best regards, Igor Stasenko.

Oct. 21, 2012

Re: [Pharo-project] Parsing Smalltalk Array/Object Literals

by Igor Stasenko

On 22 October 2012 00:23, GÃ¶ran Krampe <goran(a)krampe.se> wrote: > Hi! > > > On 10/21/2012 11:46 PM, Igor Stasenko wrote: >> >> To me the one of important part of this story is to have better code >> reuse. >> I like the idea of (re)using regular smalltalk parser to do >> heavylifting, while rest >> is built on top of it. > > > (I presume regarding external serialization format, anyway, you might note > that having "full Smalltalk" is a security issue also, and a speed issue) > > >> I like the idea with Deltas that you can rewind them or play them. >> Now the problem is that it will require substantial effort to adopt it to >> Pharo. >> Not long ago, we introduced SystemAnnouncer, and if you look at >> announces (SystemAnnouncement subclasses) you can see that it very >> closely resembling >> delta records. > > > I would suspect, yes. But still different since I presume a > SystemAnnouncement does NOT capture enough to enable undo, so they are still > "different beasts". Also, (without having looked) I guess a > SystemAnnouncement is not a "standalone object" like a Change in Deltas is. > I mean, it probably references the class, or a new method etc - and doesn't > actually contain a full copy of all state of the actual change. > right, but if we would do it right, i would make it like you said. Instead of having parallel class hierarchies which representing same event(s) in system. > >> I dont think that it will be clever to have two classes per each >> system event, i.e. >> ClassRemoved and then ClassRemovedDelta. It would be much nicer to >> have a single one. > > > See above. Different responsibilities, right? But I agree with the "itch", > not sure if it can be fully helped though. > Well, both Deltas and SystemAnnouncement reflecting the change in system. The only difference is that Delta has more capabilities than serving as a simple event. It would be nice if we could leave only single class hierachy for that (not two, parallel ones). Because same story again: once you have parallel stuff in two different places in system, it is prone to go out of sync sooner or later. > >> Another thing is what to do with SystemEditor? My main problem with >> it, that it is gone too far, >> trying to resemble many Class/Behavior protocols. > > > Personally I thought it looked "awful" - but I just wanted to be a user and > get "atomicity". IMHO one should be able to get atomicity using become > tricks instead, IIRC Henrik Gedenryd (bless his soul) did atomicity in his > old 3.3modules code using such a trick (building new classes and then at end > - do a "sweep become-replace" operation of all the modified classes) > right, but there's many pitfalls, like code which relying on new shape of a class, but not able to see it because it is still not installed. or opposite: the code which relies on old shape, but forced to 'speak' with new shape, where that part is removed/changed. > >> That means that any change/extension in Class/Behavior will have >> potential risk breaking >> SystemEditor, if we forget to sync changes there as well. >> And the last time we did pass on SystemEditor, if i remember, there >> was still no support for traits. >> >> But sure thing, i should not forget to say about what SystemEditor >> gives: atomic updates. >> My only concern is that price for having such feature is a bit too >> high as to me: >> - big and complex (okay, non-trivial ;) ) logic >> - a maintenance needed if we go and hack Class/Behavior/Traits etc). >> >> for the other features, i don't see significant advantage of using it >> over something else (less complex >> and more compact). > > > Final comment on SystemEditor - it is not hardwired into Deltas, in fact it > is kept separate and the idea was to be able to have different "Applier" > classes using different mechanisms. > Yes, personally i would like to see Deltas (modulo SystemEditor) integrated in pharo.. but the problem that who will make it happen :) > regards, GÃ¶ran -- Best regards, Igor Stasenko.

Oct. 21, 2012

Re: [Pharo-project] NativeBoost FFI

by Nicolas Cellier

5 parameters???? Igor, you're a dictator ;) Those theories are nice, but unhelpfull when applied to FFI Pragmatically there's not any chance I rewrite LAPACK+BLAS for the sake of purity. And your workaround (creating a class) is very poor, because maybe classes themselves should not have more than 5 instance variables ;) Nicolas 2012/10/22 Igor Stasenko <siguctua(a)gmail.com>: > On 21 October 2012 23:51, Nicolas Cellier > <nicolas.cellier.aka.nice(a)gmail.com> wrote: >> Hi Ciprian, >> >> I ran into same problem when I tried to interface Lapack with FFI a >> few years ago. >> So I used this simple idea to let the Compiler automatically pass an >> array of arguments, with two variants >> - hacked the compiler >> - subclassed the compiler > > Just as a side note, today i think that 15 args is too much. > I would even put less, say 5. > To my thinking, methods which taking more than 5 arguments need to be > refactored to /dev/null, > and developers who think that it is good idea to make so much > arguments be sent to correction facility :) > > >> >> See http://code.google.com/p/smallapack/wiki/SmallapackPortingOnSqueak >> http://bugs.squeak.org/view.php?id=2918 >> http://www.squeaksource.com/MetacelloRepository/ConfigurationOfSmallapack-n… >> http://www.squeaksource.com/Smallapack.html >> http://www.squeaksource.com/Smallapack/Smallapack-Compiler.trunk-nice.6.mcz >> >> Nicolas >> > > -- > Best regards, > Igor Stasenko. >

Oct. 21, 2012

Re: [Pharo-project] Parsing Smalltalk Array/Object Literals

by Dale Henrichs

Thanks Sven, I tried to look at the code, but couldn't download anything...I'm not currently shopping for another human-readable serialization format, but when I'm forced to change, I'll keep this in mind... Thanks, Dale ----- Original Message ----- | From: "Sven Van Caekenberghe" <sven(a)stfx.eu> | To: Pharo-project(a)lists.gforge.inria.fr | Sent: Sunday, October 21, 2012 2:38:17 PM | Subject: Re: [Pharo-project] Parsing Smalltalk Array/Object Literals | | Dale, | | Please, please look at the code. | | Again: no my code does _not_ depend on a Smalltalk compiler. | | And indeed, there are 2 steps: | | 1. String -> Array Literal | 2. Object Literal -> Object | | All in all, its very little code. | | Sven | | On 21 Oct 2012, at 20:23, Dale Henrichs <dhenrich(a)vmware.com> wrote: | | > Sven, | > | > I'm not asking about the parser for the literal array but the | > parser to parse the literal array itself, but I think you've | > answered my question (indirectly:) | > | > The original reason for going with JSON 9 months ago was that since | > GemStone does not have a accessible parser for Smalltalk syntax, I | > need to choose an export format for "objects" that included a | > portable parser for the syntax ... | > | > Presumably your implementation assumes the existence of a Smalltalk | > literal parser (I think that is what I was able to glean from | > Igor's mails) in which case I won't be able to use it (even if I | > thought it was usable from a readability standpoint:). | > | > Dale | | -- | Sven Van Caekenberghe | http://stfx.eu | Smalltalk is the Red Pill | | | | |

Oct. 21, 2012

Re: [Pharo-project] NativeBoost FFI

by Igor Stasenko

On 22 October 2012 00:31, Igor Stasenko <siguctua(a)gmail.com> wrote: > Hi, Ciprian. > this is not going to work unless you specify the class to use (which > contains such instance variables), > which will make signatures even more uglier, something like: > > function: aPoint > <primitive: #primitiveNativeCall module: #NativeBoostPlugin> > ^ self call: #( int function (int aPoint@Point:x, int aPoint@Point:y)) > > and that amount of mess beyond acceptance levels, to my taste :) > and even more than that, if you change the shape of a class (like remove instance variable or change their order), this method won't recompile automatically, means that next time you attempt to use it, you will crash the system, or get weird behavior. While if method sits in same class and accessing its instance variables, it is completely safe, because when you changing shape of a class, it will automatically recompile all its methods and therefore wipe generated code out. -- Best regards, Igor Stasenko.

Oct. 21, 2012

Re: [Pharo-project] Parsing Smalltalk Array/Object Literals

by Dale Henrichs

GÃ¶ran, I did not know that you had a Smalltalk literal array parser as part of Tirade. I wil keep that in mind when the JSON issues bubbles up again ... Thanks, Dale ----- Original Message ----- | From: "GÃ¶ran Krampe" <goran(a)krampe.se> | To: Pharo-project(a)lists.gforge.inria.fr | Sent: Sunday, October 21, 2012 1:25:03 PM | Subject: Re: [Pharo-project] Parsing Smalltalk Array/Object Literals | | On 10/21/2012 08:23 PM, Dale Henrichs wrote: | > Sven, | > | > I'm not asking about the parser for the literal array but the | > parser to parse the literal array itself, but I think you've | > answered my question (indirectly:) | > | > The original reason for going with JSON 9 months ago was that since | > GemStone does not have a accessible parser for Smalltalk syntax, I | > need to choose an export format for "objects" that included a | > portable parser for the syntax ... | > | > Presumably your implementation assumes the existence of a Smalltalk | > literal parser (I think that is what I was able to glean from | > Igor's mails) in which case I won't be able to use it (even if I | > thought it was usable from a readability standpoint:). | > | > Dale | | I am sounding like a broken record - but Tirade includes a hand coded | object literal parser which is fast. All of Tirade is 500 lines. But | yes, it does reuse (IIRC) the parser for literal numbers, so you | would | need to implement/port that one... ;) | | regards, GÃ¶ran | | |

Oct. 21, 2012

Re: [Pharo-project] NativeBoost FFI

by Igor Stasenko

On 21 October 2012 23:51, Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> wrote: > Hi Ciprian, > > I ran into same problem when I tried to interface Lapack with FFI a > few years ago. > So I used this simple idea to let the Compiler automatically pass an > array of arguments, with two variants > - hacked the compiler > - subclassed the compiler Just as a side note, today i think that 15 args is too much. I would even put less, say 5. To my thinking, methods which taking more than 5 arguments need to be refactored to /dev/null, and developers who think that it is good idea to make so much arguments be sent to correction facility :) > > See http://code.google.com/p/smallapack/wiki/SmallapackPortingOnSqueak > http://bugs.squeak.org/view.php?id=2918 > http://www.squeaksource.com/MetacelloRepository/ConfigurationOfSmallapack-n… > http://www.squeaksource.com/Smallapack.html > http://www.squeaksource.com/Smallapack/Smallapack-Compiler.trunk-nice.6.mcz > > Nicolas > -- Best regards, Igor Stasenko.

Oct. 21, 2012

Re: [Pharo-project] NativeBoost FFI

by Igor Stasenko

Hi, Ciprian. Yes, the number of arguments limit is a problem (which occurs 1% of the time), but if it occurs, it beating you hard :) But NativeBoost allows you to use instance variable names in function signature, (the max number of inst vars is 255, which is more than enough i think ;) ), and what is most important (as you noted) it doesn't breaking encapsulation, since methods having full right for accessing receiver's instance variables in their class scope. So, for making a binding to function which takes too many arguments, what you can do is to create a class: Object subclass: #MyBigFunctionBinding instanceVariableNames: 'var1 var2 .... var100' .. And then implement a method for FFI callout in it: MyBigFunctionBinding>>callThatFatThing <primitive: > ^ self call: #( int function (int var1, int var2 , ....... var100 )) Another approach is to create an NBExternalStructure which will have same memory layout as arguments of function (so if you pass it by value, it will push exactly same number of values in right order on stack): NBExternalStructure subclass: #BloatedArgs .. BloatedFnArgs class>>fieldsDesc #( int arg1; int arg2; . ... int arg200; ) then you can just do: MyBigFunctionBinding>>callThatFatThing: args <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (BloatedFnArgs args )) So, as you can see from above, you can deal with this problem one way or another. And given that it is just 1% of all C functions (or even less), personally, i don't think that it is worth extending a signature syntax. On 21 October 2012 21:36, Ciprian Teodorov <ciprian.teodorov(a)gmail.com> wrote: > Hi guys, > > Over the weekend I had a look at NativeBoost FFI and I have enjoyed it. I > really like this idea of taming the beast from high-level languages. > However I ran into some problems with the number of arguments one can pass > to a smaltalk method (it turns out to be 15 at max)... That bad for FFI > especially when you want to interface with legacy c code without touching > the C-stuff (header, object files, etc). > > In consequence I've hacked my way around this limitation by introduction of > a new type of arguments that we can pass to callouts. I call these arguments > "in object arguments". > The idea isn't new and is very simple, instead of passing 20 arguments to > your function just pass one an array with all the arguments you need. > Where it turns interesting is that I think we can pass more than just an > array we can pass real objects (think about ParameterObject) that hold in > their instance variables the arguments we want. > > lets say that we want to call a function " int function(int x, int y)", now > with the NativeBoost wrapper we will get a call something like > > function_x: x y: y > <primitive: #primitiveNativeCall module: #NativeBoostPlugin> > ^ self call: #( int function (int x, int y)) > > the idea is to replace x: y: with only one argument > > function: anArgument > <primitive: #primitiveNativeCall module: #NativeBoostPlugin> > ^ self call: #( int function (int anArgument@1, int anArgument@2)) > > or even > > function: aPoint > <primitive: #primitiveNativeCall module: #NativeBoostPlugin> > ^ self call: #( int function (int aPoint@x, int aPoint@y)) > > Yes this last example breaks the encapsulation a little bit by acceding > directly to the instance variable of aPoint... but... well > this is not going to work unless you specify the class to use (which contains such instance variables), which will make signatures even more uglier, something like: function: aPoint <primitive: #primitiveNativeCall module: #NativeBoostPlugin> ^ self call: #( int function (int aPoint@Point:x, int aPoint@Point:y)) and that amount of mess beyond acceptance levels, to my taste :) > another example will be: > > function_arg1: a arg2: b > <primitive: #primitiveNativeCall module: #NativeBoostPlugin> > ^ self call: #( int function (int a@1, int a@2, int b@3, int b@1, int a@6)) > > > Well that's all that I did this weekend. > > Attached to this mail you can find a file-out of these developments... they > include: > 1. the idea presented above > 2. refactoring of NBFFICallout argument loader creation > 3. a small "extension/fix" of the skipSpace method (in spec parser) to skip > all separators not just spaces (think about a header definition with tabs) > Thanks , i will look at it. > Well that's it folks, enjoy > Cheers > -- > Dr. Ciprian TEODOROV > IngÃ©nieur DÃ©veloppement CAO > > tÃ©l : 06 08 54 73 48 > mail : ciprian.teodorov(a)gmail.com > www.teodorov.ro -- Best regards, Igor Stasenko.

Oct. 21, 2012

Re: [Pharo-project] Parsing Smalltalk Array/Object Literals

by Geert Claes

Thought I'd share this here: http://jsoneditoronline.org -- View this message in context: http://forum.world.st/Parsing-Smalltalk-Array-Object-Literals-tp4652173p465… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Oct. 21, 2012

Re: [Pharo-project] Parsing Smalltalk Array/Object Literals

by GÃ¶ran Krampe

Hi! On 10/21/2012 11:46 PM, Igor Stasenko wrote: > To me the one of important part of this story is to have better code reuse. > I like the idea of (re)using regular smalltalk parser to do > heavylifting, while rest > is built on top of it. (I presume regarding external serialization format, anyway, you might note that having "full Smalltalk" is a security issue also, and a speed issue) > I like the idea with Deltas that you can rewind them or play them. > Now the problem is that it will require substantial effort to adopt it to Pharo. > Not long ago, we introduced SystemAnnouncer, and if you look at > announces (SystemAnnouncement subclasses) you can see that it very > closely resembling > delta records. I would suspect, yes. But still different since I presume a SystemAnnouncement does NOT capture enough to enable undo, so they are still "different beasts". Also, (without having looked) I guess a SystemAnnouncement is not a "standalone object" like a Change in Deltas is. I mean, it probably references the class, or a new method etc - and doesn't actually contain a full copy of all state of the actual change. > I dont think that it will be clever to have two classes per each > system event, i.e. > ClassRemoved and then ClassRemovedDelta. It would be much nicer to > have a single one. See above. Different responsibilities, right? But I agree with the "itch", not sure if it can be fully helped though. > Another thing is what to do with SystemEditor? My main problem with > it, that it is gone too far, > trying to resemble many Class/Behavior protocols. Personally I thought it looked "awful" - but I just wanted to be a user and get "atomicity". IMHO one should be able to get atomicity using become tricks instead, IIRC Henrik Gedenryd (bless his soul) did atomicity in his old 3.3modules code using such a trick (building new classes and then at end - do a "sweep become-replace" operation of all the modified classes) > That means that any change/extension in Class/Behavior will have > potential risk breaking > SystemEditor, if we forget to sync changes there as well. > And the last time we did pass on SystemEditor, if i remember, there > was still no support for traits. > > But sure thing, i should not forget to say about what SystemEditor > gives: atomic updates. > My only concern is that price for having such feature is a bit too > high as to me: > - big and complex (okay, non-trivial ;) ) logic > - a maintenance needed if we go and hack Class/Behavior/Traits etc). > > for the other features, i don't see significant advantage of using it > over something else (less complex > and more compact). Final comment on SystemEditor - it is not hardwired into Deltas, in fact it is kept separate and the idea was to be able to have different "Applier" classes using different mechanisms. regards, GÃ¶ran

Oct. 21, 2012