Stef, I've always understood why you launched Pharo. On FFI/Alien and blocking the entire VM, I injected Dolphin's overlapped calls into a discussion of the (rather serious) problems that can arise when something does not return as soon as expected (or at all), and somebody with good VM awareness (I do not recall who[*]) described a way to get around it. I recall thinking what a loss it is that good ideas often don't get packaged for general use. FWIW, here are my suggestions for ideas that should be given high priority: (1) Alien plugins for Linux and Windows; Alien as a mac-only entity is far less attractive than a cross-platform alternative. (2) For FFI, we need a generic callback mechanism. It sounds as though one can be built using a plugin and some indirection. (3) For FFI and Alien, we need something like Dolphin's overlapped calls. Hopefully a generic solution that starts a thread and waits on a semaphore can be created once, and perhaps eventually enhanced with some thread pooling options. For now, I'd take one os thread per call, or (perhaps the ideal) one os thread per Smalltalk process blocked on such a call, as I think Dolphin does now, in part due to Windows stupidities. I also recall a lengthy discussion about efficiency in making calls to COM objects from Dolphin, with someone arguing very hard to make calls a certain way because the alternative (while much simpler) was too slow. The flaw in that argument was that calls in question were by nature few and infrequent; the overhead of making the call was dwarfed by the work that resulted. Much of my use of FFI is similar. I am not arguing in favor of inefficient calls, but a sub-optimal but flexible way to make a callback (FFI) and to make a call on an os thread (FFI/Alien) would be much appreciated in most situations, even if it takes a few extra milliseconds compared to something optimally coded if I had time and knowledge to do so. I do not make millions of external calls; I make a few that do millions of floating point operations. A little extra time to put a socket connect (thinking OpenSSL) call on a background thread is trivial next to the cost of having a direct call go into limbo and hang a server process. Bill [*] seriously, it might have been "the FFI maintainer" or somebody else. I do recall that whoever it is knows a lot more about the vm than I do. ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Stéphane Ducasse [stephane.ducasse@inria.fr] Sent: Sunday, February 14, 2010 4:30 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Making FFI non-intrusive (Was: Re: Pharo changing the game) Ok thanks for the historical refresh. Some people may now understand why we finally decided to do Pharo. Now let us stop ranting and think about the blue sky of the new horizon. What do we do in Pharo 1.1? :) I'm full ears open. Stef
It is interesting to note that of course, after this was added to 3.9, there was someone *very hard* arguing that adding Pragmas was a very dumb idea...
Who that person was is left as an exercise to the reader of course ;-)
Hint: FFI has its own proprietary hardcoded pragma format. It unnecessary complicates the compiler even when not loaded. Not to mention that it breaks all tools that have their own parser, such as RB for example.
I'd like to discuss, what changes to compiler we may introduce to allow FFI to handle pragmas in non-intrusive manner.
Very simple: FFI just needs to use the pragmas as introduces in 3.9, than remove all knowledge about FFI from the Compiler. Problem solved. Isn't that beautiful?
< cdecl: void 'InvalRect' ( MacRect ) module: 'InterfaceLib'>
of that would be changed so that the stuff inside the <> would be a standard Smalltalk Message expression, problem solved.
to illustrate... using the SMaCC grammar of the NewCompiler (a bit simplified) as extendend for Pragmas, Primitives and FFI by Mathiue Suen.
A method in Squeak has Pragmas:
Method: | MethodPattern Temporaries Pragmas Statements {#methodTempsPragma:};
So they are difined like this (we can have multiple ones):
Pragmas: "<" PragmaMessage ">" {#pragma:} | Pragmas "<" PragmaMessage ">" {#pragmas:}
So it's quite simple: there is one message expression inside... simple. But there is FFI. Primitives are there, too, but that's not really a problem: they fit in the Pragmas. primitive: 15 is a valid message in Smalltalk. (Math did the gramar for primitives to have nice error messages, but that's not needed)
PragmaMessage: Apicall {#messagePragma:} | Primitive {#messagePragma:} | MessagePragma {#messagePragma:};
How Primitive and MessagePragma are done is not important. But we need Gramar rules just to be able to parse FFI:
Apicall: TypeCall ExternalType IndexName "(" ParameterApicall <rightParentheses> {#externalCall:} | TypeCall ExternalType IndexName "(" ParameterApicall <rightParentheses> "module:" <string> {#externalModuleCall:};
IndexName: <string> {#externalFunction:} | <number> {#externalIndex:};
TypeCall: "apicall:" {#callConvention:} | "cdecl:" {#callConvention:};
ParameterApicall: ExternalType {#parameterExtCall:} | ParameterApicall ExternalType {#parametersExtCall:};
ExternalType: <name> {#externalType:} | <name> "*" {#externalTypePointer:};
And this is *always* in the Grammar. Even when FFI is not loaded! (a nice example where our simple notion of modularity brakes down completely).
No the solution here is simple: a small trivial modification to FFI to use the Evil Pragmas as introduced by the Dumb People (tm) in 3.9:
< cdecl: 'void InvalRect ( MacRect )' module: 'InterfaceLib'>
or *whatever*. just something that is a valid Smalltalk message expression... can't be hard, can it?
Ah. And and about "notion of modularity"... of course the problem "how can I extend the language from my package without having to spread code over the parser, compiler, debugger, browser..." is intellectually an interesting one. And, oh, we are payed for finding these kind of Problems and especially solving these Problems in a way that goes beyond of just doing a pragmatic and simple solution (aka what I described here). (They call it "Academic Research". Involves mostly a programming language called TeX and can be very boring at times but it's complementary to the practical world... in a very interesting way).
And int his context the result is, of course, tataaa....: Lukas' Helvetia:
http://scg.unibe.ch/research/helvetia/languageboxes
See? So *everything* fits together... research and non-research, problems and solutions (on multiple levels), progress in the small and the large!
Marcus
-- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project