Re: [Pharo-project] [squeak-dev] help needed with the new FFI plugin...
On Fri, Jul 22, 2011 at 9:08 AM, Ricardo Moran <richi.moran@gmail.com>wrote:
Yes, indeed. Right now, for most use cases we use the SerialPort class, but we still have some FFI code for other stuff hanging around. I plan to write it as a plugin as soon as I have the time.
But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on. So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Let's defer gratification :)
Cheers, RIcho
On Fri, Jul 22, 2011 at 10:37 AM, Bert Freudenberg <bert@freudenbergs.de>wrote:
It's not part of Etoys yet. If its low-level interface indeed requires FFI, then that interface would have to be rewritten as plugin for Etoys to include it.
- Bert -
On 22.07.2011, at 14:55, karl ramberg wrote:
What about the Physical Etoys project ? http://tecnodacta.com.ar/gira/projects/physical-etoys/
Karl
On Fri, Jul 22, 2011 at 2:20 PM, Bert Freudenberg <bert@freudenbergs.de>wrote:
On 22.07.2011, at 02:10, Eliot Miranda wrote:
On Thu, Jul 21, 2011 at 4:57 PM, Igor Stasenko <siguctua@gmail.com>
wrote:
On 22 July 2011 02:34, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi All, today I committed the threaded FFI plugin along with the threaded VM and hope to be able to support people in doing threaded FFI work on the x86 platforms really soon now (Esteban, amongst others, this means you ;) ). But to put the image-level code that uses the new FFI plugin into production the standard VM needs to be upgraded to not crash with the ExternalFunction image changes the new threaded FFI plugin wants. The main change is that ExternalFunction changes size, gaining a stackSize inst var which is used to tell the FFI plugin how much stack spacer to reserve for marshalling (its computed on first use). The best way for this to happen is to start using the ThreadedFFIPlugin as the standard plugin in the Interpreter VM. The ThreadedFFIPlugin overall is simpler than the old FFI plugin (no assembler support code) and better (fully reentrant). But for this to happen we may need to implement the platform=specific parts of the ThreadedFFIPlugin (still entirely Smalltalk code) for PowerPC and ARM (and perhaps others, SPARC??). I understand what has to be done and I can explain it to anyone interested in working on the problem, but I don't have any hardware with which I can test the results so its pretty pointless my doing the work and producing something inevitably broken. So are there people out there who would be interested in doing the port to PPC and ARM? It means fleshing out the classes ThreadedARMFFIPlugin ThreadedFFICalloutStateForARM ThreadedPPCBEFFIPlugin ThreadedFFICalloutStateForPPC and having a solid understanding of how alloca works to do stack allocation and how the ABIs on ARM and PPC pass registers. here's hoping there are a few good low-level people who would be interested. Let me know.
I wonder if you find anyone today using these archs, not saying using squeak/pharo on them + knowing low-level details + wanting to help :)
Right, chances are thin. But then if no one is interested then the work doesn't need to be done. Bert, you mentioned the other day that PPC was still relevant for etoys. But am I right in thinking etoys never uses the FFI?
Right. Etoys should run on both ARM and PPC, but it does not use FFI.
- Bert -
-- best, Eliot
On Fri, Jul 22, 2011 at 1:25 PM, Eliot Miranda <eliot.miranda@gmail.com>wrote:
On Fri, Jul 22, 2011 at 9:08 AM, Ricardo Moran <richi.moran@gmail.com>wrote:
Yes, indeed. Right now, for most use cases we use the SerialPort class, but we still have some FFI code for other stuff hanging around. I plan to write it as a plugin as soon as I have the time.
But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on.
So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Let's defer gratification :)
Hi Eliot, I would love to help but I don't have the hardware nor the knowledge to do it. Sorry... Cheers, Richo
Cheers, RIcho
On Fri, Jul 22, 2011 at 10:37 AM, Bert Freudenberg <bert@freudenbergs.de>wrote:
It's not part of Etoys yet. If its low-level interface indeed requires FFI, then that interface would have to be rewritten as plugin for Etoys to include it.
- Bert -
On 22.07.2011, at 14:55, karl ramberg wrote:
What about the Physical Etoys project ? http://tecnodacta.com.ar/gira/projects/physical-etoys/
Karl
On Fri, Jul 22, 2011 at 2:20 PM, Bert Freudenberg <bert@freudenbergs.de>wrote:
On 22.07.2011, at 02:10, Eliot Miranda wrote:
On Thu, Jul 21, 2011 at 4:57 PM, Igor Stasenko <siguctua@gmail.com>
wrote:
On 22 July 2011 02:34, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi All, today I committed the threaded FFI plugin along with the threaded VM and hope to be able to support people in doing threaded FFI work on the x86 platforms really soon now (Esteban, amongst others, this means you ;) ). But to put the image-level code that uses the new FFI plugin into production the standard VM needs to be upgraded to not crash with the ExternalFunction image changes the new threaded FFI plugin wants. The main change is that ExternalFunction changes size, gaining a stackSize inst var which is used to tell the FFI plugin how much stack spacer to reserve for marshalling (its computed on first use). The best way for this to happen is to start using the ThreadedFFIPlugin as the standard plugin in the Interpreter VM. The ThreadedFFIPlugin overall is simpler than the old FFI plugin (no assembler support code) and better (fully reentrant). But for this to happen we may need to implement the platform=specific parts of the ThreadedFFIPlugin (still entirely Smalltalk code) for PowerPC and ARM (and perhaps others, SPARC??). I understand what has to be done and I can explain it to anyone interested in working on the problem, but I don't have any hardware with which I can test the results so its pretty pointless my doing the work and producing something inevitably broken. So are there people out there who would be interested in doing the port to PPC and ARM? It means fleshing out the classes ThreadedARMFFIPlugin ThreadedFFICalloutStateForARM ThreadedPPCBEFFIPlugin ThreadedFFICalloutStateForPPC and having a solid understanding of how alloca works to do stack allocation and how the ABIs on ARM and PPC pass registers. here's hoping there are a few good low-level people who would be interested. Let me know.
I wonder if you find anyone today using these archs, not saying using squeak/pharo on them + knowing low-level details + wanting to help :)
Right, chances are thin. But then if no one is interested then the work doesn't need to be done. Bert, you mentioned the other day that PPC was still relevant for etoys. But am I right in thinking etoys never uses the FFI?
Right. Etoys should run on both ARM and PPC, but it does not use FFI.
- Bert -
-- best, Eliot
On Fri, 22 Jul 2011, Eliot Miranda wrote:
But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? Â We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). Â So I really do appeal to people to help me put the effort in here. Â I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. Â I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. Â But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on.
So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Â Let's defer gratification :)
QEMU can be used for emulating various CPUs, so if you don't have an ARM, PPC or SPARC hardware, you can still develop/test FFI with it. Threaded FFI is great, it opens new possibilities, but nowadays the feature I miss the most is the ability to call functions in 64-bit (x86_64) libraries (from a 32-bit VM/image). Levente
On Sat, Jul 23, 2011 at 01:50:49PM +0200, Levente Uzonyi wrote:
Threaded FFI is great, it opens new possibilities, but nowadays the feature I miss the most is the ability to call functions in 64-bit (x86_64) libraries (from a 32-bit VM/image).
As far as I know, mixing 32-bit and 64-bit libraries in a single executable (VM) is not feasible, at least not on Linux. Maybe it would work in the case of statically linked libraries, I'm not certain. The traditional FFI does not work for 64-bit VMs, and I expect that this needs to be resolved for any new FFI implementation as well, because some of the issues were in the interpreter itself, not just the FFI plugin. The issue also may affect other things, e.g. Igor's proposed #primLoadModule primitive crashes the VM if compiled for 64 bit. I had intended to push the fixes for 64-bit FFI as a VM team initiative, but it's a complex set of changes that require coordination of VMMaker (oscog plus traditional), as well as of all the platforms (oscog, trunk, all platforms) so IMO this would be a lot to take on right now. But I guess that I should ask - is there interest in implementing fixes for 64-bit FFI? Some background: Mantis report and patches to implement the fixes, circa 2008. I assume some bit rot will have set in by this time: <http://bugs.squeak.org/view.php?id=7237> Status of the fixes, as tested on various platforms: <http://lists.squeakfoundation.org/pipermail/vm-dev/2008-May/001945.html> The last discussion relative to VM team priorities: <http://lists.squeakfoundation.org/pipermail/vm-dev/2009-July/002853.html> Dave
participants (4)
-
David T. Lewis -
Eliot Miranda -
Levente Uzonyi -
Ricardo Moran