I had a brief moment wondering about the general problem we have when calling out through FFI that we expose ourselves to bugs in the external library that crash the Image. I wonder... 1. Debuggers like gdb can run a program that crashes, without themselves itself crashing. Could our FFI calls be wrapped in similar mechanism? 2. Languages have exception handling, so that errors like divide by zero can be caught. Could our FFI calls be wrapped in similar mechanism? cheers -ben
Hello, 1. Debuggers like gdb can run a program that crashes, without themselves
itself crashing. Could our FFI calls be wrapped in similar mechanism?
I don't think so. A debugger does not run in the same process as the program being debugged. Also, you don't have any guarantee that you have not corrupted something important as the VM. If we have the debugger in the same process as the program being debugged, then the debugger could be corrupted by the error in the external C library. Trying to do something like that could be very dangerous.
2. Languages have exception handling, so that errors like divide by zero can be caught. Could our FFI calls be wrapped in similar mechanism? cheers -ben
The short answer is that it cannot be done automatically, without losing a lot in portability Exception handling implementations depends in the language, operating system and CPU architectude. The way that exceptions are implemented is a wild territory. For C++, you do not have a standard ABI. You cannot interface directly with C++ without losing portability. For wrapping C++, you first have to write a C wrapper for the C++ library, and then you have to do a wrapper in Pharo to the C library. In this C wrapper, you could something like catching all the exceptions and store them in some data structure that can be retrieved later. Then in the Pharo image you have to check for a pending exception in this data structure, and then transform wrapped exception into something more appropriate. Greetings, Ronie 2014-09-25 21:50 GMT-03:00 Ben Coman <btc@openinworld.com>:
I had a brief moment wondering about the general problem we have when calling out through FFI that we expose ourselves to bugs in the external library that crash the Image. I wonder...
1. Debuggers like gdb can run a program that crashes, without themselves itself crashing. Could our FFI calls be wrapped in similar mechanism?
2. Languages have exception handling, so that errors like divide by zero can be caught. Could our FFI calls be wrapped in similar mechanism? cheers -ben
Hi Ben, On Sep 25, 2014, at 5:50 PM, Ben Coman <btc@openInWorld.com> wrote:
I had a brief moment wondering about the general problem we have when calling out through FFI that we expose ourselves to bugs in the external library that crash the Image. I wonder...
1. Debuggers like gdb can run a program that crashes, without themselves itself crashing. Could our FFI calls be wrapped in similar mechanism?
2. Languages have exception handling, so that errors like divide by zero can be caught. Could our FFI calls be wrapped in similar mechanism? cheers -ben
one thing that is pretty straight-forward to do is to catch and report errors during FFI calls as primitive failures with error codes. I added this to the VisualWorks VM a while ago It works like this: 1. The VM installs a first-level exception handler (windows) or signal handlers for SIGSEGV et al (unix). (This is something the VM already does). 2. When an FFI call out is made the VM can find the activation they made an FDI call out. It must be able to do this if it is to handle a callback. 3. An FFI call out is modified to set a flag, eg inFFICall, that is tested by the first-level exception handler. The flag is cleared when the FFI call completes or cleared when taking a callback and restored when the callback returns. 4. When the first level exception/signal handler is invoked it tests the inFFICall flag. If unset it does what it always did (exit, do nothing, etc). If the flag is set it collects the error information, located the FFI activation for the FFI call, does a long jump that returns from the FFI call as a primitive failure, supplying the error information. Now this is useful because the VM doesn't crash (provided the error was localized in its effects and didnt eg corrupt the heap), and hence easy for the developer to find out which FFI call provoked the error, but not as useful as catching the error in gdb. The exception info is typically an exception code and an address, which may be difficult for the programmer to decode. Hence the mechanism needs to be made optional, typically in by default, and then disabled to investigate under gdb. Does this sound useful? Eliot (phone)
Eliot Miranda wrote:
Hi Ben,
On Sep 25, 2014, at 5:50 PM, Ben Coman <btc@openInWorld.com> wrote:
I had a brief moment wondering about the general problem we have when calling out through FFI that we expose ourselves to bugs in the external library that crash the Image. I wonder...
1. Debuggers like gdb can run a program that crashes, without themselves itself crashing. Could our FFI calls be wrapped in similar mechanism?
2. Languages have exception handling, so that errors like divide by zero can be caught. Could our FFI calls be wrapped in similar mechanism? cheers -ben
one thing that is pretty straight-forward to do is to catch and report errors during FFI calls as primitive failures with error codes. I added this to the VisualWorks VM a while ago It works like this:
1. The VM installs a first-level exception handler (windows) or signal handlers for SIGSEGV et al (unix). (This is something the VM already does).
2. When an FFI call out is made the VM can find the activation they made an FDI call out. It must be able to do this if it is to handle a callback.
3. An FFI call out is modified to set a flag, eg inFFICall, that is tested by the first-level exception handler. The flag is cleared when the FFI call completes or cleared when taking a callback and restored when the callback returns.
4. When the first level exception/signal handler is invoked it tests the inFFICall flag. If unset it does what it always did (exit, do nothing, etc). If the flag is set it collects the error information, located the FFI activation for the FFI call, does a long jump that returns from the FFI call as a primitive failure, supplying the error information.
Now this is useful because the VM doesn't crash (provided the error was localized in its effects and didnt eg corrupt the heap), and hence easy for the developer to find out which FFI call provoked the error, but not as useful as catching the error in gdb. The exception info is typically an exception code and an address, which may be difficult for the programmer to decode. Hence the mechanism needs to be made optional, typically in by default, and then disabled to investigate under gdb.
Does this sound useful?
Eliot (phone)
Its good to know how that works, and that it is standard already. Now I wonder, (though it might not be portable) would it be feasible to use something like mprotect to make the Image memory read-only for the duration of the FFI call? Maybe even use a separate segment just for FFI calls (is it even possible for processes to have additional segments than the usual heap/stack/? Thanks for the detailed reply Eliot, but note (and I should have said already), these are idle thoughts. I'll file answers for later reference, but its not something I'm working on right now. cheers -ben
On Fri, Sep 26, 2014 at 7:59 AM, Ben Coman <btc@openinworld.com> wrote:
Eliot Miranda wrote:
Hi Ben,
On Sep 25, 2014, at 5:50 PM, Ben Coman <btc@openInWorld.com> wrote:
I had a brief moment wondering about the general problem we have when
calling out through FFI that we expose ourselves to bugs in the external library that crash the Image. I wonder...
1. Debuggers like gdb can run a program that crashes, without themselves itself crashing. Could our FFI calls be wrapped in similar mechanism?
2. Languages have exception handling, so that errors like divide by zero can be caught. Could our FFI calls be wrapped in similar mechanism? cheers -ben
one thing that is pretty straight-forward to do is to catch and report errors during FFI calls as primitive failures with error codes. I added this to the VisualWorks VM a while ago It works like this:
1. The VM installs a first-level exception handler (windows) or signal handlers for SIGSEGV et al (unix). (This is something the VM already does).
2. When an FFI call out is made the VM can find the activation they made an FDI call out. It must be able to do this if it is to handle a callback.
3. An FFI call out is modified to set a flag, eg inFFICall, that is tested by the first-level exception handler. The flag is cleared when the FFI call completes or cleared when taking a callback and restored when the callback returns.
4. When the first level exception/signal handler is invoked it tests the inFFICall flag. If unset it does what it always did (exit, do nothing, etc). If the flag is set it collects the error information, located the FFI activation for the FFI call, does a long jump that returns from the FFI call as a primitive failure, supplying the error information.
Now this is useful because the VM doesn't crash (provided the error was localized in its effects and didnt eg corrupt the heap), and hence easy for the developer to find out which FFI call provoked the error, but not as useful as catching the error in gdb. The exception info is typically an exception code and an address, which may be difficult for the programmer to decode. Hence the mechanism needs to be made optional, typically in by default, and then disabled to investigate under gdb.
Does this sound useful?
Eliot (phone)
Its good to know how that works, and that it is standard already. Now I wonder, (though it might not be portable) would it be feasible to use something like mprotect to make the Image memory read-only for the duration of the FFI call? Maybe even use a separate segment just for FFI calls (is it even possible for processes to have additional segments than the usual heap/stack/?
I suspect not on conventional hardware. On the one hand, it would cost too much. You might time how long it takes to mprotect the heap. If you do let me know. But since mprotect is a system call it will have overhead at least that of a typical FFI call. On the other hand, some FFI calls are reads and are made to read data into the heap. The complications of dealing with this (only protecting part of the heap, or allocating a temporary buffer and double-copying the data) are again costs and complications to be avoided. But I was talking to David Chisnall at ESUG and he and colleagues are working on special hardware (implemented in FPGA) on which they do their OO and OS work. This is a capability machine and it allows FFI calls to be invoked cheaply with a capability that excludes writing to the heap.
Thanks for the detailed reply Eliot, but note (and I should have said already), these are idle thoughts. I'll file answers for later reference, but its not something I'm working on right now.
You're welcome. -- best, Eliot
participants (3)
-
Ben Coman -
Eliot Miranda -
Ronie Salgado