using UFFI in non-crashing way
Hi, is it possible to use UFFI and avoid crashing the image when the called code segfaults? In other words, can I somehow wrap the call and throw in-image Exception/Error on failure instead of the whole thing crashing? Or is the only way to do that have the called thing running as a separate app (e.g. via OSSubProcess) and communicate via pipes or sockets? Thanks, Peter
On 26 Mar 2017, at 15:40, Peter Uhnak <i.uhnak@gmail.com> wrote:
Hi,
is it possible to use UFFI and avoid crashing the image when the called code segfaults?
In other words, can I somehow wrap the call and throw in-image Exception/Error on failure instead of the whole thing crashing?
thatâs not possible, once you segfault, you segfault.
Or is the only way to do that have the called thing running as a separate app (e.g. via OSSubProcess) and communicate via pipes or sockets?
thatâs not done. no idea how much effort is required to implement, but I imagine is not trivial. Esteban
Thanks, Peter
On Sun, Mar 26, 2017 at 05:45:04PM +0200, Esteban Lorenzano wrote:
On 26 Mar 2017, at 15:40, Peter Uhnak <i.uhnak@gmail.com> wrote:
Hi,
is it possible to use UFFI and avoid crashing the image when the called code segfaults?
In other words, can I somehow wrap the call and throw in-image Exception/Error on failure instead of the whole thing crashing?
that???s not possible, once you segfault, you segfault.
Or is the only way to do that have the called thing running as a separate app (e.g. via OSSubProcess) and communicate via pipes or sockets?
that???s not done. no idea how much effort is required to implement, but I imagine is not trivial.
I have not tried it with FFI calls, but maybe this will do what you want: RemoteTask do: [ 2 + 2 ] ==> 4 Instead of [ 2 + 2 ] use some block containing your FFI calls. It works in Pharo. To get RemoteTask, load OSProcess and CommandShell. You also need Fuel, but I think that is in the image by default. An explanation of RemoteTask is at http://wiki.squeak.org/squeak/6176. You are right, it was not trivial to implement ;-) HTH, Dave
On Sun, Mar 26, 2017 at 05:45:04PM +0200, Esteban Lorenzano wrote:
On 26 Mar 2017, at 15:40, Peter Uhnak <i.uhnak@gmail.com> wrote:
Hi,
is it possible to use UFFI and avoid crashing the image when the called code segfaults?
In other words, can I somehow wrap the call and throw in-image Exception/Error on failure instead of the whole thing crashing?
thatâs not possible, once you segfault, you segfault.
Ah, ok.
Or is the only way to do that have the called thing running as a separate app (e.g. via OSSubProcess) and communicate via pipes or sockets?
thatâs not done. no idea how much effort is required to implement, but I imagine is not trivial.
Nono, I didn't mean making UFFI calls via socket (although that would be interesting), but rather making a C app that would make the calls, and that would provide a socket interface. (Which is what I typically do with other langs). Thanks, Peter
Le 26/03/2017 à 19:21, Peter Uhnak a écrit :
On Sun, Mar 26, 2017 at 05:45:04PM +0200, Esteban Lorenzano wrote:
On 26 Mar 2017, at 15:40, Peter Uhnak <i.uhnak@gmail.com> wrote:
Hi,
is it possible to use UFFI and avoid crashing the image when the called code segfaults?
In other words, can I somehow wrap the call and throw in-image Exception/Error on failure instead of the whole thing crashing?
thatâs not possible, once you segfault, you segfault.
Ah, ok.
Or is the only way to do that have the called thing running as a separate app (e.g. via OSSubProcess) and communicate via pipes or sockets?
thatâs not done. no idea how much effort is required to implement, but I imagine is not trivial.
Nono, I didn't mean making UFFI calls via socket (although that would be interesting), but rather making a C app that would make the calls, and that would provide a socket interface. (Which is what I typically do with other langs).
With UFFI and a standard way to marshall types over a socket (ASN.1 or even xml-rpc), this could be generated automatically. Of course, you may have a few performance issues, but you could have a long-lived child to avoid paying the startup time, and mmap / shared memory to lower the cost of moving memory back and forth. Thierry
Thanks, Peter
On Sun, Mar 26, 2017 at 7:21 PM, Peter Uhnak <i.uhnak@gmail.com> wrote:
On Sun, Mar 26, 2017 at 05:45:04PM +0200, Esteban Lorenzano wrote:
On 26 Mar 2017, at 15:40, Peter Uhnak <i.uhnak@gmail.com> wrote:
Hi,
is it possible to use UFFI and avoid crashing the image when the
called code segfaults?
In other words, can I somehow wrap the call and throw in-image
Exception/Error on failure instead of the whole thing crashing?
thatâs not possible, once you segfault, you segfault.
Ah, ok.
Well, you can capture the signal, but the problem is not the signal itself but what caused the problem. The most common case of segfaut is memory corruption (and thus a program that tries to wrongly interpret wrong addresses). If you already corrupted the memory doing something wrong, how can you "fix it"?
Or is the only way to do that have the called thing running as a
separate app (e.g. via OSSubProcess) and communicate via pipes or sockets?
thatâs not done. no idea how much effort is required to implement, but I imagine is not
trivial.
Nono, I didn't mean making UFFI calls via socket (although that would be interesting), but rather making a C app that would make the calls, and that would provide a socket interface. (Which is what I typically do with other langs).
Thanks, Peter
2017-03-27 11:34 GMT+02:00 Guillermo Polito <guillermopolito@gmail.com>:
Well, you can capture the signal, but the problem is not the signal itself but what caused the problem. The most common case of segfaut is memory corruption (and thus a program that tries to wrongly interpret wrong addresses). If you already corrupted the memory doing something wrong, how can you "fix it"?
I think important case here is creating wrappers for new C libraries. Common mistake is passing wrong structures to foreign function. Then image crashes and you need restart it and recover code - big waste of time and losing smalltalk liveness.
On Mon, Mar 27, 2017 at 12:04 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2017-03-27 11:34 GMT+02:00 Guillermo Polito <guillermopolito@gmail.com>:
Well, you can capture the signal, but the problem is not the signal itself but what caused the problem. The most common case of segfaut is memory corruption (and thus a program that tries to wrongly interpret wrong addresses). If you already corrupted the memory doing something wrong, how can you "fix it"?
I think important case here is creating wrappers for new C libraries. Common mistake is passing wrong structures to foreign function. Then image crashes and you need restart it and recover code - big waste of time and losing smalltalk liveness.
But then the problem is on "how to not lose your code", not in FFI...
On Mon, Mar 27, 2017 at 1:01 PM, Guillermo Polito <guillermopolito@gmail.com
wrote:
On Mon, Mar 27, 2017 at 12:04 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2017-03-27 11:34 GMT+02:00 Guillermo Polito <guillermopolito@gmail.com>:
Well, you can capture the signal, but the problem is not the signal itself but what caused the problem. The most common case of segfaut is memory corruption (and thus a program that tries to wrongly interpret wrong addresses). If you already corrupted the memory doing something wrong, how can you "fix it"?
I think important case here is creating wrappers for new C libraries. Common mistake is passing wrong structures to foreign function. Then image crashes and you need restart it and recover code - big waste of time and losing smalltalk liveness.
But then the problem is on "how to not lose your code", not in FFI...
Am spending my time restarting my image due to this. Very frustrating to recreate the setup etc. Havign a way to detect go back one frame would be very handy. Phil
2017-04-06 10:37 GMT+02:00 Denis Kudriashov <dionisiydk@gmail.com>:
But then the problem is on "how to not lose your code", not in FFI...
Restarting image is a problem. For any other "smalltalk" problems we got suitable debugger but not in that case.
Also when image crashed we need to recover all objects state for the script which produce crash
On 6 Apr 2017, at 10:39, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2017-04-06 10:37 GMT+02:00 Denis Kudriashov <dionisiydk@gmail.com <mailto:dionisiydk@gmail.com>>: But then the problem is on "how to not lose your code", not in FFI...
Restarting image is a problem. For any other "smalltalk" problems we got suitable debugger but not in that case.
Also when image crashed we need to recover all objects state for the script which produce crash
FFI is a border. Once you start working with it, you are in C world. And C world is nasty, hard and cumbersome. In C world, pointers not correctly allocâd produce a crash. In C world, pointers not correctly freed also produce a crash, eventually (and those are harder to fix). In C world everything pointing to wrong location will produce a crash (and you can point anything to anywhere). Thatâs why garbage collectors were invented, after all. So my only advice here is: - recheck all your allocs/free, rigorously - save your image before test - be sure epicea is working Finally, I would say the only way to prevent your image to crash is to not execute FFI calls inside it. To do this, you can always use ImageWorker (http://smalltalkhub.com/#!/~PharoExtras/ImageWorker) to actually execute your tests in a different execution. Yes is slow, but it will be safer. Esteban
2017-04-07 8:22 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com>:
But then the problem is on "how to not lose your code", not in FFI...
Restarting image is a problem. For any other "smalltalk" problems we got suitable debugger but not in that case.
Also when image crashed we need to recover all objects state for the script which produce crash
FFI is a border. Once you start working with it, you are in C world. And C world is nasty, hard and cumbersome.
Yes. And my remark was about possibility to catch C error to prevent image crash, why it will make FFI less hard.
Do you guys received my CPU exceptions solution message ? Because the issue you described is already resolved by hardware. Simple and elegant solution too On Fri, 7 Apr 2017 at 11:03, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2017-04-07 8:22 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com>:
But then the problem is on "how to not lose your code", not in FFI...
Restarting image is a problem. For any other "smalltalk" problems we got suitable debugger but not in that case.
Also when image crashed we need to recover all objects state for the script which produce crash
FFI is a border. Once you start working with it, you are in C world. And C world is nasty, hard and cumbersome.
Yes. And my remark was about possibility to catch C error to prevent image crash, why it will make FFI less hard.
On 07/04/17 18:28, Dimitris Chloupis wrote:
Do you guys received my CPU exceptions solution message ? Because the issue you described is already resolved by hardware. Simple and elegant solution too On Fri, 7 Apr 2017 at 11:03, Denis Kudriashov <dionisiydk@gmail.com <mailto:dionisiydk@gmail.com>> wrote:
2017-04-07 8:22 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>>:
But then the problem is on "how to not lose your code", not in FFI...
Restarting image is a problem. For any other "smalltalk" problems we got suitable debugger but not in that case.
Also when image crashed we need to recover all objects state for the script which produce crash
FFI is a border. Once you start working with it, you are in C world. And C world is nasty, hard and cumbersome.
Yes. And my remark was about possibility to catch C error to prevent image crash, why it will make FFI less hard.
My experience with the UFFI so far is quite positive. Of course, a lot of care needs to go to the C part, but that's a fact of life with C's unsafe computational model. So, if the C code is well written and thoroughly tested, possibly outside Pharo, then integrating it by means of the UFFI is rather straightforward. I agree that small errors using the UFFI can lead to spurious or catastrophic outcomes, so unit (or even smaller scoped) testing really becomes vital.
From what I can tell so far, the UFFI is fast, well performing and, with some care, not even so hard as it might seem at first sight.
Greetings Raffaello
On Sun, May 7, 2017 at 7:45 PM, <raffaello.giulietti@lifeware.ch> wrote:
On 07/04/17 18:28, Dimitris Chloupis wrote:
Do you guys received my CPU exceptions solution message ? Because the issue you described is already resolved by hardware. Simple and elegant solution too On Fri, 7 Apr 2017 at 11:03, Denis Kudriashov <dionisiydk@gmail.com <mailto:dionisiydk@gmail.com>> wrote:
2017-04-07 8:22 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>>:
But then the problem is on "how to not lose your code", not in FFI...
Restarting image is a problem. For any other "smalltalk" problems we got suitable debugger but not in that case.
Also when image crashed we need to recover all objects state for the script which produce crash
FFI is a border. Once you start working with it, you are in C world. And C world is nasty, hard and cumbersome.
Yes. And my remark was about possibility to catch C error to prevent image crash, why it will make FFI less hard.
My experience with the UFFI so far is quite positive. Of course, a lot of care needs to go to the C part, but that's a fact of life with C's unsafe computational model.
So, if the C code is well written and thoroughly tested, possibly outside Pharo, then integrating it by means of the UFFI is rather straightforward. I agree that small errors using the UFFI can lead to spurious or catastrophic outcomes, so unit (or even smaller scoped) testing really becomes vital.
From what I can tell so far, the UFFI is fast, well performing and, with some care, not even so hard as it might seem at first sight.
Greetings Raffaello
btw, I started a UFFI / Clang tutorial here (that I need to get back to) http://blog.openinworld.com/2016/09/pharo-libclang-ffi-part-1-preamble/ cheers -ben
Hi,
is it possible to use UFFI and avoid crashing the image when the called code segfaults?
Yes its possible. When I was exploring the means to live code with C++ I came to specific implementations of live coding for C++. One of them explored a feature called "Hardware exceptions" .Hardware exceptions are very similar to regular exception with the big difference that are handled by the CPU and not the language or the OS. Crashes is a standard behavior that is imposed by the OS kernel in the case that an app tries to access memory is not allowed to access or access it in the way it is not allowed to access it. An exception will capture this behavior it wont allow the application to crash. Hardware exceptions are a standard feature for most CPUs and obviously they are not implemented at language level. For example you can find documentation about Hardware exceptions on Windows at https://msdn.microsoft.com/en-us/library/windows/desktop/ms680657(v=vs.85).a... This means the C code you will be calling with UFFI will have to be wrapped inside a Hardware exception. When a crash is suppose to happen inside code that is wrapped with a hardware exception, the error (seffault in your example) will be displayed in the console but the application wont crash it will continue to execute depending on how you handle the exception. Generally hardware exception are to be avoided because they can hide bugs because the OS not always reports and can crash an app without an error in that you will be oblivious to what went wrong, when and how as the crash will not happen.
In other words, can I somehow wrap the call and throw in-image Exception/Error on failure instead of the whole thing crashing?
Its possible if you make a C function that contains a hardware exception take as argument a pointer to a specific function and then calls it inside the exception.
Or is the only way to do that have the called thing running as a separate app (e.g. via OSSubProcess) and communicate via pipes or sockets?
It wont make a diffirence if the crash is caused by UFFI it will still crash Pharo that applies even if you use hardware exceptions , so you need always to be careful. Plus Pharo VM at any point can crash for its own reason, that is completely unrelated to what you are doing Thanks,
Peter
So yes its possible, but you still need to be careful.
participants (10)
-
Ben Coman -
David T. Lewis -
Denis Kudriashov -
Dimitris Chloupis -
Esteban Lorenzano -
Guillermo Polito -
Peter Uhnak -
phil＠highoctane.be -
raffaello.giulietti＠lifeware.ch -
Thierry Goubier