New subject: [Pharo-project] FFI crashes VM on Linux (was: Z3950 on OSX: module yaz3 not found)

Dec. 24, 2010

      Which VM have you used ?

I cross post to vm-dev list.

Laurent.

On Fri, Dec 24, 2010 at 3:54 AM, HernÃ¡n Morales Durand <
hernan.morales@gmail.com> wrote:
...
Laurent, not much progress here :(
I just reproduced the Unix VM crash. First I've linked the library so
we do not have to change every module name call
user@ubuntu:/usr/lib$ ln -T -s /usr/lib/libyaz.so.3.0.0 yaz
and I've tried to execute the library tracer in the shell script
ltrace -f -l /usr/lib/libyaz.so.3.0.0 -o out.txt "$BASE/squeakvm" \
       -plugins "$BASE" \
       -encoding latin1 \
       -vm-display-X11 \
       "$ROOT/Contents/Resources/pharo.image"
but it crash the VM too with an unhandled exception even before the
GUI is displayed.
What I've observed is that every FFI call which receives an
ExternalStructure as parameter kills the VM. This is weird because
this doesn't happen in the Windows VM. I ran this little script I use
to debug the library
| conn lib rs rsSize records |
lib := Z3950FFILibrary default.
conn := lib
       createConnectionTo: #Alabama asZ3950Server hostName
       port: 7090.
lib createOptions.
lib
       setConnection: conn
       optionName: 'preferredRecordSyntax'
       optionValue: 'USMARC'.
rs := lib
       searchPqf: conn
       query:  '@attr 1=1003 collins'.
rsSize := lib resultSetSize: rs.
rsSize > 0
       ifFalse:  [ 'no results' ].
records := Array new: rsSize.
0 to: rsSize - 1 do: [: pos |
       record := lib resultSetRecord: rs position: pos.
       record isNil not
               ifTrue:  [
                       records
                               at: pos + 1
                               put: ( lib
                                       getRecord: record
                                       function: 'render; charset=marc8,
iso8859-1'
                                       length: nil ). ] ].
records
and the the doIt just hangs the VM, the console output going through
the Debugger is the following:
Segmentation fault
-1370902312 ContextPart>doPrimitive:method:receiver:args:
-1370902404 ContextPart>tryPrimitiveFor:receiver:args:
-1370902788 ContextPart>send:to:with:super:
-1370902896 ContextPart>send:super:numArgs:
-1370902988 InstructionStream>interpretExtension:in:for:
-1370903080 InstructionStream>interpretNextInstructionFor:
-1370903172 ContextPart>step
-1370903280 Process>step
-1370903372 Process>step:
-1370920392 Debugger>send
-1370937720 PluggableButtonMorph>performAction
-1370937812 PluggableButtonMorph>mouseUp:
-1370937996 SequenceableCollection>do:
-1370957140 PluggableButtonMorph>mouseUp:
-1370957232 Morph>handleMouseUp:
-1370957324 MouseButtonEvent>sentTo:
-1370957416 Morph>handleEvent:
-1370957584 Morph>handleFocusEvent:
-1370957676 HandMorph>sendFocusEvent:to:clear:
-1370957768 PasteUpMorph>becomeActiveDuring:
-1370957860 BlockClosure>on:do:
-1370957992 PasteUpMorph>becomeActiveDuring:
-1370958084 HandMorph>sendFocusEvent:to:clear:
-1370958260 HandMorph>sendEvent:focus:clear:
-1370958352 HandMorph>sendMouseEvent:
-1370958444 HandMorph>handleEvent:
-1370966728 HandMorph>processEvents
-1370966836 WorldState>doOneCycleNowFor:
-1370966952 SequenceableCollection>do:
-1370967044 WorldState>handsDo:
-1370967136 WorldState>doOneCycleNowFor:
-1370969680 WorldState>doOneCycleFor:
-1370969772 PasteUpMorph>doOneCycle
-1383611064 >spawnNewProcess
-1383611192 BlockClosure>newProcess
Aborted
it would be nice if someone could tell something about debugging FFI
calls, or just any idea to try.
Cheers,
HernÃ¡n
2010/12/23 laurent laffont <laurent.laffont@gmail.com>:
...
Hi,
I have unmangled all function names. Image also crashes when evaluating
the
Kant test. (but this time I get ILLEGAL INSTRUCTION on strace). I've
tried
with Cog and the VM in the Seaside OneClick.
I've uploaded the modified packages on squeaksource
here: http://www.squeaksource.com/LaurentLSandbox.html   (you can use
Z3950Slice to load all at once).
Laurent
On Thu, Dec 23, 2010 at 1:57 AM, HernÃ¡n Morales Durand
<hernan.morales@gmail.com> wrote:
...
Hi Laurent,
Thanks for the feedback, I've uploaded the repository with the missing
method but I cannot reproduce the crash in Windows. Have you unmangled
every function name tested? Are you running just the
Z3950FFILibraryTest? Notice the Z3950FFIServerLibraryTest tests are
performed against a Z39.50 server running in local host, otherwise
will fail (however that shouldn't crash the VM), so you may want to
run only the Z3950FFILibraryTest suite. If you or anyone want a local
Zebra server for Win32 - although most configuration files should work
with other platforms - for testing purposes please let me know.
Does the Kant test crash the VM too?
Z3950Client new
 server: #Acadia;
 query: ( Z3950Query new author: 'Kant' ).
(see http://67.222.12.200:8889/WebOpus/52 for sample successful
results)
...
I will set up a Linux box tomorrow and try to reproduce there.
Cheers,
2010/12/22 laurent laffont <laurent.laffont@gmail.com>:
...
Hi Hernan,
finally I switch back to Linux to make it work, OSX gives me
headaches.
...
...
So on linux the yaz module is found. I get the "Unable to find
function
address" but I found that writing:
<cdecl: ZOOMOptions 'ZOOM_options_create' (void) module: 'yaz'>
instead of
<cdecl: ZOOMOptions '_ZOOM_options_create@0' (void) module: 'yaz3'>
works. If I understand, function names are not mangled when compiling
with
gcc on Linux.
Trying to run tests I've also had a "could not coerce arguments" in
Z3950FFILibrary>>createConnectionTo:port because
Z3950TestCase>>portNumber
answers a String and
<cdecl: ZOOMConnection 'ZOOM_connection_new' (char* short) module:
'yaz'>
so it's waiting for a short. I've replaced portNumber to answer a
SmallInteger.
And now the image crashes on connectionIsIdle call:
write(1, "\ncreateConnectionTo:port:", 25) = 25
write(1, "\nconnectionIsIdle:", 18)     = 18
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
write(1, "\n", 1)                       = 1
tgkill(25830, 25830, SIGABRT)           = 0
--- SIGABRT (Aborted) @ 0 (0) ---
Any idea ?
Laurent Laffont
On Tue, Dec 21, 2010 at 7:58 AM, laurent laffont
<laurent.laffont@gmail.com>
wrote:
...
The module is found ! How:
- Using VM Squeak 5.8b12 2  (instead of Cog or the VM provided in
Seaside
One-Click image)
- go in Squeak\ 5.8b12\ 2.app/Contents/Resources/, then ln -s
/usr/local/lib/libyaz.3.dylib .
Now I have the "Unable to find function address" you have talked
earlier
:) I will look at the mangled function names.
Laurent  --- on the road to the FFI mountain.
On Tue, Dec 21, 2010 at 6:59 AM, HernÃ¡n Morales Durand
<hernan.morales@gmail.com> wrote:
...
Laurent,
I forward this e-mail to the squeak mailing list, may be someone
there
...
...
have a better idea how the OSX VM works. BTW there are more
commentaries here
http://forum.world.st/squeak-dev-Alien-amp-Squeak-FFI-issues-on-Snow-Leopard...
...
Cheers,
HernÃ¡n
2010/12/20 laurent laffont <laurent.laffont@gmail.com>:
...
Hi Hernan,
the strace equivalent on OSX seems to be dtruss (yes I'm learning
OSX
too :)
sudo dtruss -p [pid of process]
in the trace I have this:
stat("/Users/laurentlaffont/Downloads/Seaside.app/Contents/Resources/yaz3.bundle/Contents/MacOS/yaz3\0",
...
0xBFF59440, 0xBFF5AB28) = -1 Err#2
stat64("/Users/laurentlaffont/Downloads/Seaside.app/Contents/Resources/yaz3.bundle/Contents/MacOS/yaz3\0",
...
0xBFF5892C, 0xBFF5AB28) = -1 Err#2
stat("/Users/laurentlaffont/Downloads/Seaside.app/Contents/Resources/yaz3\0",
...
0xBFF59440, 0xBFF5AB28) = 0 0
stat64("/Users/laurentlaffont/Downloads/Seaside.app/Contents/Resources/yaz3\0",
...
0xBFF5892C, 0xBFF5AB28) = 0 0
open("/Users/laurentlaffont/Downloads/Seaside.app/Contents/Resources/yaz3\0",
...
0x0, 0x0) = 12 0
pread(0xC, "\317\372\355\376\a\0", 0x1000, 0x0) = 4096 0
close(0xC) = 0 0
stat("/Users/laurentlaffont/Downloads/Plugins/yaz3.bundle/Contents/MacOS/yaz3\0",
...
0xBFF59440, 0x1000) = -1 Err#2
stat64("/Users/laurentlaffont/Downloads/Plugins/yaz3.bundle/Contents/MacOS/yaz3\0",
...
0xBFF5892C, 0x1000) = -1 Err#2
stat("/Users/laurentlaffont/Downloads/Plugins/yaz3\0", 0xBFF59440,
0x1000) =
-1 Err#2
stat64("/Users/laurentlaffont/Downloads/Plugins/yaz3\0",
0xBFF5892C,
0x1000)
= -1 Err#2
so it looks like the VM is trying to find yaz3 at non standard
Unix
places
(I don't know if it's SqueakVM or OSX related).
I've created Contents/Resources/yaz3.bundle/Contents/MacOS/, put
yaz
lib
there, ln -s libyaz.3.dylib yaz3 but no success.
I've read good
post http://forum.world.st/FFI-Documentation-td2225150.html :
Module Location - where the external library file lives
      - depends on the platform
              - Mac
                      - pre Snow Leopard
                              - checks VM path and common library
paths
                      - Snow Leopard
                              - only looks in VM bundle's
Resources
file, you must either [5]:
                                      - store all external
libraries
there
                                      - ln -s path/to/library
path/to/VM/Resources/library_name
                                      - Change the VM's Info.plist
"SqueakPluginsBuiltInOrLocalOnly" key from
"true" to "false."
For this SqueakPluginsBuiltInOrLocalOnly I suppose I need to
compile
a
VM ?
Cheers,
Laurent Laffont
On Mon, Dec 20, 2010 at 4:12 PM, HernÃ¡n Morales Durand
<hernan.morales@gmail.com> wrote:
...
Hi Laurent
I've read there is a couple of system calll monitors in MacOS,
ktrace
and dtrace (http://en.wikipedia.org/wiki/Ktrace ,
http://en.wikipedia.org/wiki/DTrace) so you may attach to the VM
and
see what's missing?
Cheers,
HernÃ¡n
2010/12/20 laurent laffont <laurent.laffont@gmail.com>:
...
Thanks Hernan.
It still doesn't work. I've also tried to play with
LD_LOAD_PATH
...
...
/
LD_LIBRARY_PATH no success.... I want to make it work so I need
to
learn
:)
For the mangled function names, on Linux and OSX there's nm.
Here's what I get:
$ nm -g libyaz.3.dylib | grep ZOOM_connection_create
0000000000046a30 T _ZOOM_connection_create
Cheers,
Laurent
On Sun, Dec 19, 2010 at 6:23 PM, HernÃ¡n Morales Durand
<hernan.morales@gmail.com> wrote:
...
Hi Laurent,
Thanks for reporting. The correct #moduleName for MacOS was
missing,
I've uploaded a new version answering 'libyaz.3.dylib' which I
guess
is the corresponding for 'yaz3.dll' (according to
http://en.wikipedia.org/wiki/Dynamic_library#Naming ).
I haven't access to a MacOS, but in Windows I've had to find
the
...
mangled function names (i.e. _ZOOM_connection_create@4) so
FFI
can
find the function addresses and you don't get the classic
"Error:
Unable to find function address". I don't know if it's related
since
talks about C++, but you may want to check
http://wiki.squeak.org/squeak/3735
The tool I've used to find the mangled function names in
Windows
is
the Steve Miller's Dependency Walker
http://www.dependencywalker.com/
2010/12/19 laurent laffont <laurent.laffont@gmail.com>:
...
Hi,
I want to use http://www.squeaksource.com/Z3950
Following install instructions, I've dowloaded yaz sources
(I've
tried
4.1.2
and 3.0.52) as it seems there's no binary for OSX,
./configure
&& make
&&
make install.
Then using latest Seaside One Click image I've copied
/usr/local/lib/*yaz*
 in Seaside.app/Contents/MacOS
$ ls ~/Downloads/Seaside.app/Contents/MacOS/
Squeak VM Opt         libyaz.a              libyaz.la
libyaz_icu.a          libyaz_icu.la         libyaz_server.a
libyaz_server.la libyaz.3.dylib        libyaz.dylib
 libyaz_icu.3.dylib    libyaz_icu.dylib
 libyaz_server.3.dylib
libyaz_server.dylib
But then I have a External Module Not Found on
YAZPQFFFILibrary>>createParser
" YAZPQDFFILibrary default createParser "
<cdecl: YAZPQFParser 'yaz_pqf_create' (void) module: 'yaz3'>
^ self externalCallFailed
Any hint ?
Cheers,
Laurent Laffont -- FFI newbie
Pharo Smalltalk Screencasts: http://www.pharocasts.com/
Blog: http://magaloma.blogspot.com/
--
HernÃ¡n Morales
Information Technology Manager,
Institute of Veterinary Genetics.
National Scientific and Technical Research Council (CONICET).
La Plata (1900), Buenos Aires, Argentina.
Telephone: +54 (0221) 421-1799.
Internal: 422
Fax: 425-7980 or 421-1799.

Re: [Pharo-project] FFI crashes VM on Linux (was: Z3950 on OSX: module yaz3 not found)

laurent laffont

HernÃ¡n Morales Durand

tags

participants (2)