How to set library path for UFFI on Linux?
I've just installed the 64-bit version of Pharo 6.1 on OpenSuse linux (64 bit). The Athens demos fail because UFFI looks for libcairo in /usr/lib rather the /usr/lib64. I tried setting LD_LIBRARY_PATH but that has no effect. How can I force it to use lib64? Dan -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Hi dan We are sorry that you encounter a problem. Not sure that this is the solution but check ffiLibraryName ^ CairoLibrary CairoLibrary >> unix32ModuleName "On different flavors of linux the path to library may differ depending on OS distro or whether system is 32 or 64 bit. Packages such as snapcraft can also change the path. Look in a number of known locations, or LD_LIBRARY_PATH." #('/usr/lib/i386-linux-gnu' '/usr/lib32' '/usr/lib') , (((OSEnvironment current getEnv: 'LD_LIBRARY_PATH') ifNil: [ '' ]) substrings: ':') do: [ :path | | libraryPath | libraryPath := path asFileReference / 'libcairo.so.2'. libraryPath exists ifTrue: [ ^ libraryPath fullName ] ]. self error: 'Cannot locate cairo library. Please check if it installed on your system' Better CairoLibrary >> unix64ModuleName "On different flavors of linux the path to library may differ depending on OS distro or whether system is 32 or 64 bit. Packages such as snapcraft can also change the path. Look in a number of known locations, or LD_LIBRARY_PATH." #('/lib/x86_64-linux-gnu' '/usr/lib') , (((OSEnvironment current getEnv: 'LD_LIBRARY_PATH') ifNil: [ '' ]) substrings: ':') do: [ :path | | libraryPath | libraryPath := path asFileReference / 'libcairo.so.2'. libraryPath exists ifTrue: [ ^ libraryPath fullName ] ]. self error: 'Cannot locate cairo library. Please check if it installed on your system' On Fri, Sep 22, 2017 at 6:37 PM, Dan Wilczak <dan.g.wilczak@gmail.com> wrote:
I've just installed the 64-bit version of Pharo 6.1 on OpenSuse linux (64 bit). The Athens demos fail because UFFI looks for libcairo in /usr/lib rather the /usr/lib64. I tried setting LD_LIBRARY_PATH but that has no effect. How can I force it to use lib64?
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
That pointed me in the right direction, thanks. I've got it working now. I think there's a bug there which may bite other people --- If you have both 32-bit and 64-bit versions of the library installed, Unix64ModuleName can find the 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the lib to use. Then when UFFI loads it, it recognizes it as 32-bit and gives up rather than looking in any more directories. Dan -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Can you open a bug entry and propose your solution? Stef On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak <dan.g.wilczak@gmail.com> wrote:
That pointed me in the right direction, thanks. I've got it working now.
I think there's a bug there which may bite other people ---
If you have both 32-bit and 64-bit versions of the library installed, Unix64ModuleName can find the 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the lib to use. Then when UFFI loads it, it recognizes it as 32-bit and gives up rather than looking in any more directories.
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
hi, this is not a bug, UFFI cannot know where a library is (because it can be anywhere). but, not being a bug, this remains a problem :) One solution is to search on LD_LIBRARY_PATH but this is not generalised (it should) for the moment. And of course, this is a solution that works for linux and not other platforms, so not perfect. Also we need to consider architecture, etc. etc. etc. Esteban
On 23 Sep 2017, at 09:16, Stephane Ducasse <stepharo.self@gmail.com> wrote:
Can you open a bug entry and propose your solution?
Stef
On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak <dan.g.wilczak@gmail.com> wrote:
That pointed me in the right direction, thanks. I've got it working now.
I think there's a bug there which may bite other people ---
If you have both 32-bit and 64-bit versions of the library installed, Unix64ModuleName can find the 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the lib to use. Then when UFFI loads it, it recognizes it as 32-bit and gives up rather than looking in any more directories.
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
What about shipping the dedicated libcairo with the Linux Virtual Image? Hilaire Le 23/09/2017 à 10:03, Esteban Lorenzano a écrit :
hi,
this is not a bug, UFFI cannot know where a library is (because it can be anywhere). but, not being a bug, this remains a problem:) One solution is to search on LD_LIBRARY_PATH but this is not generalised (it should) for the moment. And of course, this is a solution that works for linux and not other platforms, so not perfect. Also we need to consider architecture, etc. etc. etc.
Esteban
-- Dr. Geo http://drgeo.eu
It does search on LD_LIBRARY_PATH now, it just fails because it finds the 32-bit libcairo before the 64-bit one. The traceback displayed inside Pharo says "Error: External module not found" which is quite misleading. A minimal fix would be to just provide a better error message. There is a traceback that prints to the console if you launched pharo from it, and it does reveal the true failure, but only to someone who knows what ELFCLASS32 means: ioLoadModule(//usr/lib/libcairo.so.2): //usr/lib/libcairo.so.2: wrong ELF class: ELFCLASS32 Error: External module not found This shows that the information for a better error message is available. A better fix would be to skip the 32-bit lib and keep searching. This looks like it would be messy because CairoLibrary provides the path but âUâ FFI has to deal with it. â As far as I can see, there's no way to just check whether a library is 32- or 64-bit, other than trying to call something in it. I'm a total Pharo novice (I wrote maybe 300 lines of Squeak code several years ago) but this design seems odd to me. Why is LD_LIBRARY_PATH being handled inâ Athens-Cairo rather than UFFI itself? Dan On Sat, Sep 23, 2017 at 4:03 AM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
this is not a bug, UFFI cannot know where a library is (because it can be anywhere). but, not being a bug, this remains a problem :) One solution is to search on LD_LIBRARY_PATH but this is not generalised (it should) for the moment. And of course, this is a solution that works for linux and not other platforms, so not perfect. Also we need to consider architecture, etc. etc. etc.
Esteban
On 23 Sep 2017, at 09:16, Stephane Ducasse <stepharo.self@gmail.com> wrote:
Can you open a bug entry and propose your solution?
Stef
On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak <dan.g.wilczak@gmail.com> wrote:
That pointed me in the right direction, thanks. I've got it working now.
I think there's a bug there which may bite other people ---
If you have both 32-bit and 64-bit versions of the library installed, Unix64ModuleName can find the 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the lib to use. Then when UFFI loads it, it recognizes it as 32-bit and gives up rather than looking in any more directories.
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Hi Dan 2017-09-23 13:24 GMT-03:00 Dan Wilczak <dan.g.wilczak@gmail.com>:
It does search on LD_LIBRARY_PATH now, it just fails because it finds the 32-bit libcairo before the 64-bit one.
The traceback displayed inside Pharo says "Error: External module not found" which is quite misleading. A minimal fix would be to just provide a better error message.
Yes, better error messages saves a lot of time. Have you opened an issue for this?
There is a traceback that prints to the console if you launched pharo from it, and it does reveal the true failure, but only to someone who knows what ELFCLASS32 means:
ioLoadModule(//usr/lib/libcairo.so.2): //usr/lib/libcairo.so.2: wrong ELF class: ELFCLASS32 Error: External module not found
This shows that the information for a better error message is available.
A better fix would be to skip the 32-bit lib and keep searching. This looks like it would be messy because CairoLibrary provides the path but U FFI has to deal with it.
You mean keep searching the whole file system?
As far as I can see, there's no way to just check whether a library is 32- or 64-bit, other than trying to call something in it.
One can use a fork() or exec() with the "file" command in Linux based OS, to find library type. It will answer something like: ELF 32-bit LSB shared object, Intel80386, version 1, not stripped or ELF 64-bit LSB shared object, version 1, not stripped
I'm a total Pharo novice (I wrote maybe 300 lines of Squeak code several years ago) but this design seems odd to me. Why is LD_LIBRARY_PATH being handled in Athens-Cairo rather than UFFI itself?
Dan
On Sat, Sep 23, 2017 at 4:03 AM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
hi,
this is not a bug, UFFI cannot know where a library is (because it can be anywhere). but, not being a bug, this remains a problem :) One solution is to search on LD_LIBRARY_PATH but this is not generalised (it should) for the moment. And of course, this is a solution that works for linux and not other platforms, so not perfect. Also we need to consider architecture, etc. etc. etc.
Esteban
On 23 Sep 2017, at 09:16, Stephane Ducasse <stepharo.self@gmail.com> wrote:
Can you open a bug entry and propose your solution?
Stef
On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak <dan.g.wilczak@gmail.com> wrote:
That pointed me in the right direction, thanks. I've got it working now.
I think there's a bug there which may bite other people ---
If you have both 32-bit and 64-bit versions of the library installed, Unix64ModuleName can find the 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the lib to use. Then when UFFI loads it, it recognizes it as 32-bit and gives up rather than looking in any more directories.
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Hernan - I haven't opened an issue - how do I do it? (I'm very new to Pharo.) About continuing the search - I only mean continuing the search of the LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be needed to accomplish this: 1) Athens-Cairo >> CairoLibrary would have to return a list of paths to all the matching libraries rather than just the first one that it finds. This part seems easy. 2) UFFI Libraries UnixDynamicLoader >> loadLibrary:flag: would have to take the list of paths (rather than just one), check them for being 32-bit or 64-bit, and load the first correct one. I can't find any way to perform that check in Pharo directly. How would you fork or exec the "file" command from inside Pharo? Dan -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
https://pharo.fogbugz.com On Sat, Sep 30, 2017 at 6:53 PM, Dan Wilczak <dan.g.wilczak@gmail.com> wrote:
Hernan -
I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
About continuing the search - I only mean continuing the search of the LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be needed to accomplish this:
1) Athens-Cairo >> CairoLibrary would have to return a list of paths to all the matching libraries rather than just the first one that it finds. This part seems easy.
2) UFFI Libraries UnixDynamicLoader >> loadLibrary:flag: would have to take the list of paths (rather than just one), check them for being 32-bit or 64-bit, and load the first correct one. I can't find any way to perform that check in Pharo directly. How would you fork or exec the "file" command from inside Pharo?
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Stephane Ducasse wrote: Would you pls care to look at the sibling thread "How to make pharo find SQLite", it seems that it struggles with similar problem, is it supposed to created symlinks on linux, or it should be able to fund the library but somehow fails? Thanks, Herby
On Sat, Sep 30, 2017 at 6:53 PM, Dan Wilczak<dan.g.wilczak@gmail.com> wrote:
Hernan -
I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
About continuing the search - I only mean continuing the search of the LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be needed to accomplish this:
1) Athens-Cairo>> CairoLibrary would have to return a list of paths to all the matching libraries rather than just the first one that it finds. This part seems easy.
2) UFFI Libraries UnixDynamicLoader>> loadLibrary:flag: would have to take the list of paths (rather than just one), check them for being 32-bit or 64-bit, and load the first correct one. I can't find any way to perform that check in Pharo directly. How would you fork or exec the "file" command from inside Pharo?
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Dan Wilczak wrote:
Hernan -
I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
About continuing the search - I only mean continuing the search of the LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be needed to accomplish this:
FWIW, I had problem with loading 'sqlite3' module. Crossposting the solution: It seems FFI for some reason struggles with 'lib' and/or '.so.0' things in linux (even if LD_LIBRARY_PATH is properly set). I had to do this: TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname` ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3` ${TARGETDIR}/sqlite3.so (so, link in plugin directory, and the name is plain 'sqlite3.so'). With that, things work. Maybe, libsqlite.so would do the trick as well, but I got no nerve to play with it more. But, frankly, do not tell me this is what ppl need to do to load external libs in linux. :-/ Herby
1) Athens-Cairo>> CairoLibrary would have to return a list of paths to all the matching libraries rather than just the first one that it finds. This part seems easy.
2) UFFI Libraries UnixDynamicLoader>> loadLibrary:flag: would have to take the list of paths (rather than just one), check them for being 32-bit or 64-bit, and load the first correct one. I can't find any way to perform that check in Pharo directly. How would you fork or exec the "file" command from inside Pharo?
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Usually there are a number of names to try but on Linux, there are less users and more distros, so people are not adding more libs to try. So, yes, symlink in the plugin directory works usually. Alternative would be to use LD_LIBRARY_PATH or pkg_config support to do this cleanly (not done at this point). Phil On Sat, Sep 30, 2017 at 11:19 PM, Herby VojÄÃk <herby@mailbox.sk> wrote:
Dan Wilczak wrote:
Hernan -
I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
About continuing the search - I only mean continuing the search of the LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be needed to accomplish this:
FWIW, I had problem with loading 'sqlite3' module. Crossposting the solution:
It seems FFI for some reason struggles with 'lib' and/or '.so.0' things in linux (even if LD_LIBRARY_PATH is properly set).
I had to do this:
TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname` ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3` ${TARGETDIR}/sqlite3.so
(so, link in plugin directory, and the name is plain 'sqlite3.so'). With that, things work. Maybe, libsqlite.so would do the trick as well, but I got no nerve to play with it more.
But, frankly, do not tell me this is what ppl need to do to load external libs in linux. :-/
Herby
1) Athens-Cairo>> CairoLibrary would have to return a list of paths to all the matching libraries rather than just the first one that it finds. This part seems easy.
2) UFFI Libraries UnixDynamicLoader>> loadLibrary:flag: would have to take the list of paths (rather than just one), check them for being 32-bit or 64-bit, and load the first correct one. I can't find any way to perform that check in Pharo directly. How would you fork or exec the "file" command from inside Pharo?
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
On October 1, 2017 12:33:04 AM GMT+02:00, "phil@highoctane.be" <phil@highoctane.be> wrote:
Usually there are a number of names to try but on Linux, there are less users and more distros, so people are not adding more libs to try.
So, yes, symlink in the plugin directory works usually.
Alternative would be to use LD_LIBRARY_PATH or pkg_config support to do this cleanly (not done at this point).
I did use LD_LIBRARY_PATH. But it did not work (it looks like Pharo vm us using it, as it is used and decorated in pharo runner script). Even link in the plugin dir did not work unless I specified simplified name. The library is libsqlite3.so.0. Using that name, it wasn't found ('lib' is probably tried, so it is likely '.0' part that somehow blocked Pharo). Shouldn't this actually be solved somehow? So linux vm loads libs using linux conventions? Or is it preinstalled wrongly in Ubuntu images on Digital Ocean when it does not have symlink named libsqlite3.so ? Herby
Phil
On Sat, Sep 30, 2017 at 11:19 PM, Herby VojÄÃk <herby@mailbox.sk> wrote:
Dan Wilczak wrote:
Hernan -
I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
About continuing the search - I only mean continuing the search of the LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be needed to accomplish this:
FWIW, I had problem with loading 'sqlite3' module. Crossposting the solution:
It seems FFI for some reason struggles with 'lib' and/or '.so.0' things in linux (even if LD_LIBRARY_PATH is properly set).
I had to do this:
TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname` ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3` ${TARGETDIR}/sqlite3.so
(so, link in plugin directory, and the name is plain 'sqlite3.so'). With that, things work. Maybe, libsqlite.so would do the trick as well, but I got no nerve to play with it more.
But, frankly, do not tell me this is what ppl need to do to load external libs in linux. :-/
Herby
1) Athens-Cairo>> CairoLibrary would have to return a list of paths to all the matching libraries rather than just the first one that it finds. This part seems easy.
2) UFFI Libraries UnixDynamicLoader>> loadLibrary:flag: would have to take the list of paths (rather than just one), check them for being 32-bit or 64-bit, and load the first correct one. I can't find any way to perform that check in Pharo directly. How would you fork or exec the "file" command from inside Pharo?
Dan
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
One obstacle to fixing these problems properly is that handling of library search paths is not centralized, but it really should be. A Pharo package for sqlite or cairo or whatever should provide only the library name; UFFI or some associated package should deal with the search for it. Otherwise there will just be more ad hoc code and fixes in the future. -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
participants (8)
-
Dan Wilczak -
Esteban Lorenzano -
Herby VojÄÃk -
herby＠mailbox.sk
-
Hernán Morales Durand -
Hilaire -
phil＠highoctane.be -
Stephane Ducasse