Re: [Pharo-users] How to make pharo find sqlite?

Sept. 30, 2017

      phil@highoctane.be wrote:
...
I am using UDBCSQLite on Windows without problems.
Me, too; when developing.

The problem was on Linux, where I deploy.

Things begin to look as if it was really that the module is not found, 
though.

I added more diagnostic output, to ExternalFunction >> 
invokeWithArguments:, so it writes to transcript whenever the primitive 
fails, writing args used to call and self. The output is suddenly filled 
with output:

19:48:10.132 + ../pharo/pharo ./filmtower.image conf/run-tests.st
19:48:10.662
19:48:10.662 TowergameServerTests
19:48:10.781 4 run, 4 passes, 0 skipped, 0 expected failures, 0 
failures, 0 errors, 0 unexpected passes
19:48:10.781
19:48:10.781 TowergameSyncTests
19:48:10.781
19:48:10.782 TowergameSyncTests>>#testPlayerCanHaveDisabledDeviceSaved
19:48:10.782 ENTER setUp
19:48:10.804
19:48:10.805 an Array('' @ 16r08FF27C0)
19:48:10.806 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'>
19:48:10.806
19:48:10.806 ENTER tearDown
19:48:10.807
19:48:10.807 TowergameSyncTests>>#testPlayerChecksStateVersion
19:48:10.807 ENTER setUp
19:48:10.815
19:48:10.815 an Array('' @ 16r08FF1718)
19:48:10.816 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'>
19:48:10.816
19:48:10.816 ENTER tearDown
19:48:10.816
19:48:10.816 
TowergameSyncTests>>#testPlayerChecksStateVersionAndHasFreshlyInstalled
19:48:10.817 ENTER setUp
19:48:10.820
19:48:10.820 an Array('' @ 16r08FF2198)
19:48:10.820 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'>
19:48:10.820
19:48:10.820 ENTER tearDown
19:48:10.821
19:48:10.821 TowergameSyncTests>>#testPlayerChecksStateVersionAndIsBehind
19:48:10.821 ENTER setUp
19:48:10.829
19:48:10.829 an Array('' @ 16r08FFFAD8)
19:48:10.829 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'>
19:48:10.829
19:48:10.830 ENTER tearDown
19:48:10.830
19:48:10.830 
TowergameSyncTests>>#testPlayerChecksStateVersionFromDifferentDevice
19:48:10.830 ENTER setUp
19:48:10.832
19:48:10.832 an Array('' @ 16r08FFFAE8)
19:48:10.832 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'>
19:48:10.832
19:48:10.832 ENTER tearDown
19:48:10.832
19:48:10.832 
TowergameSyncTests>>#testPlayerChecksStateVersionFromDifferentDeviceAndHasFreshlyInstalled
19:48:10.832 ENTER setUp
19:48:10.839
19:48:10.839 an Array('' @ 16r08FFFAF8)
19:48:10.839 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'>
19:48:10.840
19:48:10.840 ENTER tearDown
19:48:10.840
19:48:10.840 
TowergameSyncTests>>#testPlayerChecksStateVersionFromDifferentExistingDevice
19:48:10.840 ENTER setUp
19:48:10.841
19:48:10.841 an Array('' @ 16r08FFFB08)
19:48:10.841 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'>
19:48:10.842
19:48:10.842 ENTER tearDown
19:48:10.842
19:48:10.842 
TowergameSyncTests>>#testPlayerChecksStateVersionFromDisabledDevice
19:48:10.842 ENTER setUp
19:48:10.854
19:48:10.854 an Array(@ 16r00000000)
19:48:10.855 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'>
19:48:10.855
19:48:10.855 an Array(@ 16r00000000)
19:48:10.855 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'>
19:48:10.856
19:48:10.856 an Array(@ 16r00000000)
19:48:10.856 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'>
19:48:10.856
19:48:10.856 an Array(@ 16r00000000)
19:48:10.856 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'>
19:48:10.857
19:48:10.857 an Array(@ 16r00000000)
19:48:10.857 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'>
19:48:10.857
19:48:10.858 an Array(@ 16r00000000)
19:48:10.861 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'>
19:48:10.861
19:48:10.862 an Array(@ 16r00000000)
19:48:10.863 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'>
19:48:10.864 Error: External module not found
19:48:10.865 ExternalLibraryFunction(Object)>>error:
etc.

So the mystery of "where are LEAVE messages" is solved: both setUp and 
tearDown failed between ENTER and LEAVE.

Now why cannot it find the library (now I am running it in both 32bit 
env as well as 64bit env, with appropriate vm installed; but always the 
same: sqlite3 module is not found).

At least it seems it is not mysterious vm bug, but (only) failure to 
find an external module. Though I don't know how to solve it, 
LD_LIBRARY_PATH did not help... :-/
...
Phil
On Sat, Sep 30, 2017 at 9:11 PM, Herby VojÄÃk <herby@mailbox.sk
<mailto:herby@mailbox.sk>> wrote:
phil@highoctane.be <mailto:phil@highoctane.be> wrote:
Also, did you try with this VM:
http://get.pharo.org/vmTLatest60 <http://get.pharo.org/vmTLatest60>
18:51:46.191 + curl get.pharo.org/vmTLatest60
    <http://get.pharo.org/vmTLatest60>
    18:51:46.207   % Total    % Received % Xferd  Average Speed   Time
    Time     Time  Current
    18:51:46.208                                  Dload  Upload   Total
    Spent    Left  Speed
    18:51:46.208
    18:51:46.242   0     0    0     0    0     0      0      0 --:--:--
    --:--:-- --:--:--     0
    18:51:46.242 100  6126  100  6126    0     0   172k      0 --:--:--
    --:--:-- --:--:--  175k
    18:51:46.253 Downloading the latest pharoVM:
    18:51:46.253
    http://files.pharo.org/get-files/60/pharo-linux-threaded-latest.zip
    <http://files.pharo.org/get-files/60/pharo-linux-threaded-latest.zip>
    18:51:46.305 [pharo-vm/vm.zip]
    18:51:46.305   End-of-central-directory signature not found.  Either
    this file is not
    18:51:46.305   a zipfile, or it constitutes one disk of a multi-part
    archive.  In the
    18:51:46.305   latter case the central directory and zipfile comment
    will be found on
    18:51:46.305   the last disk(s) of this archive.
    18:51:46.305 unzip:  cannot find zipfile directory in one of
    pharo-vm/vm.zip or
    18:51:46.305         pharo-vm/vm.zip.zip, and cannot find
    pharo-vm/vm.zip.ZIP, period.
It probably does not exist any more (I tried 70+vm and it failed in
    other aspects, it wasn't able to load git repo).
Tried both 61+vmT and 61+vmI both; in 32vm/32os and 64vm/64os
    combinations. Always ended with same result.
Must be some error in UDBCSQLiteLibrary itself. :-/
Although the missing transcript output is scary and shows that vm
    may be culprit as well.
Herby
Phil
On Sat, Sep 30, 2017 at 1:28 PM, Herby VojÄÃk <herby@mailbox.sk
        <mailto:herby@mailbox.sk>
        <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>> wrote:
phil@highoctane.be <mailto:phil@highoctane.be>
        <mailto:phil@highoctane.be <mailto:phil@highoctane.be>> wrote:
What about
LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH  ./pharo-ui
                 some.image
Phil
Thanks for answer, did not help.
In fact it must be something different. As can be seen in
        the stack,
             it fails during finalizers, and as can be seen by looking at
             UDBCSQLite3DatabaseExternalObject
        class>>finalizeResourceData: code,
             the method it calls is sqlite close. It is hardly the first
        method
             that is should call...
I suspect something around image save / load. Again. Lots
        of errors
             in those parts. But may be something else, as it kicks in
        only when
             SQLite-using tests starts to run. :-(
Herby
P.S.: I saw there is a similar thread out there, but it has
        problems
             with 32bit loaded by 64bit vm; but here, I have 32bit
        linux, so the
             vm installed should be 32bit.
On Thu, Sep 28, 2017 at 7:40 PM, Herby VojÄÃk
        <herby@mailbox.sk <mailto:herby@mailbox.sk>
        <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>
        <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>
        <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>>> wrote:
Hello!
I try to deploy UDBCSQLite-using image in a 32bit
        ubuntu
                 16.04.3.
I do have libsqlite3:
root@32bit-agent:~# find / -name '*libsqlite*' -type f
                 2>>/dev/null
                      /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
                      /var/lib/dpkg/info/libsqlite0.list
                      /var/lib/dpkg/info/libsqlite3-0:i386.postinst
                      /var/lib/dpkg/info/libsqlite3-0:i386.md5sums
                      /var/lib/dpkg/info/libsqlite3-0:i386.shlibs
                      /var/lib/dpkg/info/libsqlite0.postrm
                      /var/lib/dpkg/info/libsqlite3-0:i386.symbols
                      /var/lib/dpkg/info/libsqlite3-0:i386.list
                      /var/lib/dpkg/info/libsqlite3-0:i386.triggers
/var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb
but I get this in the output of the CI:
17:16:54.233 + ../pharo/pharo ./filmtower.image
                 conf/run-tests.st <http://run-tests.st>
        <http://run-tests.st>
        <http://run-tests.st>
                      17:16:54.508 pthread_setschedparam failed:
        Operation not
                 permitted
                      17:16:54.509 This VM uses a separate heartbeat
        thread to
                 update its
                      internal clock
                      17:16:54.509 and handle events.  For best
        operation, this
                 thread
                      should run at a
                      17:16:54.509 higher priority, however the VM was
        unable to
                 change
                      the priority.  The
                      17:16:54.509 effect is that heavily loaded systems may
                 experience
                      some latency
                      17:16:54.509 issues.  If this occurs, please
        create the
                 appropriate
                      configuration
                      17:16:54.509 file in /etc/security/limits.d/ as
        shown below:
                      17:16:54.509
                      17:16:54.509 cat <<END | sudo tee
                 /etc/security/limits.d/pharo.conf
                      17:16:54.509 *      hard    rtprio  2
                      17:16:54.509 *      soft    rtprio  2
                      17:16:54.509 END
                      17:16:54.509
                      17:16:54.509 and report to the pharo mailing list
        whether this
                      improves behaviour.
                      17:16:54.512
                      17:16:54.512 You will need to log out and log back
        in for
                 the limits
                      to take effect.
                      17:16:54.512 For more information please see
                      17:16:54.512
        https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>>
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>>>
                      17:16:54.785
                      17:16:54.786 TowergameSyncTests
                      17:16:54.831 Error: External module not found
                      17:16:54.832 ExternalLibraryFunction(Object)>>error:
                      17:16:54.832
                 ExternalLibraryFunction(Object)>>externalCallFailed
                      17:16:54.833
ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments:
                      17:16:54.833 UDBCSQLite3DatabaseExternalObject
                      class>>finalizeResourceData:
                      17:16:54.834 FFICalloutAPI>>function:module:
                      17:16:54.834
        UDBCSQLite3Library(Object)>>ffiCall:module:
                      17:16:54.835 UDBCSQLite3DatabaseExternalObject
                      class>>finalizeResourceData:
                      17:16:54.836 FFIExternalResourceExecutor>>finalize
                      17:16:54.836 WeakFinalizerItem>>finalizeValues
                      17:16:54.845 [ each finalizeValues ] in [ :each |
        [ each
                      finalizeValues ] on: Exception fork: [ :ex | ex
        pass ] ] in
                      WeakRegistry>>finalizeValues in Block: [ each
        finalizeValues ]
                      17:16:54.846 BlockClosure>>on:do:
                      17:16:54.852 [ Processor terminateActive ] in [ :ex |
                      17:16:54.852 | copy onDoCtx process handler bottom
        thisCtx |
                      17:16:54.852 onDoCtx := thisContext.
                      17:16:54.852 thisCtx := onDoCtx home.
                      17:16:54.852
                      17:16:54.852 "find the context on stack for which this
                 method's is
                      sender"
                      17:16:54.852 [ onDoCtx sender == thisCtx ]
                      17:16:54.852    whileFalse: [ onDoCtx := onDoCtx
        sender.
                      17:16:54.852            onDoCtx
                      17:16:54.852                    ifNil: [ "Can't
        find our home
                      context. seems like we're already forked
                      17:16:54.852                            and
        handling another
                      exception in new thread. In this case, just pass
        it through
                      handler." ^ handlerAction cull: ex ] ].
                      17:16:54.852 bottom := [ Processor terminateActive
        ] asContext.
                      17:16:54.853 onDoCtx privSender: bottom.
                      17:16:54.853 handler := [ handlerAction cull: ex ]
        asContext.
                      17:16:54.853 handler privSender: thisContext sender.
                      17:16:54.853 (Process forContext: handler
        priority: Processor
                      activePriority)
                      17:16:54.853    resume.
                      17:16:54.853
                      17:16:54.853 "cut the stack of current process"
                      17:16:54.853 thisContext privSender: thisCtx.
                      17:16:54.853 nil ] in BlockClosure>>on:fork: in
        Block: [
                 Processor
                      terminateActive ]
                      17:16:54.989
Look like pharo was not able to find the sqlite3 lib.
Any help?
Thanks, Herby

Re: [Pharo-users] How to make pharo find sqlite?

Herby VojÄÃ­k

Herby VojÄÃk