phil@highoctane.be wrote:

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH ./pharo-ui some.image

Phil

Thanks for answer, did not help. In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call... I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-( Herby P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk>> wrote:

Hello!

I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

I do have libsqlite3:

root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6 /var/lib/dpkg/info/libsqlite0.list /var/lib/dpkg/info/libsqlite3-0:i386.postinst /var/lib/dpkg/info/libsqlite3-0:i386.md5sums /var/lib/dpkg/info/libsqlite3-0:i386.shlibs /var/lib/dpkg/info/libsqlite0.postrm /var/lib/dpkg/info/libsqlite3-0:i386.symbols /var/lib/dpkg/info/libsqlite3-0:i386.list /var/lib/dpkg/info/libsqlite3-0:i386.triggers /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

but I get this in the output of the CI:

17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st <http://run-tests.st> 17:16:54.508 pthread_setschedparam failed: Operation not permitted 17:16:54.509 This VM uses a separate heartbeat thread to update its internal clock 17:16:54.509 and handle events. For best operation, this thread should run at a 17:16:54.509 higher priority, however the VM was unable to change the priority. The 17:16:54.509 effect is that heavily loaded systems may experience some latency 17:16:54.509 issues. If this occurs, please create the appropriate configuration 17:16:54.509 file in /etc/security/limits.d/ as shown below: 17:16:54.509 17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf 17:16:54.509 * hard rtprio 2 17:16:54.509 * soft rtprio 2 17:16:54.509 END 17:16:54.509 17:16:54.509 and report to the pharo mailing list whether this improves behaviour. 17:16:54.512 17:16:54.512 You will need to log out and log back in for the limits to take effect. 17:16:54.512 For more information please see 17:16:54.512 https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux> 17:16:54.785 17:16:54.786 TowergameSyncTests 17:16:54.831 Error: External module not found 17:16:54.832 ExternalLibraryFunction(Object)>>error: 17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed 17:16:54.833 ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments: 17:16:54.833 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.834 FFICalloutAPI>>function:module: 17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module: 17:16:54.835 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.836 FFIExternalResourceExecutor>>finalize 17:16:54.836 WeakFinalizerItem>>finalizeValues 17:16:54.845 [ each finalizeValues ] in [ :each | [ each finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in WeakRegistry>>finalizeValues in Block: [ each finalizeValues ] 17:16:54.846 BlockClosure>>on:do: 17:16:54.852 [ Processor terminateActive ] in [ :ex | 17:16:54.852 | copy onDoCtx process handler bottom thisCtx | 17:16:54.852 onDoCtx := thisContext. 17:16:54.852 thisCtx := onDoCtx home. 17:16:54.852 17:16:54.852 "find the context on stack for which this method's is sender" 17:16:54.852 [ onDoCtx sender == thisCtx ] 17:16:54.852 whileFalse: [ onDoCtx := onDoCtx sender. 17:16:54.852 onDoCtx 17:16:54.852 ifNil: [ "Can't find our home context. seems like we're already forked 17:16:54.852 and handling another exception in new thread. In this case, just pass it through handler." ^ handlerAction cull: ex ] ]. 17:16:54.852 bottom := [ Processor terminateActive ] asContext. 17:16:54.853 onDoCtx privSender: bottom. 17:16:54.853 handler := [ handlerAction cull: ex ] asContext. 17:16:54.853 handler privSender: thisContext sender. 17:16:54.853 (Process forContext: handler priority: Processor activePriority) 17:16:54.853 resume. 17:16:54.853 17:16:54.853 "cut the stack of current process" 17:16:54.853 thisContext privSender: thisCtx. 17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor terminateActive ] 17:16:54.989

Look like pharo was not able to find the sqlite3 lib.

Any help?

Thanks, Herby

phil@highoctane.be wrote:

Is https://pharo.fogbugz.com/f/cases/19990 showing again?

What is the module being loaded ?

It is taken from this message send: UDBCSQLite3Library >> library Smalltalk os isMacOS ifTrue: [ ^ #sqlite3 ]. ^ 'sqlite3' So, sqlite3.

Phil

On Sat, Sep 30, 2017 at 1:28 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk>> wrote:

phil@highoctane.be <mailto:phil@highoctane.be> wrote:

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk> <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>> wrote:

Hello!

I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

I do have libsqlite3:

root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6 /var/lib/dpkg/info/libsqlite0.list /var/lib/dpkg/info/libsqlite3-0:i386.postinst /var/lib/dpkg/info/libsqlite3-0:i386.md5sums /var/lib/dpkg/info/libsqlite3-0:i386.shlibs /var/lib/dpkg/info/libsqlite0.postrm /var/lib/dpkg/info/libsqlite3-0:i386.symbols /var/lib/dpkg/info/libsqlite3-0:i386.list /var/lib/dpkg/info/libsqlite3-0:i386.triggers /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

but I get this in the output of the CI:

17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st <http://run-tests.st> <http://run-tests.st> 17:16:54.508 pthread_setschedparam failed: Operation not permitted 17:16:54.509 This VM uses a separate heartbeat thread to update its internal clock 17:16:54.509 and handle events. For best operation, this thread should run at a 17:16:54.509 higher priority, however the VM was unable to change the priority. The 17:16:54.509 effect is that heavily loaded systems may experience some latency 17:16:54.509 issues. If this occurs, please create the appropriate configuration 17:16:54.509 file in /etc/security/limits.d/ as shown below: 17:16:54.509 17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf 17:16:54.509 * hard rtprio 2 17:16:54.509 * soft rtprio 2 17:16:54.509 END 17:16:54.509 17:16:54.509 and report to the pharo mailing list whether this improves behaviour. 17:16:54.512 17:16:54.512 You will need to log out and log back in for the limits to take effect. 17:16:54.512 For more information please see 17:16:54.512 https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux> <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>> 17:16:54.785 17:16:54.786 TowergameSyncTests 17:16:54.831 Error: External module not found 17:16:54.832 ExternalLibraryFunction(Object)>>error: 17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed 17:16:54.833 ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments: 17:16:54.833 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.834 FFICalloutAPI>>function:module: 17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module: 17:16:54.835 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.836 FFIExternalResourceExecutor>>finalize 17:16:54.836 WeakFinalizerItem>>finalizeValues 17:16:54.845 [ each finalizeValues ] in [ :each | [ each finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in WeakRegistry>>finalizeValues in Block: [ each finalizeValues ] 17:16:54.846 BlockClosure>>on:do: 17:16:54.852 [ Processor terminateActive ] in [ :ex | 17:16:54.852 | copy onDoCtx process handler bottom thisCtx | 17:16:54.852 onDoCtx := thisContext. 17:16:54.852 thisCtx := onDoCtx home. 17:16:54.852 17:16:54.852 "find the context on stack for which this method's is sender" 17:16:54.852 [ onDoCtx sender == thisCtx ] 17:16:54.852 whileFalse: [ onDoCtx := onDoCtx sender. 17:16:54.852 onDoCtx 17:16:54.852 ifNil: [ "Can't find our home context. seems like we're already forked 17:16:54.852 and handling another exception in new thread. In this case, just pass it through handler." ^ handlerAction cull: ex ] ]. 17:16:54.852 bottom := [ Processor terminateActive ] asContext. 17:16:54.853 onDoCtx privSender: bottom. 17:16:54.853 handler := [ handlerAction cull: ex ] asContext. 17:16:54.853 handler privSender: thisContext sender. 17:16:54.853 (Process forContext: handler priority: Processor activePriority) 17:16:54.853 resume. 17:16:54.853 17:16:54.853 "cut the stack of current process" 17:16:54.853 thisContext privSender: thisCtx. 17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor terminateActive ] 17:16:54.989

Look like pharo was not able to find the sqlite3 lib.

Any help?

Thanks, Herby

phil@highoctane.be wrote:

Is https://pharo.fogbugz.com/f/cases/19990 showing again?

What is the module being loaded ?

This seems to be the important question. It seems FFI for some reason struggles with 'lib' and/or '.so.0' things in linux (even if LD_LIBRARY_PATH is properly set). I had to do this: TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname` ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3` ${TARGETDIR}/sqlite3.so (so, link in plugin directory, and the name is plain 'sqlite3.so'). With that, things work. Maybe, libsqlite.so would do the trick as well, but I got no nerve to play with it more. But, frankly, do not tell me this is what ppl need to do to load external libs in linux. :-/ Herby

Phil

On Sat, Sep 30, 2017 at 1:28 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk>> wrote:

phil@highoctane.be <mailto:phil@highoctane.be> wrote:

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk> <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>> wrote:

Hello!

I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

I do have libsqlite3:

root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6 /var/lib/dpkg/info/libsqlite0.list /var/lib/dpkg/info/libsqlite3-0:i386.postinst /var/lib/dpkg/info/libsqlite3-0:i386.md5sums /var/lib/dpkg/info/libsqlite3-0:i386.shlibs /var/lib/dpkg/info/libsqlite0.postrm /var/lib/dpkg/info/libsqlite3-0:i386.symbols /var/lib/dpkg/info/libsqlite3-0:i386.list /var/lib/dpkg/info/libsqlite3-0:i386.triggers /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

but I get this in the output of the CI:

17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st <http://run-tests.st> <http://run-tests.st> 17:16:54.508 pthread_setschedparam failed: Operation not permitted 17:16:54.509 This VM uses a separate heartbeat thread to update its internal clock 17:16:54.509 and handle events. For best operation, this thread should run at a 17:16:54.509 higher priority, however the VM was unable to change the priority. The 17:16:54.509 effect is that heavily loaded systems may experience some latency 17:16:54.509 issues. If this occurs, please create the appropriate configuration 17:16:54.509 file in /etc/security/limits.d/ as shown below: 17:16:54.509 17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf 17:16:54.509 * hard rtprio 2 17:16:54.509 * soft rtprio 2 17:16:54.509 END 17:16:54.509 17:16:54.509 and report to the pharo mailing list whether this improves behaviour. 17:16:54.512 17:16:54.512 You will need to log out and log back in for the limits to take effect. 17:16:54.512 For more information please see 17:16:54.512 https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux> <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>> 17:16:54.785 17:16:54.786 TowergameSyncTests 17:16:54.831 Error: External module not found 17:16:54.832 ExternalLibraryFunction(Object)>>error: 17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed 17:16:54.833 ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments: 17:16:54.833 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.834 FFICalloutAPI>>function:module: 17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module: 17:16:54.835 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.836 FFIExternalResourceExecutor>>finalize 17:16:54.836 WeakFinalizerItem>>finalizeValues 17:16:54.845 [ each finalizeValues ] in [ :each | [ each finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in WeakRegistry>>finalizeValues in Block: [ each finalizeValues ] 17:16:54.846 BlockClosure>>on:do: 17:16:54.852 [ Processor terminateActive ] in [ :ex | 17:16:54.852 | copy onDoCtx process handler bottom thisCtx | 17:16:54.852 onDoCtx := thisContext. 17:16:54.852 thisCtx := onDoCtx home. 17:16:54.852 17:16:54.852 "find the context on stack for which this method's is sender" 17:16:54.852 [ onDoCtx sender == thisCtx ] 17:16:54.852 whileFalse: [ onDoCtx := onDoCtx sender. 17:16:54.852 onDoCtx 17:16:54.852 ifNil: [ "Can't find our home context. seems like we're already forked 17:16:54.852 and handling another exception in new thread. In this case, just pass it through handler." ^ handlerAction cull: ex ] ]. 17:16:54.852 bottom := [ Processor terminateActive ] asContext. 17:16:54.853 onDoCtx privSender: bottom. 17:16:54.853 handler := [ handlerAction cull: ex ] asContext. 17:16:54.853 handler privSender: thisContext sender. 17:16:54.853 (Process forContext: handler priority: Processor activePriority) 17:16:54.853 resume. 17:16:54.853 17:16:54.853 "cut the stack of current process" 17:16:54.853 thisContext privSender: thisCtx. 17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor terminateActive ] 17:16:54.989

Look like pharo was not able to find the sqlite3 lib.

Any help?

Thanks, Herby

Thanks Renaud! Yes it helps! On Mon, Oct 2, 2017 at 2:57 AM, Renaud de Villemeur <renaud.devillemeur@free.fr> wrote:

Hi

I managed to get Sqlite work with Pharo under linux (Fedora), in both 32 and 64 bit. The setup has nothing to do with LD_LIBRARY_PATH. I'll use those example under fedora linux.

First, of course, you need to install either UDBCSqlite or Garage-Sqlite drivers

You first need to have sqlite3 libs installed on your system $ rpm -qa | grep sqlite sqlite-3.20.1-1.fc26.x86_64 sqlite-libs-3.20.1-1.fc26.x86_64 => for 64 bits version

sqlite-3.20.1-1.fc26.i686 sqlite-libs-3.20.1-1.fc26.i686 => for 32 bits version

Then find the path of your library

$ rpm -ql sqlite-libs.i686 /usr/lib/libsqlite3.so.0 -> this is your path in 32 bits /usr/lib/libsqlite3.so.0.8.6 /usr/share/doc/sqlite-libs /usr/share/doc/sqlite-libs/README.md gnome@rdv:~ $ rpm -ql sqlite-libs.x86_64 /usr/lib64/libsqlite3.so.0 -> this is your path in 64 bits /usr/lib64/libsqlite3.so.0.8.6 /usr/share/doc/sqlite-libs /usr/share/doc/sqlite-libs/README.md

Under Linux, you have to give the library path inside Pharo Image, or link the library from pharo VM folder:

1. Update library path from pharo image

Using UDBDSQLite, update to return the path of the library on your system

UDBCSQLite3Library >> library

Smalltalk os isMacOS ifTrue: [ ^ #sqlite3 ].

^ 'sqlite3'

to

UDBCSQLite3Library >> library

Smalltalk os isMacOS ifTrue: [ ^ #sqlite3 ].

^ '/usr/lib/libsqlite3.so.0'

If you are using Garage,Update:

GASqlite3FFI >> library

^ '/usr/lib/libsqlite3.so.0'

2. link the library from VM folder:

ln -s /usr/lib/libsqlite3.so.0 libsqlite3 or ln -s /usr/lib64/libsqlite3.so.0 sqlite3 if the module name is sqlite3 in your library method.

Then your test should pass green.

The reason it pass under windows is because the method library return by default sqlite3, which is the dll name you put under pharo VM directory to get it work. On linux, unless you link your library in the VM folder, the image has no clue where to find sqlite3.

Hope this helps. Renaud

2017-09-30 17:16 GMT-04:00 Herby Vojčík <herby@mailbox.sk>:

...

phil@highoctane.be wrote:

...

Is https://pharo.fogbugz.com/f/cases/19990 showing again?

What is the module being loaded ?

This seems to be the important question. It seems FFI for some reason struggles with 'lib' and/or '.so.0' things in linux (even if LD_LIBRARY_PATH is properly set).

I had to do this:

TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname` ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3` ${TARGETDIR}/sqlite3.so

(so, link in plugin directory, and the name is plain 'sqlite3.so'). With that, things work. Maybe, libsqlite.so would do the trick as well, but I got no nerve to play with it more.

But, frankly, do not tell me this is what ppl need to do to load external libs in linux. :-/

Herby

...

Phil

On Sat, Sep 30, 2017 at 1:28 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk>> wrote:

phil@highoctane.be <mailto:phil@highoctane.be> wrote:

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk> <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>> wrote:

Hello!

I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

I do have libsqlite3:

root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6 /var/lib/dpkg/info/libsqlite0.list /var/lib/dpkg/info/libsqlite3-0:i386.postinst /var/lib/dpkg/info/libsqlite3-0:i386.md5sums /var/lib/dpkg/info/libsqlite3-0:i386.shlibs /var/lib/dpkg/info/libsqlite0.postrm /var/lib/dpkg/info/libsqlite3-0:i386.symbols /var/lib/dpkg/info/libsqlite3-0:i386.list /var/lib/dpkg/info/libsqlite3-0:i386.triggers

/var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

but I get this in the output of the CI:

17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st <http://run-tests.st> <http://run-tests.st> 17:16:54.508 pthread_setschedparam failed: Operation not permitted 17:16:54.509 This VM uses a separate heartbeat thread to update its internal clock 17:16:54.509 and handle events. For best operation, this thread should run at a 17:16:54.509 higher priority, however the VM was unable to change the priority. The 17:16:54.509 effect is that heavily loaded systems may experience some latency 17:16:54.509 issues. If this occurs, please create the appropriate configuration 17:16:54.509 file in /etc/security/limits.d/ as shown below: 17:16:54.509 17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf 17:16:54.509 * hard rtprio 2 17:16:54.509 * soft rtprio 2 17:16:54.509 END 17:16:54.509 17:16:54.509 and report to the pharo mailing list whether this improves behaviour. 17:16:54.512 17:16:54.512 You will need to log out and log back in for the limits to take effect. 17:16:54.512 For more information please see 17:16:54.512

https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>> 17:16:54.785 17:16:54.786 TowergameSyncTests 17:16:54.831 Error: External module not found 17:16:54.832 ExternalLibraryFunction(Object)>>error: 17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed 17:16:54.833

ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments: 17:16:54.833 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.834 FFICalloutAPI>>function:module: 17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module: 17:16:54.835 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.836 FFIExternalResourceExecutor>>finalize 17:16:54.836 WeakFinalizerItem>>finalizeValues 17:16:54.845 [ each finalizeValues ] in [ :each | [ each finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in WeakRegistry>>finalizeValues in Block: [ each finalizeValues ] 17:16:54.846 BlockClosure>>on:do: 17:16:54.852 [ Processor terminateActive ] in [ :ex | 17:16:54.852 | copy onDoCtx process handler bottom thisCtx | 17:16:54.852 onDoCtx := thisContext. 17:16:54.852 thisCtx := onDoCtx home. 17:16:54.852 17:16:54.852 "find the context on stack for which this method's is sender" 17:16:54.852 [ onDoCtx sender == thisCtx ] 17:16:54.852 whileFalse: [ onDoCtx := onDoCtx sender. 17:16:54.852 onDoCtx 17:16:54.852 ifNil: [ "Can't find our home context. seems like we're already forked 17:16:54.852 and handling another exception in new thread. In this case, just pass it through handler." ^ handlerAction cull: ex ] ]. 17:16:54.852 bottom := [ Processor terminateActive ] asContext. 17:16:54.853 onDoCtx privSender: bottom. 17:16:54.853 handler := [ handlerAction cull: ex ] asContext. 17:16:54.853 handler privSender: thisContext sender. 17:16:54.853 (Process forContext: handler priority: Processor activePriority) 17:16:54.853 resume. 17:16:54.853 17:16:54.853 "cut the stack of current process" 17:16:54.853 thisContext privSender: thisCtx. 17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor terminateActive ] 17:16:54.989

Look like pharo was not able to find the sqlite3 lib.

Any help?

Thanks, Herby

Yes, all of this should work and we need to improve on this. I am willing to do something about that because it frustrates me too. Herby, How would you see it working? Phil On Mon, Oct 2, 2017 at 1:45 PM, Herby Vojčík <herby@mailbox.sk> wrote:

Renaud de Villemeur wrote:

...

Hi

[...snip...]

The reason it pass under windows is because the method library return by

...

default sqlite3, which is the dll name you put under pharo VM directory to get it work.

Not true. It is not in pharo vm directory. It finds it on %PATH%.

On linux, unless you link your library in the VM folder, the image has

...

no clue where to find sqlite3.

And this should be fixed, IMNSHO. It probably _partly_ works, but it should be made to work better.

Herby

Hope this helps.

...

Renaud

.

Also, did you try with this VM: http://get.pharo.org/vmTLatest60 Phil On Sat, Sep 30, 2017 at 1:28 PM, Herby Vojčík <herby@mailbox.sk> wrote:

phil@highoctane.be wrote:

...

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <herby@mailbox.sk

...

<mailto:herby@mailbox.sk>> wrote:

Hello!

I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

I do have libsqlite3:

root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6 /var/lib/dpkg/info/libsqlite0.list /var/lib/dpkg/info/libsqlite3-0:i386.postinst /var/lib/dpkg/info/libsqlite3-0:i386.md5sums /var/lib/dpkg/info/libsqlite3-0:i386.shlibs /var/lib/dpkg/info/libsqlite0.postrm /var/lib/dpkg/info/libsqlite3-0:i386.symbols /var/lib/dpkg/info/libsqlite3-0:i386.list /var/lib/dpkg/info/libsqlite3-0:i386.triggers /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

but I get this in the output of the CI:

17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st <http://run-tests.st> 17:16:54.508 pthread_setschedparam failed: Operation not permitted 17:16:54.509 This VM uses a separate heartbeat thread to update its internal clock 17:16:54.509 and handle events. For best operation, this thread should run at a 17:16:54.509 higher priority, however the VM was unable to change the priority. The 17:16:54.509 effect is that heavily loaded systems may experience some latency 17:16:54.509 issues. If this occurs, please create the appropriate configuration 17:16:54.509 file in /etc/security/limits.d/ as shown below: 17:16:54.509 17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf 17:16:54.509 * hard rtprio 2 17:16:54.509 * soft rtprio 2 17:16:54.509 END 17:16:54.509 17:16:54.509 and report to the pharo mailing list whether this improves behaviour. 17:16:54.512 17:16:54.512 You will need to log out and log back in for the limits to take effect. 17:16:54.512 For more information please see 17:16:54.512 https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/ tag/r3732#linux <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/ tag/r3732#linux> 17:16:54.785 17:16:54.786 TowergameSyncTests 17:16:54.831 Error: External module not found 17:16:54.832 ExternalLibraryFunction(Object)>>error: 17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed 17:16:54.833 ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments: 17:16:54.833 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.834 FFICalloutAPI>>function:module: 17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module: 17:16:54.835 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.836 FFIExternalResourceExecutor>>finalize 17:16:54.836 WeakFinalizerItem>>finalizeValues 17:16:54.845 [ each finalizeValues ] in [ :each | [ each finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in WeakRegistry>>finalizeValues in Block: [ each finalizeValues ] 17:16:54.846 BlockClosure>>on:do: 17:16:54.852 [ Processor terminateActive ] in [ :ex | 17:16:54.852 | copy onDoCtx process handler bottom thisCtx | 17:16:54.852 onDoCtx := thisContext. 17:16:54.852 thisCtx := onDoCtx home. 17:16:54.852 17:16:54.852 "find the context on stack for which this method's is sender" 17:16:54.852 [ onDoCtx sender == thisCtx ] 17:16:54.852 whileFalse: [ onDoCtx := onDoCtx sender. 17:16:54.852 onDoCtx 17:16:54.852 ifNil: [ "Can't find our home context. seems like we're already forked 17:16:54.852 and handling another exception in new thread. In this case, just pass it through handler." ^ handlerAction cull: ex ] ]. 17:16:54.852 bottom := [ Processor terminateActive ] asContext. 17:16:54.853 onDoCtx privSender: bottom. 17:16:54.853 handler := [ handlerAction cull: ex ] asContext. 17:16:54.853 handler privSender: thisContext sender. 17:16:54.853 (Process forContext: handler priority: Processor activePriority) 17:16:54.853 resume. 17:16:54.853 17:16:54.853 "cut the stack of current process" 17:16:54.853 thisContext privSender: thisCtx. 17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor terminateActive ] 17:16:54.989

Look like pharo was not able to find the sqlite3 lib.

Any help?

Thanks, Herby

I am using UDBCSQLite on Windows without problems. Phil On Sat, Sep 30, 2017 at 9:11 PM, Herby Vojčík <herby@mailbox.sk> wrote:

phil@highoctane.be wrote:

...

Also, did you try with this VM:

http://get.pharo.org/vmTLatest60

18:51:46.191 + curl get.pharo.org/vmTLatest60 18:51:46.207 % Total % Received % Xferd Average Speed Time Time Time Current 18:51:46.208 Dload Upload Total Spent Left Speed 18:51:46.208 18:51:46.242 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 18:51:46.242 100 6126 100 6126 0 0 172k 0 --:--:-- --:--:-- --:--:-- 175k 18:51:46.253 Downloading the latest pharoVM: 18:51:46.253 http://files.pharo.org/get-files/60/pharo-linux-threaded-lat est.zip 18:51:46.305 [pharo-vm/vm.zip] 18:51:46.305 End-of-central-directory signature not found. Either this file is not 18:51:46.305 a zipfile, or it constitutes one disk of a multi-part archive. In the 18:51:46.305 latter case the central directory and zipfile comment will be found on 18:51:46.305 the last disk(s) of this archive. 18:51:46.305 unzip: cannot find zipfile directory in one of pharo-vm/vm.zip or 18:51:46.305 pharo-vm/vm.zip.zip, and cannot find pharo-vm/vm.zip.ZIP, period.

It probably does not exist any more (I tried 70+vm and it failed in other aspects, it wasn't able to load git repo).

Tried both 61+vmT and 61+vmI both; in 32vm/32os and 64vm/64os combinations. Always ended with same result.

Must be some error in UDBCSQLiteLibrary itself. :-/

Although the missing transcript output is scary and shows that vm may be culprit as well.

Herby

Phil

...

On Sat, Sep 30, 2017 at 1:28 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk>> wrote:

phil@highoctane.be <mailto:phil@highoctane.be> wrote:

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk> <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>> wrote:

Hello!

I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

I do have libsqlite3:

root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6 /var/lib/dpkg/info/libsqlite0.list /var/lib/dpkg/info/libsqlite3-0:i386.postinst /var/lib/dpkg/info/libsqlite3-0:i386.md5sums /var/lib/dpkg/info/libsqlite3-0:i386.shlibs /var/lib/dpkg/info/libsqlite0.postrm /var/lib/dpkg/info/libsqlite3-0:i386.symbols /var/lib/dpkg/info/libsqlite3-0:i386.list /var/lib/dpkg/info/libsqlite3-0:i386.triggers /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386. deb

but I get this in the output of the CI:

17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st <http://run-tests.st> <http://run-tests.st> 17:16:54.508 pthread_setschedparam failed: Operation not permitted 17:16:54.509 This VM uses a separate heartbeat thread to update its internal clock 17:16:54.509 and handle events. For best operation, this thread should run at a 17:16:54.509 higher priority, however the VM was unable to change the priority. The 17:16:54.509 effect is that heavily loaded systems may experience some latency 17:16:54.509 issues. If this occurs, please create the appropriate configuration 17:16:54.509 file in /etc/security/limits.d/ as shown below: 17:16:54.509 17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf 17:16:54.509 * hard rtprio 2 17:16:54.509 * soft rtprio 2 17:16:54.509 END 17:16:54.509 17:16:54.509 and report to the pharo mailing list whether this improves behaviour. 17:16:54.512 17:16:54.512 You will need to log out and log back in for the limits to take effect. 17:16:54.512 For more information please see 17:16:54.512 https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/ tag/r3732#linux <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/ tag/r3732#linux> <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/ tag/r3732#linux <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/ tag/r3732#linux>> 17:16:54.785 17:16:54.786 TowergameSyncTests 17:16:54.831 Error: External module not found 17:16:54.832 ExternalLibraryFunction(Object)>>error: 17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed 17:16:54.833 ExternalLibraryFunction(ExternalFunction)>>invokeWithArgume nts: 17:16:54.833 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.834 FFICalloutAPI>>function:module: 17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module: 17:16:54.835 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.836 FFIExternalResourceExecutor>>finalize 17:16:54.836 WeakFinalizerItem>>finalizeValues 17:16:54.845 [ each finalizeValues ] in [ :each | [ each finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in WeakRegistry>>finalizeValues in Block: [ each finalizeValues ] 17:16:54.846 BlockClosure>>on:do: 17:16:54.852 [ Processor terminateActive ] in [ :ex | 17:16:54.852 | copy onDoCtx process handler bottom thisCtx | 17:16:54.852 onDoCtx := thisContext. 17:16:54.852 thisCtx := onDoCtx home. 17:16:54.852 17:16:54.852 "find the context on stack for which this method's is sender" 17:16:54.852 [ onDoCtx sender == thisCtx ] 17:16:54.852 whileFalse: [ onDoCtx := onDoCtx sender. 17:16:54.852 onDoCtx 17:16:54.852 ifNil: [ "Can't find our home context. seems like we're already forked 17:16:54.852 and handling another exception in new thread. In this case, just pass it through handler." ^ handlerAction cull: ex ] ]. 17:16:54.852 bottom := [ Processor terminateActive ] asContext. 17:16:54.853 onDoCtx privSender: bottom. 17:16:54.853 handler := [ handlerAction cull: ex ] asContext. 17:16:54.853 handler privSender: thisContext sender. 17:16:54.853 (Process forContext: handler priority: Processor activePriority) 17:16:54.853 resume. 17:16:54.853 17:16:54.853 "cut the stack of current process" 17:16:54.853 thisContext privSender: thisCtx. 17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor terminateActive ] 17:16:54.989

Look like pharo was not able to find the sqlite3 lib.

Any help?

Thanks, Herby

Herby Vojčík wrote:

phil@highoctane.be wrote:

...

I am using UDBCSQLite on Windows without problems.

Me, too; when developing.

The problem was on Linux, where I deploy.

Things begin to look as if it was really that the module is not found, though.

Now that I saw into the code CairoLibrary does to try to find the file name, and compose the module name from that, and that in my case the library name used is the plain 'sqlite3', I don't know - am I supposed to make a symlink on the same directory as the image? Is it the thing that is normally needed / done routinely? I may try that... (but on Windows, it just finds the sqlite dll without problems). Herby

I added more diagnostic output, to ExternalFunction >> invokeWithArguments:, so it writes to transcript whenever the primitive fails, writing args used to call and self. The output is suddenly filled with output:

19:48:10.132 + ../pharo/pharo ./filmtower.image conf/run-tests.st 19:48:10.662 19:48:10.662 TowergameServerTests 19:48:10.781 4 run, 4 passes, 0 skipped, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes 19:48:10.781 19:48:10.781 TowergameSyncTests 19:48:10.781 19:48:10.782 TowergameSyncTests>>#testPlayerCanHaveDisabledDeviceSaved 19:48:10.782 ENTER setUp 19:48:10.804 19:48:10.805 an Array('' @ 16r08FF27C0) 19:48:10.806 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'> 19:48:10.806 19:48:10.806 ENTER tearDown 19:48:10.807 19:48:10.807 TowergameSyncTests>>#testPlayerChecksStateVersion 19:48:10.807 ENTER setUp 19:48:10.815 19:48:10.815 an Array('' @ 16r08FF1718) 19:48:10.816 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'> 19:48:10.816 19:48:10.816 ENTER tearDown 19:48:10.816 19:48:10.816 TowergameSyncTests>>#testPlayerChecksStateVersionAndHasFreshlyInstalled 19:48:10.817 ENTER setUp 19:48:10.820 19:48:10.820 an Array('' @ 16r08FF2198) 19:48:10.820 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'> 19:48:10.820 19:48:10.820 ENTER tearDown 19:48:10.821 19:48:10.821 TowergameSyncTests>>#testPlayerChecksStateVersionAndIsBehind 19:48:10.821 ENTER setUp 19:48:10.829 19:48:10.829 an Array('' @ 16r08FFFAD8) 19:48:10.829 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'> 19:48:10.829 19:48:10.830 ENTER tearDown 19:48:10.830 19:48:10.830 TowergameSyncTests>>#testPlayerChecksStateVersionFromDifferentDevice 19:48:10.830 ENTER setUp 19:48:10.832 19:48:10.832 an Array('' @ 16r08FFFAE8) 19:48:10.832 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'> 19:48:10.832 19:48:10.832 ENTER tearDown 19:48:10.832 19:48:10.832 TowergameSyncTests>>#testPlayerChecksStateVersionFromDifferentDeviceAndHasFreshlyInstalled

19:48:10.832 ENTER setUp 19:48:10.839 19:48:10.839 an Array('' @ 16r08FFFAF8) 19:48:10.839 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'> 19:48:10.840 19:48:10.840 ENTER tearDown 19:48:10.840 19:48:10.840 TowergameSyncTests>>#testPlayerChecksStateVersionFromDifferentExistingDevice

19:48:10.840 ENTER setUp 19:48:10.841 19:48:10.841 an Array('' @ 16r08FFFB08) 19:48:10.841 <cdecl: long 'sqlite3_open' (char* void*) module: 'sqlite3'> 19:48:10.842 19:48:10.842 ENTER tearDown 19:48:10.842 19:48:10.842 TowergameSyncTests>>#testPlayerChecksStateVersionFromDisabledDevice 19:48:10.842 ENTER setUp 19:48:10.854 19:48:10.854 an Array(@ 16r00000000) 19:48:10.855 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'> 19:48:10.855 19:48:10.855 an Array(@ 16r00000000) 19:48:10.855 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'> 19:48:10.856 19:48:10.856 an Array(@ 16r00000000) 19:48:10.856 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'> 19:48:10.856 19:48:10.856 an Array(@ 16r00000000) 19:48:10.856 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'> 19:48:10.857 19:48:10.857 an Array(@ 16r00000000) 19:48:10.857 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'> 19:48:10.857 19:48:10.858 an Array(@ 16r00000000) 19:48:10.861 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'> 19:48:10.861 19:48:10.862 an Array(@ 16r00000000) 19:48:10.863 <cdecl: long 'sqlite3_close' (void*) module: 'sqlite3'> 19:48:10.864 Error: External module not found 19:48:10.865 ExternalLibraryFunction(Object)>>error: etc.

So the mystery of "where are LEAVE messages" is solved: both setUp and tearDown failed between ENTER and LEAVE.

Now why cannot it find the library (now I am running it in both 32bit env as well as 64bit env, with appropriate vm installed; but always the same: sqlite3 module is not found).

At least it seems it is not mysterious vm bug, but (only) failure to find an external module. Though I don't know how to solve it, LD_LIBRARY_PATH did not help... :-/

...

Phil

On Sat, Sep 30, 2017 at 9:11 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk>> wrote:

phil@highoctane.be <mailto:phil@highoctane.be> wrote:

Also, did you try with this VM:

http://get.pharo.org/vmTLatest60 <http://get.pharo.org/vmTLatest60>

18:51:46.191 + curl get.pharo.org/vmTLatest60 <http://get.pharo.org/vmTLatest60> 18:51:46.207 % Total % Received % Xferd Average Speed Time Time Time Current 18:51:46.208 Dload Upload Total Spent Left Speed 18:51:46.208 18:51:46.242 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 18:51:46.242 100 6126 100 6126 0 0 172k 0 --:--:-- --:--:-- --:--:-- 175k 18:51:46.253 Downloading the latest pharoVM: 18:51:46.253 http://files.pharo.org/get-files/60/pharo-linux-threaded-latest.zip <http://files.pharo.org/get-files/60/pharo-linux-threaded-latest.zip> 18:51:46.305 [pharo-vm/vm.zip] 18:51:46.305 End-of-central-directory signature not found. Either this file is not 18:51:46.305 a zipfile, or it constitutes one disk of a multi-part archive. In the 18:51:46.305 latter case the central directory and zipfile comment will be found on 18:51:46.305 the last disk(s) of this archive. 18:51:46.305 unzip: cannot find zipfile directory in one of pharo-vm/vm.zip or 18:51:46.305 pharo-vm/vm.zip.zip, and cannot find pharo-vm/vm.zip.ZIP, period.

It probably does not exist any more (I tried 70+vm and it failed in other aspects, it wasn't able to load git repo).

Tried both 61+vmT and 61+vmI both; in 32vm/32os and 64vm/64os combinations. Always ended with same result.

Must be some error in UDBCSQLiteLibrary itself. :-/

Although the missing transcript output is scary and shows that vm may be culprit as well.

Herby

Phil

On Sat, Sep 30, 2017 at 1:28 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk> <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>> wrote:

phil@highoctane.be <mailto:phil@highoctane.be> <mailto:phil@highoctane.be <mailto:phil@highoctane.be>> wrote:

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <herby@mailbox.sk <mailto:herby@mailbox.sk> <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>> <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk> <mailto:herby@mailbox.sk <mailto:herby@mailbox.sk>>>> wrote:

Hello!

I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

I do have libsqlite3:

root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6 /var/lib/dpkg/info/libsqlite0.list /var/lib/dpkg/info/libsqlite3-0:i386.postinst /var/lib/dpkg/info/libsqlite3-0:i386.md5sums /var/lib/dpkg/info/libsqlite3-0:i386.shlibs /var/lib/dpkg/info/libsqlite0.postrm /var/lib/dpkg/info/libsqlite3-0:i386.symbols /var/lib/dpkg/info/libsqlite3-0:i386.list /var/lib/dpkg/info/libsqlite3-0:i386.triggers

/var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

but I get this in the output of the CI:

17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st <http://run-tests.st> <http://run-tests.st> <http://run-tests.st> 17:16:54.508 pthread_setschedparam failed: Operation not permitted 17:16:54.509 This VM uses a separate heartbeat thread to update its internal clock 17:16:54.509 and handle events. For best operation, this thread should run at a 17:16:54.509 higher priority, however the VM was unable to change the priority. The 17:16:54.509 effect is that heavily loaded systems may experience some latency 17:16:54.509 issues. If this occurs, please create the appropriate configuration 17:16:54.509 file in /etc/security/limits.d/ as shown below: 17:16:54.509 17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf 17:16:54.509 * hard rtprio 2 17:16:54.509 * soft rtprio 2 17:16:54.509 END 17:16:54.509 17:16:54.509 and report to the pharo mailing list whether this improves behaviour. 17:16:54.512 17:16:54.512 You will need to log out and log back in for the limits to take effect. 17:16:54.512 For more information please see 17:16:54.512 https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>>

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux

<https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>>>

17:16:54.785 17:16:54.786 TowergameSyncTests 17:16:54.831 Error: External module not found 17:16:54.832 ExternalLibraryFunction(Object)>>error: 17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed 17:16:54.833

ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments: 17:16:54.833 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.834 FFICalloutAPI>>function:module: 17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module: 17:16:54.835 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: 17:16:54.836 FFIExternalResourceExecutor>>finalize 17:16:54.836 WeakFinalizerItem>>finalizeValues 17:16:54.845 [ each finalizeValues ] in [ :each | [ each finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in WeakRegistry>>finalizeValues in Block: [ each finalizeValues ] 17:16:54.846 BlockClosure>>on:do: 17:16:54.852 [ Processor terminateActive ] in [ :ex | 17:16:54.852 | copy onDoCtx process handler bottom thisCtx | 17:16:54.852 onDoCtx := thisContext. 17:16:54.852 thisCtx := onDoCtx home. 17:16:54.852 17:16:54.852 "find the context on stack for which this method's is sender" 17:16:54.852 [ onDoCtx sender == thisCtx ] 17:16:54.852 whileFalse: [ onDoCtx := onDoCtx sender. 17:16:54.852 onDoCtx 17:16:54.852 ifNil: [ "Can't find our home context. seems like we're already forked 17:16:54.852 and handling another exception in new thread. In this case, just pass it through handler." ^ handlerAction cull: ex ] ]. 17:16:54.852 bottom := [ Processor terminateActive ] asContext. 17:16:54.853 onDoCtx privSender: bottom. 17:16:54.853 handler := [ handlerAction cull: ex ] asContext. 17:16:54.853 handler privSender: thisContext sender. 17:16:54.853 (Process forContext: handler priority: Processor activePriority) 17:16:54.853 resume. 17:16:54.853 17:16:54.853 "cut the stack of current process" 17:16:54.853 thisContext privSender: thisCtx. 17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor terminateActive ] 17:16:54.989

Look like pharo was not able to find the sqlite3 lib.

Any help?

Thanks, Herby

Nice. We should make a booklet of all this. There is also the Ansible playbook we have and the Docker story as well. Phil On Tue, Oct 3, 2017 at 3:12 AM, Pierce Ng <pierce@samadhiweb.com> wrote:

On Sat, Sep 30, 2017 at 10:16:47PM +0200, Herby Vojčík wrote:

...

case the library name used is the plain 'sqlite3', I don't know - am I supposed to make a symlink on the same directory as the image? Is it the thing that is normally needed / done routinely?

Below is how I've been doing it since early days of Pharo. Desktop OS = Linux Mint. Current server OS = Ubuntu 16.04.

Firstly, I don't use the distro-provided libsqlite3.so. My self-built copy includes features like FTS, JSON, etc.

I place libsqlite3.so in the VM directory, say /pkg/pharo5vm/. I have a script /pkg/pharo5vm/gopharo:

#!/bin/sh PHAROVMPATH=$(dirname `readlink -f "$0"`) LD_LIBRARY_PATH="$PHAROVMPATH" exec "$PHAROVMPATH/pharo" $@ &

On desktop, I start Pharo using gopharo. On server, I use daemontools. Say my application directory on the server is /pkg/app1. Here's the daemontools run file /pkg/app1/run:

#!/bin/sh /usr/bin/setuidgid app1 \ /pkg/pharo5vm/gopharo -vm-display-null -vm-sound-null smallcms1.image --no-quit

The other files in /pkg/app1 are PharoV*.sources and the application image/changes files.

/pkg/pharo5vm contains the pharo executable, vm-*, and the *so* files/symlinks that come with Pharo plus any others that I custom build myself such as libsqlite3.so and libshacrypt.so.

HTH.

Pierce

Oh my, am doing too much Windows at the moment. Phil On Sat, Oct 7, 2017 at 11:47 PM, Alistair Grant <akgrant0710@gmail.com> wrote:

On Sat, Sep 30, 2017 at 01:28:06PM +0200, Herby Voj????k wrote:

...

phil@highoctane.be wrote:

...

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

Sorry I'm a bit late to the party...

I'm using Glorp / UDBCSQLite3 on Ubuntu 16.04 without any problems (admittedly 64 bit Ubuntu, but as you say, that shouldn't matter).

Just in case you copy-and-pasted the code, Phil's example has a typo: the path separator is a colon (:), not semicolon (;).

Install libsqlite3-dev and adding /usr/lib/i386-linux-gnu to LD_LIBRARY_PATH is all I had to do to get it to work (no changes to UDBCSQLite3Library>>library). Actually LD_LIBRARY_PATH normally shouldn't be required, but I'm running inside a snap container, which is an unusual environment.

...

root@32bit-agent:~# find / -name '*libsqlite*' -type f

2>>/dev/null

...

/usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6

It seems strange that there isn't libsqlite3.so. I think the real solution would be to find out why libsqlite3.so doesn't exist, but after installing libsqlite3-dev, a workaround would be to try creating a symbolic link from libsqlite3.so.0.8.6 to libsqlite3.so.

Cheers, Alistair