On 27 November 2017 at 00:24, Todd Blanchard <tblanchard@mac.com> wrote:

��
i'm getting the idea that we should probably write a test suite/library for FFI

I noticed these...

* https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/ffi-test-main.c

*��https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/SqueakFFIPrims/sqFFITestFuncs.c

*��https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/IA32ABI/AlienSUnitTestProcedures.c

These look like c-code test frames for FFI.�� Could these be built by the OpenSmalltalk CI to be normally shipped with the VM so that Image-side CI can test against them?

And also some general info...

*��https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/00README

��

On Nov 24, 2017, at 12:54 AM, Ben Coman <btc@openinworld.com> wrote:

On 24 November 2017 at 13:16, Ben Coman��<btc@openinworld.com>��wrote:

On 22 November 2017 at 21:59, Ben Coman��<btc@openinworld.com>��wrote:

On 22 November 2017 at 13:38, Todd Blanchard��<tblanchard@mac.com>��wrote:
��
I've been trying to track this down for a couple weeks now.

I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated.�� The struct's memory is all zero'd.

I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations.�� After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.

I've carved out the wee bit of clang to reproduce this in a tiny library.

The gist of it is below and the entire file is included.�� Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.

My last analysis discovered something interesting about strings defined inside shared libraries being handled differently, but only later realised I had chased the wrong rabbit down the hole.��

��

I've now investigated the premise you actually poses, and I agree, that structs being zero is some cases.��

The attached zipfile containing�� libstruct.c�� a few comparison cases - three "good" layouts that work fine and one "bad" that mostly gets zeros but sometimes other weird numbers.�� There is a Makefile with three main targets:

1. make layout - statically compiles�� libstruct.c�� and runs produced�� a.out�� to display structure layouts��

2. make run - downloads Pharo, starts it loading�� LibStruct.st�� , then you manually run LibStruct>>>LibStructTest>>#testStructs and observe structure values on console

3. make debug - starts LLDB to run Pharo with breakpoints pre-configured for when you run #testStructs.��

�� Note Pharo will freeze and you need to move to LLDB.�� Try these commands...��

�� frame variable

�� call print_struct(&GoodStruct1_fmt, &aStruct)

�� continue

That works on Ubuntu 16.04 64 bit.��

You will need to tune it for OSX.

clang and lldb are required.

================================

The offsets configured in the class variables of all ExternalStructs��

correctly matches that reported by the C code test frame results here...

$ make layout��

clang -g libstruct.c

./a.out

GoodStruct1:

uint32_t:int1:�� 01 4��

uint32_t:int2:�� 05 4��

GoodStruct2:

uint32_t:int1:�� 01 4��

uint32_t:int2:�� 05 4��

void*:ptr_data:�� 09 8��

GoodStruct3:

void*:ptr_data:�� 01 8��

uint32_t:int1:�� 09 4��

uint32_t:int2:�� 13 4��

BadStruct:

void*:ptr_data1:�� 01 8��

void*:ptr_data2:�� 09 8��

uint32_t:int1:�� 17 4��

uint32_t:int2:�� 21 4��

================================

$ make run��

clang -g -o libstruct.so -shared -fPIC libstruct.c

getpharo/pharo-vm/lib/pharo/5.0-201707201942/pharo getpharo/Pharo.image ../LibStruct.st

Now manually browse to and run�� LibStruct>>>LibStructTest>>#testStructs

LibStructTest>>testStructs

|good1Struct good2Struct good3Struct badStruct |

good1Struct�� := GoodStruct1�� new int1: 2; int2: 3.

good2Struct�� := GoodStruct2�� new int1: 2; int2: 3; ptr_force_int: 4.

good3Struct�� := GoodStruct3�� new int1: 2; int2: 3; ptr_force_int: 4.

badStruct�� := BadStruct�� new int1: 2; int2: 3; ptr_force_int1: 4; ptr_force_int1: 5.

self assert: (LibStruct tryGood1: good1Struct)�� equals: 6.

self assert: (LibStruct tryGood2: good2Struct)�� equals: 6.

self assert: (LibStruct tryGood3: good3Struct)�� equals: 6.

"Problem exposed in next line"

self assert: (LibStruct tryBad: badStruct)�� equals: 6.

which on console produces...

GoodStruct1:

uint32_t:int1:�� 01 4 = 02 00 00 00

uint32_t:int2:�� 05 4 = 03 00 00 00

GoodStruct2:

uint32_t:int1:�� 01 4 = 02 00 00 00

uint32_t:int2:�� 05 4 = 03 00 00 00

void*:ptr_data:�� 09 8 = 04 00 00 00 00 00 00 00

GoodStruct3:

void*:ptr_data:�� 01 8 = 04 00 00 00 00 00 00 00

uint32_t:int1:�� 09 4 = 02 00 00 00

uint32_t:int2:�� 13 4 = 03 00 00 00

BadStruct:

void*:ptr_data1:�� 01 8 = 00 00 00 00 00 00 00 00

void*:ptr_data2:�� 09 8 = 00 00 00 00 00 00 00 00

uint32_t:int1:�� 17 4 = 29 03 03 00

uint32_t:int2:�� 21 4 = 00 00 00 00

Comparing GoodStruct3 and BadStruct, it seems one pointer is handled fine, but not two.

The first time this is run after Image boots seems like BadStruct��gets some random data.

================================

In same image, subsequent runs of�� LibStruct>>>LibStructTest>>testStructs��

give only all zeros for BadStruct, as Todd observed.

GoodStruct1:

uint32_t:int1:�� 01 4 = 02 00 00 00

uint32_t:int2:�� 05 4 = 03 00 00 00

GoodStruct2:

uint32_t:int1:�� 01 4 = 02 00 00 00

uint32_t:int2:�� 05 4 = 03 00 00 00

void*:ptr_data:�� 09 8 = 00 00 00 00 00 00 00 00

GoodStruct3:

void*:ptr_data:�� 01 8 = 00 00 00 00 00 00 00 00

uint32_t:int1:�� 09 4 = 02 00 00 00

uint32_t:int2:�� 13 4 = 03 00 00 00

BadStruct:

void*:ptr_data1:�� 01 8 = 00 00 00 00 00 00 00 00

void*:ptr_data2:�� 09 8 = 00 00 00 00 00 00 00 00

uint32_t:int1:�� 17 4 = 00 00 00 00

uint32_t:int2:�� 21 4 = 00 00 00 00

cheers -ben