On 27 Apr 2015, at 10:42, Nicolai Hess <nicolaihess@web.de> wrote:

Hi Max

2015-04-26 19:29 GMT+02:00 Max Leske <maxleske@gmail.com <mailto:maxleske@gmail.com>>: Hi guys

I have a problem with NBExternalArray. AFAICT the generated assembly should be able to handle arbitrary elements in the array (as long as they are all of the same type). The operations required to read / write an element are given by the external type specified for the collection (e.g ‘char’ or ‘int’). All of this works wonderfully for char, int, NBExternalAddress etc. But I’m trying to get it working for strings. So here is the code that I would expect to work:

arrayClass := NBExternalArray anonymousSubclassInitElementType: 'String'. array := arrayClass new: 1. array at: 1 put: ‘foo'. array at: 1. “—> should procude ‘foo’, but produces arbitrary strings”

interesting

I can make it work with NBExternalAddress of course but I don’t want to if I can avoid it.

How would you use NBExternalAddress in this case?

By using 'char *’ or 'NBExternalAddress' as type and then converting back to a String by sending #readString to the address. My current solution acutally does that but it seems that the allocated memory is being freed sometimes without my doing, so I’m trying to let NBExternalAddress do as much as possible for me.

I’d appreciate it if someone (Nicolai?) could take a look at NBExternalArray>>emitRead / emitWrite and see if there’s an easy solution. Especially for strings I don’t think it should be too hard since all that’s needed is already present in the string handling facilities.

Somehow it only works half the way, either write the String / or read the String.

exactly…

this one: arrayClass := NBExternalArray ofType: 'String'. array := arrayClass new: 1. array at: 1 put: 'foo'. array at:1

reads the data back to a string object, but I think the data was written wrong.

this one: arrayClass := NBExternalArray ofType: 'char*'. array := arrayClass new: 1. array at: 1 put: 'foo'.

at least writes the right data, but now array at:1 contains an external address and it is not automatically converted back to a string. You can confirm that the data is written correctly by explicitly creating a string from the external address. (array at:1) readString -> 'foo'

Now, why is the String data written wrong in the first case? (Maybe this has something to do with #pointerArity. I think NBExternalString should have pointerArity 1. I changed this for NBExternalString, but it seems this does not make a difference - and may break other things). It looks like some "double conversion" ST-String -> char*, that is, it converts a String as NBExternalString and uses this to do the conversion again:

I played it bit with the code (NB makes really fun, no irony, it really makes fun, it is a nice piece of work). and created a new NBExternalArray subclass StringArray:

NBExternalArray subclass: #StringArray instanceVariableNames: '' classVariableNames: '' category: 'NativeBoost-Core-Objects'

define the class initialize method

initialize self initElementType: #String

and overwrite emitWrite, the same code as in NBExternalArray, but replace

function: ' oop ( oop value , uint32 index , void * data , ' , self class elementType , ' value )' with function: ' oop ( oop value , uint32 index , void * data , char* value )’

Smart! If that works conistently (I’ll test it) we should be able to find the problem.

now, this works: arrayClass := StringArray. array := arrayClass new: 1. array at: 1 put: 'foo'. (array at:1) -> 'foo'

So, maybe the default function prototype in NBExternalArray>>#emitWrite is responsible for the double conversion or the pointer arity.

I don't fully understand what is going on :), but maybe this helps.

Thanks a lot Nicolai, I really appreciate your help. Cheers, Max

nicolai

Cheers, Max

On 27 Apr 2015, at 16:14, Henrik Johansen <henrik.s.johansen@veloxit.no> wrote:

...

On 26 Apr 2015, at 7:29 , Max Leske <maxleske@gmail.com> wrote:

Hi guys

I have a problem with NBExternalArray. AFAICT the generated assembly should be able to handle arbitrary elements in the array (as long as they are all of the same type). The operations required to read / write an element are given by the external type specified for the collection (e.g ‘char’ or ‘int’). All of this works wonderfully for char, int, NBExternalAddress etc. But I’m trying to get it working for strings. So here is the code that I would expect to work:

arrayClass := NBExternalArray anonymousSubclassInitElementType: 'String'. array := arrayClass new: 1. array at: 1 put: ‘foo'. array at: 1. “—> should procude ‘foo’, but produces arbitrary strings”

I can make it work with NBExternalAddress of course but I don’t want to if I can avoid it.

I’d appreciate it if someone (Nicolai?) could take a look at NBExternalArray>>emitRead / emitWrite and see if there’s an easy solution. Especially for strings I don’t think it should be too hard since all that’s needed is already present in the string handling facilities.

Cheers, Max

There's a solution, but I doubt you could call it simple. NBExternalArray is meant to hold const sized objects, so is allocated with the proper amount of slots up front. If you want it to hold pointers to variably sized objects (like strings), since we can't pin the objects, they need to be copied (well, would have to do that anyways for Strings, which need it'd need to do dynamic allocation (and freeing when appropriate), so emitWrite would become quite a bit more complicated, + necessary object finalization would have to occur.

Right. So space is allocated for the constant sized objects or the pointers but not for the variable sized objects… That explains what I’m seeing with the memory being used sometimes. I guess allocation and finalization can easily be done in a dedicated subclass, so I’ll try that. Thanks a lot Henry! Cheers, Max

You *can* work around it, by using an element type that is externally allocated, but it adds quite a bit of boilerplate, here's the workspace PoC code:

string := 'herru€'. bytes := ZnUTF8Encoder new encodeString: string. mem := NativeBoost allocate: bytes size + 1. NativeBoost memCopy: bytes to: mem size: bytes size.

arrClass := NBExternalArray anonymousSubclassInitElementType: 'char *'. arr := arrClass new: 1. arr at: 1 put: mem. ZnUTF8Encoder new decodeBytes: (arr at:1) readString asByteArray

Cheers, Henry