Hi, I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI. My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres... Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs. RG
usually, something more or less like this: sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1 and or course you can play with the starting points and sizes. cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
Hi Esteban, not sure I'm understanding your snippet correctly. Below is my try. It throws a SubscriptOutOfBounds exception in the last line. The reason is that the implementation of ByteArray replaceFrom:to:with:startingAt: does not consider that ea is meant to be a pointer to a C byte array, not an object whose content is to be copied directly. | ea ba | ea := ExternalAddress gcallocate: 200. ba := ByteArray new: 100. ba replaceFrom: 1 to: 50 with: ea startingAt: 51. Am I missing some point? On 2017-02-21 17:20, Esteban Lorenzano wrote:
usually, something more or less like this:
sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1
and or course you can play with the starting points and sizes.
cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
Hi, browsing through ByteArray's and ExternalAddress's code, I'm getting the impression that copying *portions* of ByteArrays from/to the C heap can be quite costly. Copying *to* the C heap sometimes requires first copying the portion of the ByteArray into a newly instantiated temporary ByteArray, then copying the latter to the C heap. Copying *from* the C heap might require explicitly looping over the single bytes. I say "might" because I still cannot find more direct way, as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work. Any suggestion on more effective copy operations? Thanks Raffaello On 22/02/17 15:08, Raffaello Giulietti wrote:
Hi Esteban,
not sure I'm understanding your snippet correctly.
Below is my try. It throws a SubscriptOutOfBounds exception in the last line. The reason is that the implementation of ByteArray replaceFrom:to:with:startingAt: does not consider that ea is meant to be a pointer to a C byte array, not an object whose content is to be copied directly.
| ea ba | ea := ExternalAddress gcallocate: 200. ba := ByteArray new: 100. ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
Am I missing some point?
On 2017-02-21 17:20, Esteban Lorenzano wrote:
usually, something more or less like this:
sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1
and or course you can play with the starting points and sizes.
cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
Sorry, I forgot to answer and now I'm in movement. Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image. And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too. Esteban
On 23 Feb 2017, at 17:21, raffaello.giulietti@lifeware.ch wrote:
Hi,
browsing through ByteArray's and ExternalAddress's code, I'm getting the impression that copying *portions* of ByteArrays from/to the C heap can be quite costly.
Copying *to* the C heap sometimes requires first copying the portion of the ByteArray into a newly instantiated temporary ByteArray, then copying the latter to the C heap.
Copying *from* the C heap might require explicitly looping over the single bytes. I say "might" because I still cannot find more direct way, as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
Any suggestion on more effective copy operations?
Thanks Raffaello
On 22/02/17 15:08, Raffaello Giulietti wrote: Hi Esteban,
not sure I'm understanding your snippet correctly.
Below is my try. It throws a SubscriptOutOfBounds exception in the last line. The reason is that the implementation of ByteArray replaceFrom:to:with:startingAt: does not consider that ea is meant to be a pointer to a C byte array, not an object whose content is to be copied directly.
| ea ba | ea := ExternalAddress gcallocate: 200. ba := ByteArray new: 100. ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
Am I missing some point?
On 2017-02-21 17:20, Esteban Lorenzano wrote: usually, something more or less like this:
sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1
and or course you can play with the starting points and sizes.
cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
Hi Esteban, yes, I already looked at LibC's memcpy, but that doesn't help in case you need to copy a *portion* of the ByteArray that does not start at index 1. While one can do pointer arithmetic on ExternalAddress, this does not, of course, apply to ByteArray. To sum up, I think Pharo 5 currently lacks an efficient way to copy parts of ByteArrays to/from the C heap. One has to instantiate additional temporary ByteArrays and perform copies between ByteArrays in addition to LibC>>memcpy. But I see there's an Alien class which seems to better support such copying. Thus, I wonder if it is somehow compatible with the rest of UFFI. On 23/02/17 17:35, Esteban Lorenzano wrote:
Sorry, I forgot to answer and now I'm in movement. Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.
And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.
Esteban
On 23 Feb 2017, at 17:21, raffaello.giulietti@lifeware.ch wrote:
Hi,
browsing through ByteArray's and ExternalAddress's code, I'm getting the impression that copying *portions* of ByteArrays from/to the C heap can be quite costly.
Copying *to* the C heap sometimes requires first copying the portion of the ByteArray into a newly instantiated temporary ByteArray, then copying the latter to the C heap.
Copying *from* the C heap might require explicitly looping over the single bytes. I say "might" because I still cannot find more direct way, as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
Any suggestion on more effective copy operations?
Thanks Raffaello
On 22/02/17 15:08, Raffaello Giulietti wrote: Hi Esteban,
not sure I'm understanding your snippet correctly.
Below is my try. It throws a SubscriptOutOfBounds exception in the last line. The reason is that the implementation of ByteArray replaceFrom:to:with:startingAt: does not consider that ea is meant to be a pointer to a C byte array, not an object whose content is to be copied directly.
| ea ba | ea := ExternalAddress gcallocate: 200. ba := ByteArray new: 100. ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
Am I missing some point?
On 2017-02-21 17:20, Esteban Lorenzano wrote: usually, something more or less like this:
sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1
and or course you can play with the starting points and sizes.
cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
On 23 Feb 2017, at 19:11, raffaello.giulietti@lifeware.ch wrote:
Hi Esteban,
yes, I already looked at LibC's memcpy, but that doesn't help in case you need to copy a *portion* of the ByteArray that does not start at index 1. While one can do pointer arithmetic on ExternalAddress, this does not, of course, apply to ByteArray.
To sum up, I think Pharo 5 currently lacks an efficient way to copy parts of ByteArrays to/from the C heap. One has to instantiate additional temporary ByteArrays and perform copies between ByteArrays in addition to LibC>>memcpy.
But I see there's an Alien class which seems to better support such copying. Thus, I wonder if it is somehow compatible with the rest of UFFI.
Which Alien class? Esteban
On 23/02/17 17:35, Esteban Lorenzano wrote:
Sorry, I forgot to answer and now I'm in movement. Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.
And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.
Esteban
On 23 Feb 2017, at 17:21, raffaello.giulietti@lifeware.ch wrote:
Hi,
browsing through ByteArray's and ExternalAddress's code, I'm getting the impression that copying *portions* of ByteArrays from/to the C heap can be quite costly.
Copying *to* the C heap sometimes requires first copying the portion of the ByteArray into a newly instantiated temporary ByteArray, then copying the latter to the C heap.
Copying *from* the C heap might require explicitly looping over the single bytes. I say "might" because I still cannot find more direct way, as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
Any suggestion on more effective copy operations?
Thanks Raffaello
On 22/02/17 15:08, Raffaello Giulietti wrote: Hi Esteban,
not sure I'm understanding your snippet correctly.
Below is my try. It throws a SubscriptOutOfBounds exception in the last line. The reason is that the implementation of ByteArray replaceFrom:to:with:startingAt: does not consider that ea is meant to be a pointer to a C byte array, not an object whose content is to be copied directly.
| ea ba | ea := ExternalAddress gcallocate: 200. ba := ByteArray new: 100. ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
Am I missing some point?
On 2017-02-21 17:20, Esteban Lorenzano wrote: usually, something more or less like this:
sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1
and or course you can play with the starting points and sizes.
cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
On 23/02/17 19:17, Esteban Lorenzano wrote:
On 23 Feb 2017, at 19:11, raffaello.giulietti@lifeware.ch wrote:
Hi Esteban,
yes, I already looked at LibC's memcpy, but that doesn't help in case you need to copy a *portion* of the ByteArray that does not start at index 1. While one can do pointer arithmetic on ExternalAddress, this does not, of course, apply to ByteArray.
To sum up, I think Pharo 5 currently lacks an efficient way to copy parts of ByteArrays to/from the C heap. One has to instantiate additional temporary ByteArrays and perform copies between ByteArrays in addition to LibC>>memcpy.
But I see there's an Alien class which seems to better support such copying. Thus, I wonder if it is somehow compatible with the rest of UFFI.
Which Alien class?
Esteban
What do you mean? I see it in package Alien-Core.
On 23/02/17 17:35, Esteban Lorenzano wrote:
Sorry, I forgot to answer and now I'm in movement. Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.
And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.
Esteban
On 23 Feb 2017, at 17:21, raffaello.giulietti@lifeware.ch wrote:
Hi,
browsing through ByteArray's and ExternalAddress's code, I'm getting the impression that copying *portions* of ByteArrays from/to the C heap can be quite costly.
Copying *to* the C heap sometimes requires first copying the portion of the ByteArray into a newly instantiated temporary ByteArray, then copying the latter to the C heap.
Copying *from* the C heap might require explicitly looping over the single bytes. I say "might" because I still cannot find more direct way, as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
Any suggestion on more effective copy operations?
Thanks Raffaello
On 22/02/17 15:08, Raffaello Giulietti wrote: Hi Esteban,
not sure I'm understanding your snippet correctly.
Below is my try. It throws a SubscriptOutOfBounds exception in the last line. The reason is that the implementation of ByteArray replaceFrom:to:with:startingAt: does not consider that ea is meant to be a pointer to a C byte array, not an object whose content is to be copied directly.
| ea ba | ea := ExternalAddress gcallocate: 200. ba := ByteArray new: 100. ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
Am I missing some point?
On 2017-02-21 17:20, Esteban Lorenzano wrote: usually, something more or less like this:
sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1
and or course you can play with the starting points and sizes.
cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
On 23 Feb 2017, at 19:21, raffaello.giulietti@lifeware.ch wrote:
On 23/02/17 19:17, Esteban Lorenzano wrote:
On 23 Feb 2017, at 19:11, raffaello.giulietti@lifeware.ch wrote:
Hi Esteban,
yes, I already looked at LibC's memcpy, but that doesn't help in case you need to copy a *portion* of the ByteArray that does not start at index 1. While one can do pointer arithmetic on ExternalAddress, this does not, of course, apply to ByteArray.
To sum up, I think Pharo 5 currently lacks an efficient way to copy parts of ByteArrays to/from the C heap. One has to instantiate additional temporary ByteArrays and perform copies between ByteArrays in addition to LibC>>memcpy.
But I see there's an Alien class which seems to better support such copying. Thus, I wonder if it is somehow compatible with the rest of UFFI.
Which Alien class?
Esteban
What do you mean? I see it in package Alien-Core.
ah, problem is that #copyInto:from:to:in:startingAt: requires an Alien instead a ExternalAddress (and they are not the same). But I guess you can use it if you have two ByteArrays. in the times of old NativeBoost, Igor implemented a memcpy as you want it, because yes, is a problem we have⦠if is seriously needed, you can always compile your own library with such symbol⦠a memcpy with start positions. function in C is more or less trivial because you will receive two pointers and you can perform pointer arithmetic on both. Esteban
On 23/02/17 17:35, Esteban Lorenzano wrote:
Sorry, I forgot to answer and now I'm in movement. Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.
And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.
Esteban
On 23 Feb 2017, at 17:21, raffaello.giulietti@lifeware.ch wrote:
Hi,
browsing through ByteArray's and ExternalAddress's code, I'm getting the impression that copying *portions* of ByteArrays from/to the C heap can be quite costly.
Copying *to* the C heap sometimes requires first copying the portion of the ByteArray into a newly instantiated temporary ByteArray, then copying the latter to the C heap.
Copying *from* the C heap might require explicitly looping over the single bytes. I say "might" because I still cannot find more direct way, as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
Any suggestion on more effective copy operations?
Thanks Raffaello
On 22/02/17 15:08, Raffaello Giulietti wrote: Hi Esteban,
not sure I'm understanding your snippet correctly.
Below is my try. It throws a SubscriptOutOfBounds exception in the last line. The reason is that the implementation of ByteArray replaceFrom:to:with:startingAt: does not consider that ea is meant to be a pointer to a C byte array, not an object whose content is to be copied directly.
| ea ba | ea := ExternalAddress gcallocate: 200. ba := ByteArray new: 100. ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
Am I missing some point?
On 2017-02-21 17:20, Esteban Lorenzano wrote: usually, something more or less like this:
sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1
and or course you can play with the starting points and sizes.
cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
On 23/02/17 19:26, Esteban Lorenzano wrote:
On 23 Feb 2017, at 19:21, raffaello.giulietti@lifeware.ch <mailto:raffaello.giulietti@lifeware.ch> wrote:
On 23/02/17 19:17, Esteban Lorenzano wrote:
On 23 Feb 2017, at 19:11, raffaello.giulietti@lifeware.ch <mailto:raffaello.giulietti@lifeware.ch> wrote:
Hi Esteban,
yes, I already looked at LibC's memcpy, but that doesn't help in case you need to copy a *portion* of the ByteArray that does not start at index 1. While one can do pointer arithmetic on ExternalAddress, this does not, of course, apply to ByteArray.
To sum up, I think Pharo 5 currently lacks an efficient way to copy parts of ByteArrays to/from the C heap. One has to instantiate additional temporary ByteArrays and perform copies between ByteArrays in addition to LibC>>memcpy.
But I see there's an Alien class which seems to better support such copying. Thus, I wonder if it is somehow compatible with the rest of UFFI.
Which Alien class?
Esteban
What do you mean? I see it in package Alien-Core.
ah, problem is that #copyInto:from:to:in:startingAt: requires an Alien instead a ExternalAddress (and they are not the same). But I guess you can use it if you have two ByteArrays.
in the times of old NativeBoost, Igor implemented a memcpy as you want it, because yes, is a problem we have⦠if is seriously needed, you can always compile your own library with such symbol⦠a memcpy with start positions. function in C is more or less trivial because you will receive two pointers and you can perform pointer arithmetic on both.
Esteban
What about an existing library that I don't want or cannot modify? Really, I agree that this is a problem that Pharo has to solve. I think the UFFI deserves a more general and cheaper way of copying between ByteArrays and the C heap. Can I expect it to appear in the short term? Can I help somehow? Raffaello
What about an existing library that I don't want or cannot modify?
Really, I agree that this is a problem that Pharo has to solve.
Yes we all agree on that. For the record, esteban rewrote from Scratch an FFI and it was not in our plans due to the change in Spur. We would have preferred that somebody else does it because we burnt around 8 months of work to get throw.
I think the UFFI deserves a more general and cheaper way of copying between ByteArrays and the C heap.
Yes anything that makes interaction with C libraries easier, faster, nicer is needed. Now having 64 bits is also key and we cannot clone esteban so far.
Can I expect it to appear in the short term? Can I help somehow?
I'm sure. Stef
Hi rafaello once you find the solution can you do a pull request on the UFFI doc? Your question is important we should improve the documentation. tx Stf
Hi,
browsing through ByteArray's and ExternalAddress's code, I'm getting the impression that copying *portions* of ByteArrays from/to the C heap can be quite costly.
Copying *to* the C heap sometimes requires first copying the portion of the ByteArray into a newly instantiated temporary ByteArray, then copying the latter to the C heap.
Copying *from* the C heap might require explicitly looping over the single bytes. I say "might" because I still cannot find more direct way, as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
Any suggestion on more effective copy operations?
Thanks Raffaello
On 22/02/17 15:08, Raffaello Giulietti wrote:
Hi Esteban,
not sure I'm understanding your snippet correctly.
Below is my try. It throws a SubscriptOutOfBounds exception in the last line. The reason is that the implementation of ByteArray replaceFrom:to:with:startingAt: does not consider that ea is meant to be a pointer to a C byte array, not an object whose content is to be copied directly.
| ea ba | ea := ExternalAddress gcallocate: 200. ba := ByteArray new: 100. ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
Am I missing some point?
On 2017-02-21 17:20, Esteban Lorenzano wrote:
usually, something more or less like this:
sourceByteArrayOrExternalAddress := ⦠some ... destEnternalAddressOrByteArray replaceFrom: 1 to: sourceByteArrayOrExternalAddress size with: sourceByteArrayOrExternalAddress startingAt: 1
and or course you can play with the starting points and sizes.
cheers, Esteban
On 21 Feb 2017, at 17:13, Raffaello Giulietti <raffaello.giulietti@lifeware.ch> wrote:
Hi,
I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgres...
Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
RG
-- Using Opera's mail client: http://www.opera.com/mail/
participants (4)
-
Esteban Lorenzano -
Raffaello Giulietti -
raffaello.giulietti＠lifeware.ch
-
stepharong