Pinning Objects in Pharo
As some of you are aware of Spur added memory pinning and in Pharo 6.1 (#60510) as well as Pharo 7 we currently have Object>>pin Object>>unpin Object>>isPinned and also Object>>setPinned:. Object pinning is explained by Eliot here [1]. I would propose two changes (at least for Pharo but ideally also for Squeak): 1. IMHO #setPinned: should be moved to a private protocol 2. Provide a #pinDuring: aBlock method like: pinDuring: aBlock [ self pin. aBlock value ] ensure: [ self unpin ] so we can ensure that a pinned object is unpinned after an operation. What do you think? Also I wonder why FFICallbackTests>>#testCqsortWithByteArray is sending #pin but not #unpin afterwards. Maybe that is the reason why there is a flag in this method that it is crashing on Windows. Bye T. [1] http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-August/195129.ht...
Hi Torsten, Hi Esteban,
On Aug 28, 2017, at 9:17 AM, Torsten Bergmann <astares@gmx.de> wrote:
As some of you are aware of Spur added memory pinning and in Pharo 6.1 (#60510) as well as Pharo 7 we currently have
Object>>pin Object>>unpin Object>>isPinned
and also Object>>setPinned:.
Object pinning is explained by Eliot here [1].
I would propose two changes (at least for Pharo but ideally also for Squeak):
1. IMHO #setPinned: should be moved to a private protocol 2. Provide a #pinDuring: aBlock method like:
pinDuring: aBlock [ self pin. aBlock value ] ensure: [ self unpin ]
so we can ensure that a pinned object is unpinned after an operation.
What do you think?
Also I wonder why FFICallbackTests>>#testCqsortWithByteArray is sending #pin but not #unpin afterwards. Maybe that is the reason why there is a flag in this method that it is crashing on Windows.
This test is news to me. Neither the test nor the class is in FFI-Tests at source.squeak.org, which is where the package lives. Why is it missing?
Bye T.
[1] http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-August/195129.ht...
Hi Eliot, this test is part of UnifiedFFI (which is Pharo only so far) - not FFI. This was just a side note. I would rather be interested if we would like to have a common #pinDuring: in Squeak, Pharo, Cuis, ... Thanks Torsten
Gesendet: Montag, 28. August 2017 um 22:08 Uhr Von: "Eliot Miranda" <eliot.miranda@gmail.com> An: "Torsten Bergmann" <astares@gmx.de>, "Esteban Lorenzano" <estebanlm@gmail.com> Cc: "Pharo Development List" <pharo-dev@lists.pharo.org>, vm-dev@lists.squeakfoundation.org Betreff: Re: Pinning Objects in Pharo
Hi Torsten, Hi Esteban,
On Aug 28, 2017, at 9:17 AM, Torsten Bergmann <astares@gmx.de> wrote:
As some of you are aware of Spur added memory pinning and in Pharo 6.1 (#60510) as well as Pharo 7 we currently have
Object>>pin Object>>unpin Object>>isPinned
and also Object>>setPinned:.
Object pinning is explained by Eliot here [1].
I would propose two changes (at least for Pharo but ideally also for Squeak):
1. IMHO #setPinned: should be moved to a private protocol 2. Provide a #pinDuring: aBlock method like:
pinDuring: aBlock [ self pin. aBlock value ] ensure: [ self unpin ]
so we can ensure that a pinned object is unpinned after an operation.
What do you think?
Also I wonder why FFICallbackTests>>#testCqsortWithByteArray is sending #pin but not #unpin afterwards. Maybe that is the reason why there is a flag in this method that it is crashing on Windows.
This test is news to me. Neither the test nor the class is in FFI-Tests at source.squeak.org, which is where the package lives. Why is it missing?
Bye T.
[1] http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-August/195129.ht...
On 29 Aug 2017, at 09:59, Torsten Bergmann <astares@gmx.de> wrote:
Hi Eliot,
this test is part of UnifiedFFI (which is Pharo only so far) - not FFI. This was just a side note.
yes, and itâs purpose is to test callbacks in the UFFI context, not callbacks itself :) Esteban
I would rather be interested if we would like to have a common #pinDuring: in Squeak, Pharo, Cuis, ...
Thanks Torsten
Gesendet: Montag, 28. August 2017 um 22:08 Uhr Von: "Eliot Miranda" <eliot.miranda@gmail.com> An: "Torsten Bergmann" <astares@gmx.de>, "Esteban Lorenzano" <estebanlm@gmail.com> Cc: "Pharo Development List" <pharo-dev@lists.pharo.org>, vm-dev@lists.squeakfoundation.org Betreff: Re: Pinning Objects in Pharo
Hi Torsten, Hi Esteban,
On Aug 28, 2017, at 9:17 AM, Torsten Bergmann <astares@gmx.de> wrote:
As some of you are aware of Spur added memory pinning and in Pharo 6.1 (#60510) as well as Pharo 7 we currently have
Object>>pin Object>>unpin Object>>isPinned
and also Object>>setPinned:.
Object pinning is explained by Eliot here [1].
I would propose two changes (at least for Pharo but ideally also for Squeak):
1. IMHO #setPinned: should be moved to a private protocol 2. Provide a #pinDuring: aBlock method like:
pinDuring: aBlock [ self pin. aBlock value ] ensure: [ self unpin ]
so we can ensure that a pinned object is unpinned after an operation.
What do you think?
Also I wonder why FFICallbackTests>>#testCqsortWithByteArray is sending #pin but not #unpin afterwards. Maybe that is the reason why there is a flag in this method that it is crashing on Windows.
This test is news to me. Neither the test nor the class is in FFI-Tests at source.squeak.org, which is where the package lives. Why is it missing?
Bye T.
[1] http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-August/195129.ht...
Hi Torsten, Hi All, re pinDuring:
On Aug 28, 2017, at 9:17 AM, Torsten Bergmann <astares@gmx.de> wrote:
As some of you are aware of Spur added memory pinning and in Pharo 6.1 (#60510) as well as Pharo 7 we currently have
Object>>pin Object>>unpin Object>>isPinned
and also Object>>setPinned:.
Object pinning is explained by Eliot here [1].
I would propose two changes (at least for Pharo but ideally also for Squeak):
1. IMHO #setPinned: should be moved to a private protocol 2. Provide a #pinDuring: aBlock method like:
pinDuring: aBlock [ self pin. aBlock value ] ensure: [ self unpin ]
so we can ensure that a pinned object is unpinned after an operation.
From an implementation perspective, I suspect this is unnecessary. There are two reasons, one to do with collecting pinned objects and the other to do with compaction.
Pinning an object forces it into oldSpace if it was not there already, but does not prevent it being garbage collected. So any un references pinned object in oldSpace will be collected at the next full GC. It is possible (but in implemented) to move not large objects back to newSpace but the forwarder left in oldSpace will prevent its collection until the next full GC so trying to move an object back into newSpace is pointless. Conclusion, there is no point unpinning for garbage collectability. The compactor does a good job of moving unpinned objects around pinned objects, typically leaving very small gaps before pinned objects. [One can observe how much free space exists in an image file to get an idea of how much free space this causes. It's low]. The number of uncollectable pinned objects in an application, which will be objects that are long-lived that have been passed through the FFI, is likely to be small. Conclusion there is little to be gained in avoiding fragmentation by unpinning long-lived objects pinned on an FFI call. In summary, collectible ones objects will be collected anyway, and the few long-lived pinned objects should not cause significant fragmentation. So pinDuring: is unlikely to provide any benefit. Simply pin and leave it to the GC and its compactor to clean up afterwards.
What do you think?
Also I wonder why FFICallbackTests>>#testCqsortWithByteArray is sending #pin but not #unpin afterwards. Maybe that is the reason why there is a flag in this method that it is crashing on Windows.
Bye T.
[1] http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-August/195129.ht...
thanks for the explanation. I think that we should add a shorter version to the pin method. On Wed, Aug 30, 2017 at 5:46 PM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Torsten, Hi All,
re pinDuring:
On Aug 28, 2017, at 9:17 AM, Torsten Bergmann <astares@gmx.de> wrote:
As some of you are aware of Spur added memory pinning and in Pharo 6.1 (#60510) as well as Pharo 7 we currently have
Object>>pin Object>>unpin Object>>isPinned
and also Object>>setPinned:.
Object pinning is explained by Eliot here [1].
I would propose two changes (at least for Pharo but ideally also for Squeak):
1. IMHO #setPinned: should be moved to a private protocol 2. Provide a #pinDuring: aBlock method like:
pinDuring: aBlock [ self pin. aBlock value ] ensure: [ self unpin ]
so we can ensure that a pinned object is unpinned after an operation.
From an implementation perspective, I suspect this is unnecessary. There are two reasons, one to do with collecting pinned objects and the other to do with compaction.
Pinning an object forces it into oldSpace if it was not there already, but does not prevent it being garbage collected. So any un references pinned object in oldSpace will be collected at the next full GC.
It is possible (but in implemented) to move not large objects back to newSpace but the forwarder left in oldSpace will prevent its collection until the next full GC so trying to move an object back into newSpace is pointless. Conclusion, there is no point unpinning for garbage collectability.
The compactor does a good job of moving unpinned objects around pinned objects, typically leaving very small gaps before pinned objects. [One can observe how much free space exists in an image file to get an idea of how much free space this causes. It's low]. The number of uncollectable pinned objects in an application, which will be objects that are long-lived that have been passed through the FFI, is likely to be small. Conclusion there is little to be gained in avoiding fragmentation by unpinning long-lived objects pinned on an FFI call.
In summary, collectible ones objects will be collected anyway, and the few long-lived pinned objects should not cause significant fragmentation. So pinDuring: is unlikely to provide any benefit. Simply pin and leave it to the GC and its compactor to clean up afterwards.
What do you think?
Also I wonder why FFICallbackTests>>#testCqsortWithByteArray is sending #pin but not #unpin afterwards. Maybe that is the reason why there is a flag in this method that it is crashing on Windows.
Bye T.
[1] http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-August/195129.ht...
participants (4)
-
Eliot Miranda -
Esteban Lorenzano -
Stephane Ducasse -
Torsten Bergmann