Hi Guille,

�� ��good news!�� But I'm seeing something wrong with finalisation of weak arrays using the new scheme.�� In Squeak method source access is done by default by opening a new read-only file for each method's source read (crazy, but that's not the issue).�� So by default something that accesses lots of source ends up running out of file descriptors, which causes primOpen:writable: to fail.�� The surrounding code then uses retryWithGC:until:forFileNamed: to do a GC to try and reclaim non-longer referenced files, close file descriptors and continue:

�� �� �� �� StandardFileStream retryWithGC:[self primOpen: f writable: writeMode]��
until:[:id| id notNil]��
forFileNamed: fileName.

But in my tests I'm not seeing any files reclaimed.�� I wonder whether the new finalisation code is failing to finalise weak arrays properly.�� I wonder whether your weak tests work properly with the new scheme or not.

Anyway, I think I can reproduce the pathology in the simulator if I modify it to implement a small limit on the number of file descriptors.�� I'm going to try that to shed light on the problem.�� I can't easily debug in a running image because...I run out of file descriptors ;-)

On Tue, Jun 7, 2016 at 2:32 AM, Guille Polito <guillermopolito@gmail.com> wrote:
Hi All,

Since this morning, in Pharo #60065, Ephemeron support is in the image. Most of the changes are infrastructural, so far transparent for the users. It is important to notice that even while the support is there, it is not enabled by default. Also, this required changes in the virtual machine that are not yet distributed everywhere. For the ones that would like more detail, I invite you to read the following :)

* On the infrastructure side
�� - There is support to create Ephemeric classes and load them from monticello
�� - There is a new finalization mechanism (by default disabled) that will process Ephemerons using a finalization queue. This will avoid scanning collections in look for weak objects to finalize ,as it happens now with the WeakDependent mechanism in WeakArray.
�� - System-Finalization features two new classes *Ephemeron* and *EphemeronRegistry*. For the ones that want more details on Ephemerons, you can read the associated paper [1], or the class comment of Ephemeron:

I represent ephemeric key-value objects. Ephemerons are key-value objects (subclasses of Association) with special semantics during garbage collection.�� My special behavior can resumed as follows:

- The garbage collection will iterate my instances only if the key is not referenced strongly by another object.
- Then, if no strong references to the key are found, then the values of this ephemeron are hold weakly.
- Otherwise, the values are hold strongly.

In this implementation, an Ephemeron can hold more than one value, which are all treated in the same manner. This ephemeron instance knows its container, which allows the ephemeron to remove itself from a container (such as a Dictionary) upon finalization.

!! Example usages

In general terms, do not use myself directly. Use instead an Ephemeric container like EphemeronRegistry. An Ephemeron registry will guarantee the collection of keys and values of the object inside the Ephemeron.

Otherwise, if you want to use it, you can create an Ephemeron as any association:

ephemeron := Ephemeron key: aKey value: aValue.
ephemeron container: aContainer.

!! Ephemeron Finalization

When an ephemeron's key is hold strongly just by the ephemeron itself, the Ephemeron will be mourned (finalized). That means that the VM will:
- put the Ephemeron in the mourning queue waiting for the image to take care of mourning
- make the Ephemeron non ephemeric. That is, the ephemeron instance cannot be reused.

On the image side, the finalization process will send the message #mourn to an Ephemeron.�� #mourn will #finalize the Ephemeron's key, and remove the Ephemeron from it's container to allow its collection during a subsequent garbage collection.

!! More Documentation

You can read the associated paper to understand better the semantics of ephemerons:

[1]Ephemerons: A New Finalization Mechanism. Barry Hayes. OOPSLA '97


�� - WARNING: to be able to use ephemerons, you need to use the *latestVm* that has several fixes for making ephemerons work, and you need to enable ephemerons on the image side by evaluating:

���������������������� Smalltalk supportsQueueingFinalization: true.

�� - With latest vm and ephemerons enabled, tests should be green, otherwise they are skipped



* From the user point of view:

�� - The Weak registries were not yet migrated to the new finalization mechanism.
�� - We expect nothing will change from the user point of view. Just less memory leaks.
��
* Next steps (in order)
�� 1) Bless the latest vm as stable
�� 2) Enable queueing finalization by default
�� 3) Replace Weak Registry by Ephemeron Registry.


Informed by: Guille



--
_,,,^..^,,,_
best,��Eliot