On Fri, Feb 17, 2017 at 1:48 PM, Cyril Ferlicot D. <cyril.ferlicot@gmail.com> wrote:
Le 17/02/2017 �� 13:31, Cl��ment Bera a ��crit :
>
>
> It may solve part of the problem.
>
> Another problem is that tenure heuristics needs to be changed when
> allocating lots of long-lived objects. One needs to implement a GCPolicy
> object in the image and use it from the application code...
>
>

What would be needed in order to make this work? I don't know the GC
enough to know what this imply.

Basically you need to have a look at the GC heuristics and analyse how they impact GC performance.

Based on that, you can have a GCPolicy instance to which you can send messages such as #changePolicyForLargeHeap, #changePolicyForSmallHeaps or #growingPolicyDuring: aBlock. Internally the GCPolicy object changes GC heuristics.

Some heuristics are already present in the vmParameter primitive (Smalltalk vm parameterAt: 46 for example), but some of them currently require the VM to be restarted. A first step would be to investigate those parameters.

For example, the GCPolicy could read those parameters to know what is the current status:

2 size of young/new-space (read-only)
3 size of heap (read-only)
6 survivor count tenuring threshold (read-write)
7 full GCs since startup (read-only)
8 total milliseconds in full GCs since startup (read-only)
9 scavenges since startup (read-only)
...
55 ratio of growth and image size at or above which a GC will be performed post scavenge
�� �� �� �� 63 total milliseconds in compiled code compactions since startup (read-only; Cog only; otherwise nil)

Based on this and the message sent to the GCPolicy, one can set new GC heuristics through those parameters:

6 survivor count tenuring threshold (read-write)
�� �� �� �� ...
45 desired size of eden, in bytes (stored in image file header)
55 ratio of growth and image size at or above which a GC will be performed post scavenge


What has to be done is:

- create a GCPolicy object with methods such as #changePolicyForLargeHeap, #changePolicyForSmallHeaps or #growingPolicyDuring: aBlock.
- use the VM parameters in those methods to analyse the current state and define new heuristics
- define a range in which the parameters can be changed without crashing the VM
- add new parameters on heuristics that are important but not configurable right now
- add the possibility to change several heuristics without restarting the image (for example change of Eden size or existing memory segment size)

For example, let's say you grow your image from 30 Mb to 8Gb (I think one of our larger user in 64 bits use 8 Gb images right now and the default Pharo image is around 30 Mb). An uncommonly high rate of objects are promoted from young space to old space and the memory is expected to grow by 26667%. So you may want to:
- decrease the survivor count for tenures. This way your objects are promoted earlier to old objects.
- increase the ratio of growth. Instead of doing a full GC each time the image size grow by 33% you may want to do it each time the image size is increased by 900%. This way instead of having 20 fullGC while growing you have only 3. You switch back the policy at the end and trigger manually a fourth fullGC and everything's fine.

Maybe we don't need a GCPolicy object, but more a VMPolicy object, so we can conveniently tweak the stack (number and size of stack page) and machine code zone.

If somebody wants to have fun :-)

��

>
> No one is working on this right now, but it's next step to improve the GC.
>
> When the development starts on this feature I can give you more
> information. If a paying customer requests it, it should be done in a
> couple months.
>
>

--
Cyril Ferlicot

http://www.synectique.eu

2 rue Jacques Pr��vert 01,
59650 Villeneuve d'ascq France