Re: [Pharo-dev] FogBugz (Case [Issue]10840) Collection - WeakSet>>#size utterly broken
On 7 juin 2013, at 11:19, Pharo Issue Tracker wrote:
A FogBugz case was edited by Stephane Ducasse.
Case ID: 10840 Title: WeakSet>>#size utterly broken Status: Resolved (Fix Review Needed) Category: Bug Project: Collection Area: Misc Priority: 3 - Must Fix Milestone: Pharo3.0: 30/03/2014 Assigned To: Camille Teruel
URL: https://pharo.fogbugz.com/f/cases/10840
Last message: Please can you send a mail to the mailing-list so that we all discuss this problems.
Hello everyone, WeakSet>>#size is broken as demonstrated by the following snippet: set := WeakSet new. set add: Object new. Smalltalk garbageCollect. set size. "answers 1 instead of 0" That is because a WeakSet has no mean to know when its items get garbage collected, and thus the tally cannot be updated. So we need to override #size in Weak, I propose: size | counter | counter := 0. self do: [ :e | counter := counter + 1 ]. ^ counter But that it rather inefficient for such a simple query. Any proposition? The bug entry is: https://pharo.fogbugz.com/f/cases/10840
If you do not want to receive automatic notifications anymore, change your preferences in the Options screen. (https://pharo.fogbugz.com/default.asp?pg=pgPrefs)
On Jun 7, 2013, at 11:34 AM, Camille Teruel wrote:
On 7 juin 2013, at 11:19, Pharo Issue Tracker wrote:
A FogBugz case was edited by Stephane Ducasse.
Case ID: 10840 Title: WeakSet>>#size utterly broken Status: Resolved (Fix Review Needed) Category: Bug Project: Collection Area: Misc Priority: 3 - Must Fix Milestone: Pharo3.0: 30/03/2014 Assigned To: Camille Teruel
URL: https://pharo.fogbugz.com/f/cases/10840
Last message: Please can you send a mail to the mailing-list so that we all discuss this problems.
Hello everyone,
WeakSet>>#size is broken as demonstrated by the following snippet:
set := WeakSet new. set add: Object new. Smalltalk garbageCollect. set size. "answers 1 instead of 0"
That is because a WeakSet has no mean to know when its items get garbage collected, and thus the tally cannot be updated. So we need to override #size in Weak, I propose: size | counter | counter := 0. self do: [ :e | counter := counter + 1 ]. ^ counter But that it rather inefficient for such a simple query. Any proposition?
I would leave size as is, with a comment that by default, it should not be trusted. Then provide an alternative WeakSet constructor, which returns an instance which registers objects added to it for finalization, with a finalizing action basically doing size := size - 1. One complication is to make sure to detect and handle the case where object being added is already registered for finalization. I think the current size-inaccurate should still be the default, as the alternate will (for obvious reasons) sacrifice speed of normal ops for a fast, more* accurate size, and as such should only be used when that is important. Cheers, Henry * In the sense that, I'm not sure what GC actions actually trigger the finalization process⦠scavenging? Either way higher-than-finalization priority threads can still strictly speaking return inaccurate sizes
On Fri, Jun 7, 2013 at 2:34 AM, Camille Teruel <camille.teruel@gmail.com>wrote:
On 7 juin 2013, at 11:19, Pharo Issue Tracker wrote:
A FogBugz case was edited by Stephane Ducasse.
Case ID: 10840 Title: WeakSet>>#size utterly broken Status: Resolved (Fix Review Needed) Category: Bug Project: Collection Area: Misc Priority: 3 - Must Fix Milestone: Pharo3.0: 30/03/2014 Assigned To: Camille Teruel
URL: https://pharo.fogbugz.com/f/cases/10840
Last message: Please can you send a mail to the mailing-list so that we all discuss this problems.
Hello everyone,
WeakSet>>#size is broken as demonstrated by the following snippet:
set := WeakSet new. set add: Object new. Smalltalk garbageCollect. set size. "answers 1 instead of 0"
That is because a WeakSet has no mean to know when its items get garbage collected, and thus the tally cannot be updated. So we need to override #size in Weak, I propose: size | counter | counter := 0. self do: [ :e | counter := counter + 1 ]. ^ counter But that it rather inefficient for such a simple query. Any proposition?
One way is to add WeakSets to FInalizationDependents and implement finalizeValues to update the size. But that's making work. A lazy solution might be better. You *could*, I suppose, use the VM's count of the number of incremental and full GCs (alas there's no single count). I think this is too ugly and one would need something neater, but it would work like this: size | newGCCount | newGCCount := self getGCCount. newGCCount == existingGCCount ifTrue: [^tally]. tally := self computeActualSIze. existingGCCount := newGCCount where getGCCount ^(Smalltalk vmParameterAt: 7) + (Smalltalk vmParameterAt: 9) There are obviously thread-safety issues here, but there are issues with the existing code and your suggestion too ;) The most useful single count here would be a count that was incremented once in each GC that finalized one or more references. Note also that for this to be reliable instances' record of the count would have to be invalidated on snapshot or start-up, which could entail an expensive allInstancesDo:. Perhaps the class WeakSet should maintain its instances in (of course) a WeakSet and then add only itself to FinalizationDependents and enumerate the set on finalization? I note that in the VisualWorks VM, objects that have references finalized are added to a queue so every instance that loses a reference gets sent finalize, instead of only dependents as in Squeak. This has its won problems (the VM must cope with a potentially large list) but I think it provides more convenient control.
The bug entry is: https://pharo.fogbugz.com/f/cases/10840
If you do not want to receive automatic notifications anymore, change your preferences in the Options screen. ( https://pharo.fogbugz.com/default.asp?pg=pgPrefs)
-- best, Eliot
Oops, I'm soooo stupid :) Instead, put the count in the finalization loop, and compare against it. e.g. finalizationProcess [true] whileTrue: [ WeakFinalizationList initTestPair. FinalizationSemaphore wait.
FinalizationCount := FinalizationCount + 1. FinalizationLock critical: [ WeakFinalizationList checkTestPair. FinalizationDependents do: [:weakDependent | weakDependent ifNotNil: [ [ weakDependent finalizeValues ] on: Exception fork: [:ex | ex pass ] ]]]].
finalizationCount ^FinalizationCount Set subclass: #WeakSet instanceVariableNames: 'flag finalizationCount' classVariableNames: '' poolDictionaries: '' category: 'Collections-Weak' size | newFinalizationCount | newFinalizationCount := WeakArray getFinalizationCount. newFinalizationCount == finalizationCount ifTrue: [^tally]. tally := self computeActualSIze. finalizationCount := newFinalizationCount On Fri, Jun 7, 2013 at 11:30 AM, Eliot Miranda <eliot.miranda@gmail.com>wrote:
On Fri, Jun 7, 2013 at 2:34 AM, Camille Teruel <camille.teruel@gmail.com>wrote:
On 7 juin 2013, at 11:19, Pharo Issue Tracker wrote:
A FogBugz case was edited by Stephane Ducasse.
Case ID: 10840 Title: WeakSet>>#size utterly broken Status: Resolved (Fix Review Needed) Category: Bug Project: Collection Area: Misc Priority: 3 - Must Fix Milestone: Pharo3.0: 30/03/2014 Assigned To: Camille Teruel
URL: https://pharo.fogbugz.com/f/cases/10840
Last message: Please can you send a mail to the mailing-list so that we all discuss this problems.
Hello everyone,
WeakSet>>#size is broken as demonstrated by the following snippet:
set := WeakSet new. set add: Object new. Smalltalk garbageCollect. set size. "answers 1 instead of 0"
That is because a WeakSet has no mean to know when its items get garbage collected, and thus the tally cannot be updated. So we need to override #size in Weak, I propose: size | counter | counter := 0. self do: [ :e | counter := counter + 1 ]. ^ counter But that it rather inefficient for such a simple query. Any proposition?
One way is to add WeakSets to FInalizationDependents and implement finalizeValues to update the size. But that's making work. A lazy solution might be better.
You *could*, I suppose, use the VM's count of the number of incremental and full GCs (alas there's no single count). I think this is too ugly and one would need something neater, but it would work like this:
size | newGCCount | newGCCount := self getGCCount. newGCCount == existingGCCount ifTrue: [^tally]. tally := self computeActualSIze. existingGCCount := newGCCount
where getGCCount ^(Smalltalk vmParameterAt: 7) + (Smalltalk vmParameterAt: 9)
There are obviously thread-safety issues here, but there are issues with the existing code and your suggestion too ;)
The most useful single count here would be a count that was incremented once in each GC that finalized one or more references.
Note also that for this to be reliable instances' record of the count would have to be invalidated on snapshot or start-up, which could entail an expensive allInstancesDo:.
Perhaps the class WeakSet should maintain its instances in (of course) a WeakSet and then add only itself to FinalizationDependents and enumerate the set on finalization?
I note that in the VisualWorks VM, objects that have references finalized are added to a queue so every instance that loses a reference gets sent finalize, instead of only dependents as in Squeak. This has its won problems (the VM must cope with a potentially large list) but I think it provides more convenient control.
The bug entry is: https://pharo.fogbugz.com/f/cases/10840
If you do not want to receive automatic notifications anymore, change your preferences in the Options screen. ( https://pharo.fogbugz.com/default.asp?pg=pgPrefs)
-- best, Eliot
-- best, Eliot
Hi eliot probably a really stupid question. But why count the number of finalization would have a link with the size? Is there an invariant that would mean that there is a one one relations between object in the set and their garbage collection. I mean if two objects of the same weakset are garbage collected will the finalization process runs twice I did not look at the class comment because may be it explains it and now I 'm dead. Stef
Here is a simple solution for you: set asArray size. But come on, why asking for weak container size??? What you gonna do with this information? Iterate over it? And besides, there is #slowSize. A size is left as it is. Because it is fast. There is no way to get accurate size of weak container either way, because at the very next message, GC can be triggered and some items turn garbage => size is invalid. So, it doesn't matters if you calculate size slow or fast, it will be always inaccurate. And issue is invalid. -- Best regards, Igor Stasenko.
On Fri, Jun 7, 2013 at 1:24 PM, Stéphane Ducasse <stephane.ducasse@inria.fr>wrote:
Hi eliot
probably a really stupid question. But why count the number of finalization would have a link with the size?
The size can only change when a) an element is added or removed, or b) when something is finalized. a) adjusts the tally, b) does not. So invalidating the size on b) means that it will always be recomputed if finalization affects the size.
Is there an invariant that would mean that there is a one one relations between object in the set and their garbage collection. I mean if two objects of the same weakset are garbage collected will the finalization process runs twice
No. Finalization runs once after each GC that finalizes something. i.e. the GC runs, and if it deletes one or more elements from any of the weak collections in the system it signals the finalization semaphore. So the finalization loop runs once for each GC that finalizes at least one weak collection. If a GC finalizes many weak collections the finalization loop still runs only once for the GC.
I did not look at the class comment because may be it explains it and now I 'm dead.
Stef
-- best, Eliot
thanks it would be so good to have such information in class comments. On Jun 10, 2013, at 5:38 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Fri, Jun 7, 2013 at 1:24 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote: Hi eliot
probably a really stupid question. But why count the number of finalization would have a link with the size?
The size can only change when a) an element is added or removed, or b) when something is finalized. a) adjusts the tally, b) does not. So invalidating the size on b) means that it will always be recomputed if finalization affects the size.
Is there an invariant that would mean that there is a one one relations between object in the set and their garbage collection. I mean if two objects of the same weakset are garbage collected will the finalization process runs twice
No. Finalization runs once after each GC that finalizes something. i.e. the GC runs, and if it deletes one or more elements from any of the weak collections in the system it signals the finalization semaphore. So the finalization loop runs once for each GC that finalizes at least one weak collection. If a GC finalizes many weak collections the finalization loop still runs only once for the GC.
I did not look at the class comment because may be it explains it and now I 'm dead.
Stef
-- best, Eliot
On 10 June 2013 05:38, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Fri, Jun 7, 2013 at 1:24 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Hi eliot
probably a really stupid question. But why count the number of finalization would have a link with the size?
The size can only change when a) an element is added or removed, or b) when something is finalized. a) adjusts the tally, b) does not. So invalidating the size on b) means that it will always be recomputed if finalization affects the size.
i correcting you b) when something is garbage collected.
Is there an invariant that would mean that there is a one one relations between object in the set and their garbage collection. I mean if two objects of the same weakset are garbage collected will the finalization process runs twice
No. Finalization runs once after each GC that finalizes something. i.e. the GC runs, and if it deletes one or more elements from any of the weak collections in the system it signals the finalization semaphore. So the finalization loop runs once for each GC that finalizes at least one weak collection. If a GC finalizes many weak collections the finalization loop still runs only once for the GC.
Except from the cases, when you put too many things into finalization process, producing much garbage, which causing another GC while running it.. Then it turns into self-supporting process: - running finalization process causing GC - GC causing running finalization process :)
I did not look at the class comment because may be it explains it and now I 'm dead.
Stef
-- best, Eliot
-- Best regards, Igor Stasenko.
participants (5)
-
Camille Teruel -
Eliot Miranda -
Henrik Johansen -
Igor Stasenko -
Stéphane Ducasse