[Pharo-project] Another finalization concern: error handling
Hello, here a situation, with which we can deal in more safer manner: Suppose you have a weak registry, populated by different objects and their executors. Now, when some of them died, a weak registry performs finalization. The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage. What you think, should we handle this more graciously? (Consider a following meta-code) WeakRegistry>>finalizeValues | executors | executors := self gatherExecutorsToRun. executors do: [:ex | [ ex finalize ] fork. ]. in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling. P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize. -- Best regards, Igor Stasenko AKA sig.
On 11 October 2010 14:40, Igor Stasenko <siguctua@gmail.com> wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | Â executors := self gatherExecutorsToRun.
 executors do: [:ex |   [ ex finalize ] fork.  ].
oh, and this gets even more complicated, if we keep supporting multiple finalizers per single object. :) executors do: [:ex | ex hasMultipleExecutors ifTrue: [ ex do: [ :eex | [ eex finalize ] fork ] ] ifFalse: [ [ ex finalize ] fork ].
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
-- Best regards, Igor Stasenko AKA sig.
On Mon, 11 Oct 2010, Igor Stasenko wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | executors := self gatherExecutorsToRun.
executors do: [:ex | [ ex finalize ] fork. ].
The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs. There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors. So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
Wouldn't it be better to use an exception handler? Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Levente, A similar discussion arose around Dolphin's event (#trigger*) mechanism. My recollection is that it was not fully addressed due to performance concerns. Forking and error handlers both have their costs. I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt. I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 9:51 AM To: Pharo Development Cc: The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling On Mon, 11 Oct 2010, Igor Stasenko wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | executors := self gatherExecutorsToRun.
executors do: [:ex | [ ex finalize ] fork. ].
The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs. There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors. So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
Wouldn't it be better to use an exception handler? Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
A similar discussion arose around Dolphin's event (#trigger*) mechanism. My recollection is that it was not fully addressed due to performance concerns. Forking and error handlers both have their costs. I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Smalltalk is not C. Try this: | file | file := StandardFileStream fileNamed: 'foo.txt'. file close. file primClose: (file instVarNamed: #fileID). "Those pesky plugins save us all the time. ;)" Levente
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 9:51 AM To: Pharo Development Cc: The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
On Mon, 11 Oct 2010, Igor Stasenko wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | executors := self gatherExecutorsToRun.
executors do: [:ex | [ ex finalize ] fork. ].
The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs.
There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors.
So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
Wouldn't it be better to use an exception handler?
Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Levente, Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources. Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections. This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety. Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem. Maybe you have made a strong argument for multiple executors; if so, I've missed it. Right now, it looks like a design flaw instead of a feature. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 10:24 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
A similar discussion arose around Dolphin's event (#trigger*) mechanism. My recollection is that it was not fully addressed due to performance concerns. Forking and error handlers both have their costs. I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Smalltalk is not C. Try this: | file | file := StandardFileStream fileNamed: 'foo.txt'. file close. file primClose: (file instVarNamed: #fileID). "Those pesky plugins save us all the time. ;)" Levente
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 9:51 AM To: Pharo Development Cc: The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
On Mon, 11 Oct 2010, Igor Stasenko wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | executors := self gatherExecutorsToRun.
executors do: [:ex | [ ex finalize ] fork. ].
The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs.
There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors.
So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
Wouldn't it be better to use an exception handler?
Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources. Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections. This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety. Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem.
Maybe you have made a strong argument for multiple executors; if so, I've missed it. Right now, it looks like a design flaw instead of a feature.
There's code that relies on multiple executors. Every object understands #toFinalizeSend:to:with:. When you use it, you expect that your finalizer will be evaluated when the object is garbage collected. Btw I offered a simple solution for cases like yours. The behavior of WeakRegistry >> #add:executor: can be easily made pluggable when an object has an executor in the registry. Levente P.S.: how will you use GSL knowing that it's license is GPL? Will you keep the code for yourself or release it under the GPL license?
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 10:24 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
A similar discussion arose around Dolphin's event (#trigger*) mechanism. My recollection is that it was not fully addressed due to performance concerns. Forking and error handlers both have their costs. I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Smalltalk is not C. Try this:
| file | file := StandardFileStream fileNamed: 'foo.txt'. file close. file primClose: (file instVarNamed: #fileID). "Those pesky plugins save us all the time. ;)"
Levente
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 9:51 AM To: Pharo Development Cc: The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
On Mon, 11 Oct 2010, Igor Stasenko wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | executors := self gatherExecutorsToRun.
executors do: [:ex | [ ex finalize ] fork. ].
The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs.
There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors.
So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
Wouldn't it be better to use an exception handler?
Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
GPL is a problem. It means that nothing I write on top of GSL gets out the door. However, I see no reason not to give others the same mix of capability and concern in the form of an interface to it. ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 10:59 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources. Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections. This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety. Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem.
Maybe you have made a strong argument for multiple executors; if so, I've missed it. Right now, it looks like a design flaw instead of a feature.
There's code that relies on multiple executors. Every object understands #toFinalizeSend:to:with:. When you use it, you expect that your finalizer will be evaluated when the object is garbage collected. Btw I offered a simple solution for cases like yours. The behavior of WeakRegistry >> #add:executor: can be easily made pluggable when an object has an executor in the registry. Levente P.S.: how will you use GSL knowing that it's license is GPL? Will you keep the code for yourself or release it under the GPL license?
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 10:24 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
A similar discussion arose around Dolphin's event (#trigger*) mechanism. My recollection is that it was not fully addressed due to performance concerns. Forking and error handlers both have their costs. I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Smalltalk is not C. Try this:
| file | file := StandardFileStream fileNamed: 'foo.txt'. file close. file primClose: (file instVarNamed: #fileID). "Those pesky plugins save us all the time. ;)"
Levente
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 9:51 AM To: Pharo Development Cc: The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
On Mon, 11 Oct 2010, Igor Stasenko wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | executors := self gatherExecutorsToRun.
executors do: [:ex | [ ex finalize ] fork. ].
The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs.
There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors.
So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
Wouldn't it be better to use an exception handler?
Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 11 October 2010 17:34, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Levente,
Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources. Â Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections. Â This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety. Â Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem.
Maybe you have made a strong argument for multiple executors; if so, I've missed it. Â Right now, it looks like a design flaw instead of a feature.
I second that. Knowing that VM/OS does the best to not pushish those who attempting to free already freed resource, doesn't means that we should be careless about it! OS tries the best to not give away a same file handle for newly opened file as recently closed file(s). malloc() tries the best to not give away the pointer to same memory region, which was recently freed. But obviously, a well-designed application should never rely on such behavior. -- Best regards, Igor Stasenko AKA sig.
On 11 October 2010 17:24, Levente Uzonyi <leves@elte.hu> wrote:
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
A similar discussion arose around Dolphin's event (#trigger*) mechanism. Â My recollection is that it was not fully addressed due to performance concerns. Â Forking and error handlers both have their costs. Â I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Smalltalk is not C. Try this:
| file | file := StandardFileStream fileNamed: 'foo.txt'. file close. file primClose: (file instVarNamed: #fileID). "Those pesky plugins save us all the time. ;)"
Don't let me starting again on this. You proposing to care about it in multiple various places, where we could fix it in a single one. It is like adding ifNotNil: test at each place you using setter, instead of putting a single ifNotNil: test inside a setter itself. try this: coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj := nil. Smalltalk garbageCollect. The above works silently only because ObjectFinalizer simply swallows any errors: ObjectFinalizer>>finalize "Finalize the resource associated with the receiver. This message should only be sent during the finalization process. There is NO garantuee that the resource associated with the receiver hasn't been free'd before so take care that you don't run into trouble - this all may happen with interrupt priority." [self value] on: Error do:[:ex| ex return]. now, replace this implementation with just self value and you'll see what will happen. Such behavior is completely unacceptable in terms of finding the bugs & problems in your code. -- Best regards, Igor Stasenko AKA sig.
On Mon, 11 Oct 2010, Igor Stasenko wrote:
On 11 October 2010 17:24, Levente Uzonyi <leves@elte.hu> wrote: On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
A similar discussion arose around Dolphin's event (#trigger*) mechanism. Â My recollection is that it was not fully addressed due to performance concerns. Â Forking and error handlers both have their costs. Â I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Smalltalk is not C. Try this:
| file | file := StandardFileStream fileNamed: 'foo.txt'. file close. file primClose: (file instVarNamed: #fileID). "Those pesky plugins save us all the time. ;)"
Don't let me starting again on this. You proposing to care about it in multiple various places, where we could fix it in a single one. It is like adding ifNotNil: test at each place you using setter, instead of putting a single ifNotNil: test inside a setter itself. try this: coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj := nil. Smalltalk garbageCollect. The above works silently only because ObjectFinalizer simply swallows any errors: ObjectFinalizer>>finalize "Finalize the resource associated with the receiver. This message should only be sent during the finalization process. There is NO garantuee that the resource associated with the receiver hasn't been free'd before so take care that you don't run into trouble - this all may happen with interrupt priority." [self value] on: Error do:[:ex| ex return]. now, replace this implementation with just self value and you'll see what will happen. Such behavior is completely unacceptable in terms of finding the bugs & problems in your code. Interrupting the finalization process is also a bad idea, so we need a better solution. Levente -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Sig, I'm not saying to leave the finalizer wide open to the wrath of errors from poorly coded objects. I am saying that we should strive for one finalizer per object (preferably the object itself) and that to get there, we need to make the registry thread safe and have some vm support. Failing the latter, the object can't be its own executor, but there still should be at most one executor per object. Any errors should be raised on the attempt to create the second executor. Arrange for that, and somebody somewhere will ask whether all of this should be thread safe. BTW, Dolphin treats #beFinalizable/#beUnfinalizable by manipulating a mask that controls the vm behavior - haven't looked in a while, but it must be part of the object header. An object can thus be marked as finalizable or not any number of times during its life; no objects are created thanks to the vm support. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 10:47 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On 11 October 2010 17:24, Levente Uzonyi <leves@elte.hu> wrote:
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Levente,
A similar discussion arose around Dolphin's event (#trigger*) mechanism. My recollection is that it was not fully addressed due to performance concerns. Forking and error handlers both have their costs. I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Smalltalk is not C. Try this:
| file | file := StandardFileStream fileNamed: 'foo.txt'. file close. file primClose: (file instVarNamed: #fileID). "Those pesky plugins save us all the time. ;)"
Don't let me starting again on this. You proposing to care about it in multiple various places, where we could fix it in a single one. It is like adding ifNotNil: test at each place you using setter, instead of putting a single ifNotNil: test inside a setter itself. try this: coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj := nil. Smalltalk garbageCollect. The above works silently only because ObjectFinalizer simply swallows any errors: ObjectFinalizer>>finalize "Finalize the resource associated with the receiver. This message should only be sent during the finalization process. There is NO garantuee that the resource associated with the receiver hasn't been free'd before so take care that you don't run into trouble - this all may happen with interrupt priority." [self value] on: Error do:[:ex| ex return]. now, replace this implementation with just self value and you'll see what will happen. Such behavior is completely unacceptable in terms of finding the bugs & problems in your code. -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote:
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Bill
I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about. You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:. Nor do I lose any sleep thinking of ways people could possibly abuse #become: Cheers, Henry
Henry, Ok, what valid use of multiple executors have I missed? Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Henrik Johansen [henrik.s.johansen@veloxit.no] Sent: Monday, October 11, 2010 11:23 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote:
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Bill
I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about. You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:. Nor do I lose any sleep thinking of ways people could possibly abuse #become: Cheers, Henry _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Henry,
Ok, what valid use of multiple executors have I missed?
I described it earlier how the AXAnnouncements project uses this feature. Levente
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Henrik Johansen [henrik.s.johansen@veloxit.no] Sent: Monday, October 11, 2010 11:23 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling
On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote:
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Bill
I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about. You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:. Nor do I lose any sleep thinking of ways people could possibly abuse #become:
Cheers, Henry _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Ok, I'll dig around for that and have a look. ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Levente Uzonyi [leves@elte.hu] Sent: Monday, October 11, 2010 12:31 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
Henry,
Ok, what valid use of multiple executors have I missed?
I described it earlier how the AXAnnouncements project uses this feature. Levente
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Henrik Johansen [henrik.s.johansen@veloxit.no] Sent: Monday, October 11, 2010 11:23 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling
On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote:
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Bill
I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about. You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:. Nor do I lose any sleep thinking of ways people could possibly abuse #become:
Cheers, Henry _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 11 October 2010 18:23, Henrik Johansen <henrik.s.johansen@veloxit.no> wrote:
On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote:
I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
Bill
I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about. You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:. Nor do I lose any sleep thinking of ways people could possibly abuse #become:
Henrik, in previous discussion i shown that multiple finalizers could be effectively replaced with multiple private weak registries. There is one good reason for that: do not let anyone else to do something 'good' with your private stuff. Because 'good' quite often turns out to be 'bad', once you leaking a reference to some critical object outside your model. See http://www.erights.org/index.html#SmartContracts multiple finalizers per object is not a product of a smart contract. Because it means that same object accessible by two different, unrelated frameworks, which then will attempt to race for its ownership as to 'who's gonna finalize it first', or 'who having right to remove it from weak registry etc'. More to that, in context of current discussion, i see that multiple finalizers adding a lot of complexity for implementing correct error handling.
Cheers, Henry _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
Meanwhile, i'll try to implement two test cases for WeakRegistryTest. One, should cover following: coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj := nil. Smalltalk garbageCollect. i.e somehow, user should be notified that there is an error during finalization. And second test case is to make sure that if one finalizer unable to complete due to error, the other ones (and finalization process itself) should continue running, skipping over errorneous finalizer. -- Best regards, Igor Stasenko AKA sig.
Sig, The Dolphin approach is to restart any of the finalizer, main, timer, idler threads (I *think* there is one more in a baseline image) any time they quit; an #ensure: block forks a new thread of the type that terminated. That way, whether they are taken down by an error doing what they are supposed to do or just by a stray #terminate or mistake with an attached debugger, the system rolls onward intact. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 12:14 PM To: Pharo-project@lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling Meanwhile, i'll try to implement two test cases for WeakRegistryTest. One, should cover following: coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj := nil. Smalltalk garbageCollect. i.e somehow, user should be notified that there is an error during finalization. And second test case is to make sure that if one finalizer unable to complete due to error, the other ones (and finalization process itself) should continue running, skipping over errorneous finalizer. -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 11 October 2010 19:28, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The Dolphin approach is to restart any of the finalizer, main, timer, idler threads (I *think* there is one more in a baseline image) any time they quit; an #ensure: block forks a new thread of the type that terminated. Â That way, whether they are taken down by an error doing what they are supposed to do or just by a stray #terminate or mistake with an attached debugger, the system rolls onward intact.
If it would be only about restarting, then its a piece of cake. We should make sure that all finalizations detected prior to error, did not get lost and are able to proceed as if nothing happens, except a single one, which failed with error.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 12:14 PM To: Pharo-project@lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
Meanwhile, i'll try to implement two test cases for WeakRegistryTest.
One, should cover following:
coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper.
obj := nil. Smalltalk garbageCollect.
i.e somehow, user should be notified that there is an error during finalization.
And second test case is to make sure that if one finalizer unable to complete due to error, the other ones (and finalization process itself) should continue running, skipping over errorneous finalizer.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
Sig, Understood, but are they all that different? The ones that were served ahead of the error should stay that way (proper recording of same is what needs attention), and those yet to be served, if it's done well, should be ready for the new thread to handle. Just something to consider. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 1:02 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On 11 October 2010 19:28, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The Dolphin approach is to restart any of the finalizer, main, timer, idler threads (I *think* there is one more in a baseline image) any time they quit; an #ensure: block forks a new thread of the type that terminated. That way, whether they are taken down by an error doing what they are supposed to do or just by a stray #terminate or mistake with an attached debugger, the system rolls onward intact.
If it would be only about restarting, then its a piece of cake. We should make sure that all finalizations detected prior to error, did not get lost and are able to proceed as if nothing happens, except a single one, which failed with error.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 12:14 PM To: Pharo-project@lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
Meanwhile, i'll try to implement two test cases for WeakRegistryTest.
One, should cover following:
coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper.
obj := nil. Smalltalk garbageCollect.
i.e somehow, user should be notified that there is an error during finalization.
And second test case is to make sure that if one finalizer unable to complete due to error, the other ones (and finalization process itself) should continue running, skipping over errorneous finalizer.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 11 October 2010 20:04, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
Understood, but are they all that different? Â The ones that were served ahead of the error should stay that way (proper recording of same is what needs attention), and those yet to be served, if it's done well, should be ready for the new thread to handle. Â Just something to consider.
Devil is always in details. The problem can be illustrated by following: | todoList | todoList := collection select: [ some criteria ]. collection removeAllFoundIn: todoList. todoList do: [ :each | each doSomething ]. now, if inside of #doSomething you got an error, a loop is interrupted, and if you restart the process, your todoList temp is gone, since it is local to particular context, which will be discarded. You also can't restore todoList by rescanning a collection, since you removed all elements, which met your criteria.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 1:02 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling
On 11 October 2010 19:28, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The Dolphin approach is to restart any of the finalizer, main, timer, idler threads (I *think* there is one more in a baseline image) any time they quit; an #ensure: block forks a new thread of the type that terminated. Â That way, whether they are taken down by an error doing what they are supposed to do or just by a stray #terminate or mistake with an attached debugger, the system rolls onward intact.
If it would be only about restarting, then its a piece of cake. We should make sure that all finalizations detected prior to error, did not get lost and are able to proceed as if nothing happens, except a single one, which failed with error.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 12:14 PM To: Pharo-project@lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
Meanwhile, i'll try to implement two test cases for WeakRegistryTest.
One, should cover following:
coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper.
obj := nil. Smalltalk garbageCollect.
i.e somehow, user should be notified that there is an error during finalization.
And second test case is to make sure that if one finalizer unable to complete due to error, the other ones (and finalization process itself) should continue running, skipping over errorneous finalizer.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
Sig, As a friend here: when did I say I'd remove all? Remove and process one at a time. I will admit to having a bias toward doing this with a single executor per object, but even if multiples turn out to have value (need to hunt down and read Levente's description of that), the same idea should apply. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 1:39 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On 11 October 2010 20:04, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
Understood, but are they all that different? The ones that were served ahead of the error should stay that way (proper recording of same is what needs attention), and those yet to be served, if it's done well, should be ready for the new thread to handle. Just something to consider.
Devil is always in details. The problem can be illustrated by following: | todoList | todoList := collection select: [ some criteria ]. collection removeAllFoundIn: todoList. todoList do: [ :each | each doSomething ]. now, if inside of #doSomething you got an error, a loop is interrupted, and if you restart the process, your todoList temp is gone, since it is local to particular context, which will be discarded. You also can't restore todoList by rescanning a collection, since you removed all elements, which met your criteria.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 1:02 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling
On 11 October 2010 19:28, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The Dolphin approach is to restart any of the finalizer, main, timer, idler threads (I *think* there is one more in a baseline image) any time they quit; an #ensure: block forks a new thread of the type that terminated. That way, whether they are taken down by an error doing what they are supposed to do or just by a stray #terminate or mistake with an attached debugger, the system rolls onward intact.
If it would be only about restarting, then its a piece of cake. We should make sure that all finalizations detected prior to error, did not get lost and are able to proceed as if nothing happens, except a single one, which failed with error.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 12:14 PM To: Pharo-project@lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
Meanwhile, i'll try to implement two test cases for WeakRegistryTest.
One, should cover following:
coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper.
obj := nil. Smalltalk garbageCollect.
i.e somehow, user should be notified that there is an error during finalization.
And second test case is to make sure that if one finalizer unable to complete due to error, the other ones (and finalization process itself) should continue running, skipping over errorneous finalizer.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 11 October 2010 21:07, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
As a friend here: when did I say I'd remove all? Â Remove and process one at a time. Â I will admit to having a bias toward doing this with a single executor per object, but even if multiples turn out to have value (need to hunt down and read Levente's description of that), the same idea should apply.
Removing and processing one at a time could do the job, but it will be very inefficient (especially on VMs which not support new finalization). Because: - you have to enter critical section for scanning valueDictionary - you then have to leave critical section for sending a single #finalize - then you should start over and look for next element with nil key (if any) So, as i said, devil in details.
Bill
-- Best regards, Igor Stasenko AKA sig.
On Mon, 11 Oct 2010, Igor Stasenko wrote:
On 11 October 2010 21:07, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote: Sig,
As a friend here: when did I say I'd remove all? Â Remove and process one at a time. Â I will admit to having a bias toward doing this with a single executor per object, but even if multiples turn out to have value (need to hunt down and read Levente's description of that), the same idea should apply.
Removing and processing one at a time could do the job, but it will be very inefficient (especially on VMs which not support new finalization). Because: - you have to enter critical section for scanning valueDictionary - you then have to leave critical section for sending a single #finalize - then you should start over and look for next element with nil key (if any) So, as i said, devil in details. You can select the executors to a collection and finalize them later, so you don't have to restart the scanning of valueDictionary. Levente
Bill
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Sig, The most important words in there are "critical section." Carry on :) Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 2:36 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On 11 October 2010 21:07, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
As a friend here: when did I say I'd remove all? Remove and process one at a time. I will admit to having a bias toward doing this with a single executor per object, but even if multiples turn out to have value (need to hunt down and read Levente's description of that), the same idea should apply.
Removing and processing one at a time could do the job, but it will be very inefficient (especially on VMs which not support new finalization). Because: - you have to enter critical section for scanning valueDictionary - you then have to leave critical section for sending a single #finalize - then you should start over and look for next element with nil key (if any) So, as i said, devil in details.
Bill
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 11 October 2010 22:49, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The most important words in there are "critical section." Â Carry on :)
Oh, please. This is not too hard to code. My mind rolling around following choice(s) (there may be others i don't see). What would be a proper way to handle error during #finalize. [ executor finalize ] on: Error do: [:ex | self handleFinalizationError: ex "where self is registry" ]. or: [ executor finalize ] on: Error do: [:ex | executor handleFinalizationError: ex ]. of course, i should catch this error in test, so i can verify that: a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors). Also, i used #ensure: and #ifCurtailed: but i tend to forget where they are applicable and how. So, little help in this regard will be wellcome.
Bill
-- Best regards, Igor Stasenko AKA sig.
Sig, How hard it is or isn't is not nearly as important as your tackling it and doing so with the right tools. So again I say, carry on :) On self vs. executor as the receiver of #handleFinalizationError:, I might lean toward "self" as it will be closer to the system. There is probably little the executor can do, and it either missed or raised the error in the first place, right? I *think* the most you are going to to do is log that something went wrong, but I could be missing something. #ensure: attempts to evaluate the argument block whenever the receiver of it is completed (modulo some ominous sounding questions raised in the past months to a year??). #ifCurtailed: should be similar, but evaluating the argument/block only on abnormal (including, I think, ^) exit of the block. Does that help at all? Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 4:17 PM To: Pharo-project@lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling On 11 October 2010 22:49, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The most important words in there are "critical section." Carry on :)
Oh, please. This is not too hard to code. My mind rolling around following choice(s) (there may be others i don't see). What would be a proper way to handle error during #finalize. [ executor finalize ] on: Error do: [:ex | self handleFinalizationError: ex "where self is registry" ]. or: [ executor finalize ] on: Error do: [:ex | executor handleFinalizationError: ex ]. of course, i should catch this error in test, so i can verify that: a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors). Also, i used #ensure: and #ifCurtailed: but i tend to forget where they are applicable and how. So, little help in this regard will be wellcome.
Bill
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 12 October 2010 00:32, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
How hard it is or isn't is not nearly as important as your tackling it and doing so with the right tools. Â So again I say, carry on :)
On self vs. executor as the receiver of #handleFinalizationError:, I might lean toward "self" as it will be closer to the system. Â There is probably little the executor can do, and it either missed or raised the error in the first place, right? Â I *think* the most you are going to to do is log that something went wrong, but I could be missing something.
I'm not sure about that .
From other size, a finalizer could better know why error can occur and how to handle it graciously, if its possible, and not just showing 'you suck' at the center of screen ;)
Because weak registry can tell even less about nature of error, because it propably knows nothing about its finalizers, except that they understand #finalize message.
#ensure: attempts to evaluate the argument block whenever the receiver of it is completed (modulo some ominous sounding questions raised in the past months to a year??). Â #ifCurtailed: should be similar, but evaluating the argument/block only on abnormal (including, I think, ^) exit of the block.
Does that help at all?
Yes. Thanks. I now remember semantics of it. Been there when dealt with Process>>terminate voodo.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 4:17 PM To: Pharo-project@lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
On 11 October 2010 22:49, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The most important words in there are "critical section." Â Carry on :)
Oh, please. This is not too hard to code.
My mind rolling around following choice(s) Â (there may be others i don't see). What would be a proper way to handle error during #finalize.
[ executor finalize ] on: Error do: [:ex |  self handleFinalizationError: ex  "where self is registry"  ].
or:
[ executor finalize ] on: Error do: [:ex |  executor handleFinalizationError: ex  ].
of course, i should catch this error in test, so i can verify that:
a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors).
Also, i used #ensure: and #ifCurtailed: but i tend to forget where they are applicable and how. So, little help in this regard will be wellcome.
Bill
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
Sig, You seem to be doing a good job of this, so if you think I'm off, feel free to prove it. The executors (in Dolphin, the objects themselves) probably know more than the registry about how to handle errors, and (it seems to me) either generated them or could have readily caught them if there were something different to do based on the information. Maybe the crux of the matter is as follows: are you thinking of an executor class with subclasses that implement #handleFinalizationError: (a hook), or is it more "this happened, thought you might like to know." I think I read it more as the latter, probably assuming that #finalize would retry if that could reasonably be expected to help. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 6:35 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Another finalization concern: error handling On 12 October 2010 00:32, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
How hard it is or isn't is not nearly as important as your tackling it and doing so with the right tools. So again I say, carry on :)
On self vs. executor as the receiver of #handleFinalizationError:, I might lean toward "self" as it will be closer to the system. There is probably little the executor can do, and it either missed or raised the error in the first place, right? I *think* the most you are going to to do is log that something went wrong, but I could be missing something.
I'm not sure about that .
From other size, a finalizer could better know why error can occur and how to handle it graciously, if its possible, and not just showing 'you suck' at the center of screen ;)
Because weak registry can tell even less about nature of error, because it propably knows nothing about its finalizers, except that they understand #finalize message.
#ensure: attempts to evaluate the argument block whenever the receiver of it is completed (modulo some ominous sounding questions raised in the past months to a year??). #ifCurtailed: should be similar, but evaluating the argument/block only on abnormal (including, I think, ^) exit of the block.
Does that help at all?
Yes. Thanks. I now remember semantics of it. Been there when dealt with Process>>terminate voodo.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 4:17 PM To: Pharo-project@lists.gforge.inria.fr; The general-purpose Squeak developers list Subject: Re: [Pharo-project] Another finalization concern: error handling
On 11 October 2010 22:49, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The most important words in there are "critical section." Carry on :)
Oh, please. This is not too hard to code.
My mind rolling around following choice(s) (there may be others i don't see). What would be a proper way to handle error during #finalize.
[ executor finalize ] on: Error do: [:ex | self handleFinalizationError: ex "where self is registry" ].
or:
[ executor finalize ] on: Error do: [:ex | executor handleFinalizationError: ex ].
of course, i should catch this error in test, so i can verify that:
a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors).
Also, i used #ensure: and #ifCurtailed: but i tend to forget where they are applicable and how. So, little help in this regard will be wellcome.
Bill
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Den 11. okt. 2010 kl. 22:17 skrev Igor Stasenko <siguctua@gmail.com>:
Also, i used #ensure: and #ifCurtailed: but i tend to forget where they are applicable and how. So, little help in this regard will be wellcome.
Ensure: block always execute. IfCurtailed blocks only run if an error is encountered, or the thread is terminated. Cheers, Henry
On Oct 11, 2010, at 10:17 18PM, Igor Stasenko wrote:
On 11 October 2010 22:49, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The most important words in there are "critical section." Carry on :)
Oh, please. This is not too hard to code.
My mind rolling around following choice(s) (there may be others i don't see). What would be a proper way to handle error during #finalize.
[ executor finalize ] on: Error do: [:ex | self handleFinalizationError: ex "where self is registry" ].
or:
[ executor finalize ] on: Error do: [:ex | executor handleFinalizationError: ex ].
of course, i should catch this error in test, so i can verify that:
a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors).
I agree it should be the executor doing handling. However, if you get to this point, the executor already had the chance to do so in the finalize method itself, so I don't really see the point of sending it back.. Personally, I feel the only promises FinalizationRegistry should make wrt. to errors during finalization, is it should never lead to: - termination of the (or a) finalization process, - discarding another finalization action. IE I am perfectly fine with an error here being thrown in your face (hopefully in a somewhat debuggable way, unlike what happened in your first example when removing the error-swallowing), as you've already passed the point a correction should be made, and there's nothing more intelligible to do than making sure everything keeps working, and inform of the error. (Pretty sure the error swallowing was a quick hack for keeping everything working in the first place ;) ) Cheers, Henry
On 12 October 2010 14:25, Henrik Johansen <henrik.s.johansen@veloxit.no> wrote:
On Oct 11, 2010, at 10:17 18PM, Igor Stasenko wrote:
On 11 October 2010 22:49, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The most important words in there are "critical section." Â Carry on :)
Oh, please. This is not too hard to code.
My mind rolling around following choice(s) Â (there may be others i don't see). What would be a proper way to handle error during #finalize.
[ executor finalize ] on: Error do: [:ex |  self handleFinalizationError: ex  "where self is registry" ].
or:
[ executor finalize ] on: Error do: [:ex | Â executor handleFinalizationError: ex ].
of course, i should catch this error in test, so i can verify that:
a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors).
I agree it should be the executor doing handling. However, if you get to this point, the executor already had the chance to do so in the finalize method itself, so I don't really see the point of sending it back..
well, its easier to add a default error handler in Object class, so any executor will have a default handler, and then some specific one may choose to override it, without putting a boilerplate of [ ] on: .. do: [] in every #finalize method. Besides, its natural , when you writing a code, you always expect that it will work w/o errors, and even if it fails, then in some concrete place, which means you usually not wrapping whole thing with exception handler. This leaves a chance to make another mistakes, like 'hey, despite i put error handler in my #finalize, something is still hangs finalization process'.
Personally, I feel the only promises FinalizationRegistry should make wrt. to errors during finalization, is it should never lead to: - termination of the (or a) finalization process, - discarding another finalization action.
Yes this is the main intent.
IE I am perfectly fine with an error here being thrown in your face (hopefully in a somewhat debuggable way, unlike what happened in your first example when removing the error-swallowing), as you've already passed the point a correction should be made, and there's nothing more intelligible to do than making sure everything keeps working, and inform of the error.
(Pretty sure the error swallowing was a quick hack for keeping everything working in the first place ;) )
What i like in doing #finalize in forked process, that it doesn't matter where you screwed up: in finalization code or even in error handling code, you still don't have a chance to damage the system. And moreover, you can open debugger and debug it. Spice must flooow :)
Cheers, Henry _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
On Oct 12, 2010, at 2:00 42PM, Igor Stasenko wrote:
On 12 October 2010 14:25, Henrik Johansen <henrik.s.johansen@veloxit.no> wrote:
On Oct 11, 2010, at 10:17 18PM, Igor Stasenko wrote:
On 11 October 2010 22:49, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The most important words in there are "critical section." Carry on :)
Oh, please. This is not too hard to code.
My mind rolling around following choice(s) (there may be others i don't see). What would be a proper way to handle error during #finalize.
[ executor finalize ] on: Error do: [:ex | self handleFinalizationError: ex "where self is registry" ].
or:
[ executor finalize ] on: Error do: [:ex | executor handleFinalizationError: ex ].
of course, i should catch this error in test, so i can verify that:
a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors).
I agree it should be the executor doing handling. However, if you get to this point, the executor already had the chance to do so in the finalize method itself, so I don't really see the point of sending it back..
well, its easier to add a default error handler in Object class, so any executor will have a default handler, and then some specific one may choose to override it, without putting a boilerplate of [ ] on: .. do: [] in every #finalize method. Besides, its natural , when you writing a code, you always expect that it will work w/o errors, and even if it fails, then in some concrete place, which means you usually not wrapping whole thing with exception handler. This leaves a chance to make another mistakes, like 'hey, despite i put error handler in my #finalize, something is still hangs finalization process'.
At which point I'd go "Fix the damn handlers in the finalize method" rather than add more indirection ;) KISS. You still have to write the handling if you do encounter an error, whether that initial error is #dnu handleFinalizationError:, or the actual error is of little consequence. What we want away from, is the default case now where an unhandled error is assumed to be inconsequential and simply skipped over by default
Personally, I feel the only promises FinalizationRegistry should make wrt. to errors during finalization, is it should never lead to: - termination of the (or a) finalization process, - discarding another finalization action.
Yes this is the main intent.
IE I am perfectly fine with an error here being thrown in your face (hopefully in a somewhat debuggable way, unlike what happened in your first example when removing the error-swallowing), as you've already passed the point a correction should be made, and there's nothing more intelligible to do than making sure everything keeps working, and inform of the error.
(Pretty sure the error swallowing was a quick hack for keeping everything working in the first place ;) )
What i like in doing #finalize in forked process, that it doesn't matter where you screwed up: in finalization code or even in error handling code, you still don't have a chance to damage the system. And moreover, you can open debugger and debug it.
Spice must flooow :)
How about something like: (warning, untested code) WeakArray >> finalizationProcess ^[[[true] whileTrue: [FinalizationSemaphore wait. FinalizationLock critical: [FinalizationDependents do: [:weakDependent | weakDependent ifNotNil: [weakDependent finalizeValues]]]]] ifCurtailed: [self startFinalizationProcess]] newProcess priority: Processor userInterruptPriority WeakArray >>startFinalizationProcess FinalizationSemaphore := Smalltalk specialObjectsArray at: 42. FinalizationDependents ifNil: [FinalizationDependents := WeakArray new: 10]. FinalizationLock := Semaphore forMutualExclusion. FinalizationProcess := self finalizationProcess. FinalizationProcess resume WeakArray >>restartFinalizationProcess "Killing the current Finalization process will start a new one" FinalizationProcess terminate WeakRegistry finalizeValues: snip self finalize: finiObjects WeakArray >> finalize: finalizationObjects [[finalizationObjects isEmpty] whileFalse: [finalizationObjects removeFirst finalize]] on: Error do: [:error | self finalize: (finalizationObjects). error pass]. The last has some issues of course, if more than one error is encountered, the first ones won't be passed if the last ones are terminated. Changing it to fork of the error block would mean having to change finalizationObjects to something that is threadsafe rather than OrderedCollection though. Cheers, Henry
On 12 October 2010 15:33, Henrik Johansen <henrik.s.johansen@veloxit.no> wrote:
On Oct 12, 2010, at 2:00 42PM, Igor Stasenko wrote:
On 12 October 2010 14:25, Henrik Johansen <henrik.s.johansen@veloxit.no> wrote:
On Oct 11, 2010, at 10:17 18PM, Igor Stasenko wrote:
On 11 October 2010 22:49, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Sig,
The most important words in there are "critical section." Â Carry on :)
Oh, please. This is not too hard to code.
My mind rolling around following choice(s) Â (there may be others i don't see). What would be a proper way to handle error during #finalize.
[ executor finalize ] on: Error do: [:ex |  self handleFinalizationError: ex  "where self is registry" ].
or:
[ executor finalize ] on: Error do: [:ex | Â executor handleFinalizationError: ex ].
of course, i should catch this error in test, so i can verify that:
a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors).
I agree it should be the executor doing handling. However, if you get to this point, the executor already had the chance to do so in the finalize method itself, so I don't really see the point of sending it back..
well, Â its easier to add a default error handler in Object class, so any executor will have a default handler, and then some specific one may choose to override it, without putting a boilerplate of [ ] on: .. do: [] in every #finalize method. Besides, its natural , when you writing a code, you always expect that it will work w/o errors, and even if it fails, then in some concrete place, which means you usually not wrapping whole thing with exception handler. This leaves a chance to make another mistakes, like 'hey, despite i put error handler in my #finalize, something is still hangs finalization process'.
At which point I'd go "Fix the damn handlers in the finalize method" rather than add more indirection ;) KISS. You still have to write the handling if you do encounter an error, whether that initial error is #dnu handleFinalizationError:, or the actual error is of little consequence. What we want away from, is the default case now where an unhandled error is assumed to be inconsequential and simply skipped over by default
Personally, I feel the only promises FinalizationRegistry should make wrt. to errors during finalization, is it should never lead to: - termination of the (or a) finalization process, - discarding another finalization action.
Yes this is the main intent.
IE I am perfectly fine with an error here being thrown in your face (hopefully in a somewhat debuggable way, unlike what happened in your first example when removing the error-swallowing), as you've already passed the point a correction should be made, and there's nothing more intelligible to do than making sure everything keeps working, and inform of the error.
(Pretty sure the error swallowing was a quick hack for keeping everything working in the first place ;) )
What i like in doing #finalize in forked process, that it doesn't matter where you screwed up: in finalization code or even in error handling code, you still don't have a chance to damage the system. And moreover, you can open debugger and debug it.
Spice must flooow :)
How about something like: (warning, untested code)
WeakArray >> finalizationProcess
    ^[[[true] whileTrue:         [FinalizationSemaphore wait.         FinalizationLock critical:             [FinalizationDependents do:                 [:weakDependent |                 weakDependent ifNotNil:                     [weakDependent finalizeValues]]]]]         ifCurtailed: [self startFinalizationProcess]] newProcess priority:  Processor userInterruptPriority
WeakArray >>startFinalizationProcess
    FinalizationSemaphore := Smalltalk specialObjectsArray at: 42.     FinalizationDependents ifNil: [FinalizationDependents := WeakArray new: 10].     FinalizationLock := Semaphore forMutualExclusion.     FinalizationProcess := self finalizationProcess.     FinalizationProcess resume
WeakArray >>restartFinalizationProcess     "Killing the current Finalization process will start a new one"     FinalizationProcess terminate
WeakRegistry finalizeValues:
snip self finalize: finiObjects
WeakArray >> finalize: finalizationObjects     [[finalizationObjects isEmpty]         whileFalse: [finalizationObjects removeFirst finalize]]         on: Error         do: [:error |             self finalize: (finalizationObjects).             error pass].
The last has some issues of course, if more than one error is encountered, the first ones won't be passed if the last ones are terminated. Changing it to fork of the error block would mean having to change finalizationObjects to something that is threadsafe rather than OrderedCollection though.
With new finalization, i got a list of finalizers i.e first -> next -> next lists is much easier to manipulate in thread-safe manner. But i still have to care about old ways... pfff :)
Cheers, Henry _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
On 11 October 2010 16:51, Levente Uzonyi <leves@elte.hu> wrote:
On Mon, 11 Oct 2010, Igor Stasenko wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | Â executors := self gatherExecutorsToRun.
 executors do: [:ex |   [ ex finalize ] fork.  ].
The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs.
There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors.
err no. valuesDictionary are held privately by weak registry. and any access to it possible only through instance of weak registry, which always using protect: [].
So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block
agreed on this one. But weak registry lost this feature before my intervention: This is a code from Squeak 4.1. image: finalizeValues "Some of our elements may have gone away. Look for those and activate the associated executors." self protected: [ valueDictionary finalizeValues ] ul 2/22/2010 14:23 · finalization · 2 implementors · in no change set · So, i thought, it not a big sacrifice, besides it runs faster. And to my thinking, the code which needs to manipulate weak registry during finalization smells badly.
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
Wouldn't it be better to use an exception handler?
Well, its a bit complicated, since what i want is to: - report an error - but continue run #finalize for rest executors or: - report an error (stop finalization process), then - continue running for next executor , if user press 'abandon' - or continue with running fixed code If i'm not mistaken, currently, if you put halt inside a finalization process (like in some #finalize method) and then abandon it, you won't have finalization process in system anymore, because it will be terminated.
Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
On Mon, 11 Oct 2010, Igor Stasenko wrote:
On 11 October 2010 16:51, Levente Uzonyi <leves@elte.hu> wrote: On Mon, 11 Oct 2010, Igor Stasenko wrote:
Hello,
here a situation, with which we can deal in more safer manner:
Suppose you have a weak registry, populated by different objects and their executors.
Now, when some of them died, a weak registry performs finalization.
The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage.
What you think, should we handle this more graciously?
(Consider a following meta-code)
WeakRegistry>>finalizeValues | executors | Â executors := self gatherExecutorsToRun.
 executors do: [:ex |   [ ex finalize ] fork.  ].
The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs.
There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors.
err no. valuesDictionary are held privately by weak registry. and any access to it possible only through instance of weak registry, which always using protect: []. Hm. You missed the point. When someone sends #remove: to the WeakRegistry, it will send #removeKey:ifAbsent to valuesDictionary. If valuesDictionary has an association in the removed element's chain which key was garbage collected, then it will be finalized. (See WeakKeyDictionary >> #removeKey:ifAbsent:). Since the currently installed finalizer is #finalizeValues, the finalization will be done by a process other than the finalization process.
So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block
agreed on this one. But weak registry lost this feature before my intervention: This is a code from Squeak 4.1. image: finalizeValues "Some of our elements may have gone away. Look for those and activate the associated executors." self protected: [ valueDictionary finalizeValues ] ul 2/22/2010 14:23 ËË finalization ËË 2 implementors ËË in no change set ËË So, i thought, it not a big sacrifice, besides it runs faster. And to my thinking, the code which needs to manipulate weak registry during finalization smells badly. I was talking about the code in the Squeak 4.2 code.
in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling.
Wouldn't it be better to use an exception handler?
Well, its a bit complicated, since what i want is to: - report an error - but continue run #finalize for rest executors or: - report an error (stop finalization process), then - continue running for next executor , if user press 'abandon' - or continue with running fixed code If i'm not mistaken, currently, if you put halt inside a finalization process (like in some #finalize method) and then abandon it, you won't have finalization process in system anymore, because it will be terminated. That's right. I more like the first version, because not all systems are interactive. Stopping the finalization process for a long time cause a lot of problems. Btw, WeakArray >> #finalizationProcess has a similar issue. Levente
Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Dolphin has all of its required processes rigged to restart themselves if terminated; we must follow that lead. As far as getting error information from one thread to another, in one case I grab a callstack (just the This>>that, That>>this text with line feeds) and capture it to be included as text with an error signaled from another thread that was blocked while the offending thread was doing its thing. It's not perfect, but it at least shows something about what the system was trying to do, and most importantly, that there was an error vs. just ignoring it. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko [siguctua@gmail.com] Sent: Monday, October 11, 2010 10:30 AM To: The general-purpose Squeak developers list; Pharo Development Subject: Re: [Pharo-project] [squeak-dev] Re: Another finalization concern: error handling
Wouldn't it be better to use an exception handler?
Well, its a bit complicated, since what i want is to: - report an error - but continue run #finalize for rest executors or: - report an error (stop finalization process), then - continue running for next executor , if user press 'abandon' - or continue with running fixed code If i'm not mistaken, currently, if you put halt inside a finalization process (like in some #finalize method) and then abandon it, you won't have finalization process in system anymore, because it will be terminated.
Levente
P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
participants (4)
-
Henrik Johansen -
Igor Stasenko -
Levente Uzonyi -
Schwab,Wilhelm K