[Pharo-project] Should a serializer do something in particular with weak references?
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? what could be the impact of materializing such weak references in a new image for example? Thanks your sharing your thoughts. -- Mariano http://marianopeck.wordpress.com
Good question. I *think* the answer is to take the referenced objects across as one normally would, and then wrap them in a weak collection (if that is what was serialized). If one had only a weak collection holding the objects, they will be short-lived. However, something like a registry should be able to save/re-loaded, probably much like Dolphin's STBProxies: the effect of serializing the weak collection includes running some code to recreate conditions as needed. Looking forward to a lively debate... Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Mariano Martinez Peck [marianopeck@gmail.com] Sent: Tuesday, September 20, 2011 1:38 PM To: Pharo Development Subject: [Pharo-project] Should a serializer do something in particular with weak references? Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? what could be the impact of materializing such weak references in a new image for example? Thanks your sharing your thoughts. -- Mariano http://marianopeck.wordpress.com
Hmm, storing (and then restoring) weak containers as they are , i don't see anything special which must be done there. The special logic how to deal with weak containers are still remains same, as long as you correctly restoring the object, and as long as this logic is correctly written. So, i cannot see what Fuel needs to do special with weak containers. You may need to deal with storing instances of WeakRegistry, since it registers itself into WeakDependents array, which is scanned for finalization. So, you have to make sure that if WeakRegistry instance was registered in weak-dependents array, then you should also register it when restoring it back. (and if it was not registered - then of course not register). But WeakRegistry is a regular object (not a weak container by itself, and only indirectly it using weak containers by delegation). On 20 September 2011 20:55, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Good question. Â I *think* the answer is to take the referenced objects across as one normally would, and then wrap them in a weak collection (if that is what was serialized). Â If one had only a weak collection holding the objects, they will be short-lived. Â However, something like a registry should be able to save/re-loaded, probably much like Dolphin's STBProxies: the effect of serializing the weak collection includes running some code to recreate conditions as needed.
Looking forward to a lively debate...
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Mariano Martinez Peck [marianopeck@gmail.com] Sent: Tuesday, September 20, 2011 1:38 PM To: Pharo Development Subject: [Pharo-project] Should a serializer do something in particular with   weak references?
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? Â what could be the impact of materializing such weak references in a new image for example?
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
On Tue, Sep 20, 2011 at 10:37 PM, Igor Stasenko <siguctua@gmail.com> wrote:
Hmm, storing (and then restoring) weak containers as they are , i don't see anything special which must be done there.
The special logic how to deal with weak containers are still remains same, as long as you correctly restoring the object, and as long as this logic is correctly written. So, i cannot see what Fuel needs to do special with weak containers.
Yes, we thoght the same. But we wanted to ask just to be sure... Bill said the same, so we all agree :)
You may need to deal with storing instances of WeakRegistry, since it registers itself into WeakDependents array, which is scanned for finalization. So, you have to make sure that if WeakRegistry instance was registered in weak-dependents array, then you should also register it when restoring it back. (and if it was not registered - then of course not register).
But WeakRegistry is a regular object (not a weak container by itself, and only indirectly it using weak containers by delegation).
Yes, thanks for the pointer.
On 20 September 2011 20:55, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Good question. I *think* the answer is to take the referenced objects across as one normally would, and then wrap them in a weak collection (if that is what was serialized). If one had only a weak collection holding the objects, they will be short-lived. However, something like a registry should be able to save/re-loaded, probably much like Dolphin's STBProxies: the effect of serializing the weak collection includes running some code to recreate conditions as needed.
Looking forward to a lively debate...
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [ pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Mariano Martinez Peck [marianopeck@gmail.com] Sent: Tuesday, September 20, 2011 1:38 PM To: Pharo Development Subject: [Pharo-project] Should a serializer do something in particular with weak references?
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? what could be the impact of materializing such weak references in a new image for example?
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
-- Mariano http://marianopeck.wordpress.com
On 20 September 2011 22:43, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
On Tue, Sep 20, 2011 at 10:37 PM, Igor Stasenko <siguctua@gmail.com> wrote:
Hmm, storing (and then restoring) weak containers as they are , i don't see anything special which must be done there.
The special logic how to deal with weak containers are still remains same, as long as you correctly restoring the object, and as long as this logic is correctly written. So, i cannot see what Fuel needs to do special with weak containers.
Yes, we thoght the same. But we wanted to ask just to be sure... Bill said the same, so we all agree :)
You may need to deal with storing instances of WeakRegistry, since it registers itself into WeakDependents array, which is scanned for finalization. So, you have to make sure that if WeakRegistry instance was registered in weak-dependents array, then you should also register it when restoring it back. (and if it was not registered - then of course not register).
But WeakRegistry is a regular object (not a weak container by itself, and only indirectly it using weak containers by delegation).
Yes, thanks for the pointer.
if you look at the problem in wider perspective, then right question to ask, is how to deal with objects which are register themselves in global facilities, like weak finalization, startup/shutdown lists etc. For those objects you need to have special (de)serialization cases. But for objects with particular format (weak) you don't have to do anything special.
On 20 September 2011 20:55, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Good question. Â I *think* the answer is to take the referenced objects across as one normally would, and then wrap them in a weak collection (if that is what was serialized). Â If one had only a weak collection holding the objects, they will be short-lived. Â However, something like a registry should be able to save/re-loaded, probably much like Dolphin's STBProxies: the effect of serializing the weak collection includes running some code to recreate conditions as needed.
Looking forward to a lively debate...
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Mariano Martinez Peck [marianopeck@gmail.com] Sent: Tuesday, September 20, 2011 1:38 PM To: Pharo Development Subject: [Pharo-project] Should a serializer do something in particular with   weak references?
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? Â what could be the impact of materializing such weak references in a new image for example?
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
On Tue, Sep 20, 2011 at 10:51 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 20 September 2011 22:43, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
On Tue, Sep 20, 2011 at 10:37 PM, Igor Stasenko <siguctua@gmail.com>
wrote:
Hmm, storing (and then restoring) weak containers as they are , i don't see anything special which must be done there.
The special logic how to deal with weak containers are still remains same, as long as you correctly restoring the object, and as long as this logic is correctly written. So, i cannot see what Fuel needs to do special with weak containers.
Yes, we thoght the same. But we wanted to ask just to be sure... Bill said the same, so we all agree :)
You may need to deal with storing instances of WeakRegistry, since it registers itself into WeakDependents array, which is scanned for finalization. So, you have to make sure that if WeakRegistry instance was registered in weak-dependents array, then you should also register it when restoring it back. (and if it was not registered - then of course not register).
But WeakRegistry is a regular object (not a weak container by itself, and only indirectly it using weak containers by delegation).
Yes, thanks for the pointer.
if you look at the problem in wider perspective, then right question to ask, is how to deal with objects which are register themselves in global facilities, like weak finalization, startup/shutdown lists etc. For those objects you need to have special (de)serialization cases. But for objects with particular format (weak) you don't have to do anything special.
This is easy: you implement #fuelAfterMaterialization and do whatever you want.
On 20 September 2011 20:55, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Good question. I *think* the answer is to take the referenced objects across as one normally would, and then wrap them in a weak collection
(if
that is what was serialized). If one had only a weak collection holding the objects, they will be short-lived. However, something like a registry should be able to save/re-loaded, probably much like Dolphin's STBProxies: the effect of serializing the weak collection includes running some code to recreate conditions as needed.
Looking forward to a lively debate...
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Mariano Martinez Peck [marianopeck@gmail.com] Sent: Tuesday, September 20, 2011 1:38 PM To: Pharo Development Subject: [Pharo-project] Should a serializer do something in particular with weak references?
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? what could be the impact of materializing such weak references in a new image for example?
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
-- Mariano http://marianopeck.wordpress.com
2011/9/20 Igor Stasenko <siguctua@gmail.com>:
On 20 September 2011 22:43, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
On Tue, Sep 20, 2011 at 10:37 PM, Igor Stasenko <siguctua@gmail.com> wrote:
Hmm, storing (and then restoring) weak containers as they are , i don't see anything special which must be done there.
The special logic how to deal with weak containers are still remains same, as long as you correctly restoring the object, and as long as this logic is correctly written. So, i cannot see what Fuel needs to do special with weak containers.
Yes, we thoght the same. But we wanted to ask just to be sure... Bill said the same, so we all agree :)
You may need to deal with storing instances of WeakRegistry, since it registers itself into WeakDependents array, which is scanned for finalization. So, you have to make sure that if WeakRegistry instance was registered in weak-dependents array, then you should also register it when restoring it back. (and if it was not registered - then of course not register).
But WeakRegistry is a regular object (not a weak container by itself, and only indirectly it using weak containers by delegation).
Yes, thanks for the pointer.
if you look at the problem in wider perspective, then right question to ask, is how to deal with objects which are register themselves in global facilities, like weak finalization, startup/shutdown lists etc. For those objects you need to have special (de)serialization cases. But for objects with particular format (weak) you don't have to do anything special.
Except that you must not forget to follow the weak links in order to trace the whole object graph... But since you use in-image serialization and no special garbage-collector like VM trick, it must not be a problem. Unless... wait, wait. A bad case would be to serialize a bunch of objects still pointed only by the WeakCollection, but not yet garbage collected. You would just waste space and time because they will die soon after materialization. So it might be a bad idea to follow those links... A worst case would be to serialize some objets weakly shared by your graph, but strongly shared by unsaved objects. In which case the graph you would save would not be a correct delimitation... If you wanted to check that, you should garbage collect first, then assert there is no weak orphan (an object pointed only weakly in your graph...). Nicolas
On 20 September 2011 20:55, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Good question. Â I *think* the answer is to take the referenced objects across as one normally would, and then wrap them in a weak collection (if that is what was serialized). Â If one had only a weak collection holding the objects, they will be short-lived. Â However, something like a registry should be able to save/re-loaded, probably much like Dolphin's STBProxies: the effect of serializing the weak collection includes running some code to recreate conditions as needed.
Looking forward to a lively debate...
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Mariano Martinez Peck [marianopeck@gmail.com] Sent: Tuesday, September 20, 2011 1:38 PM To: Pharo Development Subject: [Pharo-project] Should a serializer do something in particular with   weak references?
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? Â what could be the impact of materializing such weak references in a new image for example?
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
On Tue, 20 Sep 2011, Mariano Martinez Peck wrote:
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? what could be the impact of materializing such weak references in a new image for example?
You should ignore (serialize nil instead of them) the elements of a weak object (#weakSubclass:...) which aren't referenced strongly from other serialized objects. Levente
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
2011/9/21 Levente Uzonyi <leves@elte.hu>:
On Tue, 20 Sep 2011, Mariano Martinez Peck wrote:
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? Â what could be the impact of materializing such weak references in a new image for example?
You should ignore (serialize nil instead of them) the elements of a weak object (#weakSubclass:...) which aren't referenced strongly from other serialized objects.
Levente
Ach, you commited shorter and quicker ;) Nicolas
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
But how does Fuel "know" that the objects are not otherwise referenced? The idea of ignoring weak collections sounds a lot like a compiler optimization that sounds efficient and harmless - sometimes they cause huge problems. Think of it this way, if the objects are not otherwise referenced, sure they will be gc's soon - why are are they still around??? The comments re globally registered objects are well taken too. Dolphin's proxies are a great way to handle such things: some objects delegate their serialization to a proxy that contains logic to do global registration, etc. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice@gmail.com] Sent: Tuesday, September 20, 2011 6:08 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Should a serializer do something in particular with weak references? 2011/9/21 Levente Uzonyi <leves@elte.hu>:
On Tue, 20 Sep 2011, Mariano Martinez Peck wrote:
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? what could be the impact of materializing such weak references in a new image for example?
You should ignore (serialize nil instead of them) the elements of a weak object (#weakSubclass:...) which aren't referenced strongly from other serialized objects.
Levente
Ach, you commited shorter and quicker ;) Nicolas
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
On 21 September 2011 03:15, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
But how does Fuel "know" that the objects are not otherwise referenced? Â The idea of ignoring weak collections sounds a lot like a compiler optimization that sounds efficient and harmless - sometimes they cause huge problems. Â Think of it this way, if the objects are not otherwise referenced, sure they will be gc's soon - why are are they still around???
the main problem i think is to make fuel to know which objects are belong to your model, and which is not. because in Levente's suggestion, this makes a difference between either serializing a pointer or just replacing it with nil. But then, once you introduce "this object belongs to my model" logic, a new question immediately arises: what if some object strongly referencing an object outside your model? So, the problem is more general than just do it selectively only for weak containers. Either you doing it uniformly (providing facilities to deal with everything which outside your domain/model somehow), or you just walk the graph and storing everything (treating all object found as ones , who belong to your model). And there is nothing special to do with weak containers.
The comments re globally registered objects are well taken too. Â Dolphin's proxies are a great way to handle such things: some objects delegate their serialization to a proxy that contains logic to do global registration, etc.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice@gmail.com] Sent: Tuesday, September 20, 2011 6:08 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Should a serializer do something in particular with weak references?
2011/9/21 Levente Uzonyi <leves@elte.hu>:
On Tue, 20 Sep 2011, Mariano Martinez Peck wrote:
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? Â what could be the impact of materializing such weak references in a new image for example?
You should ignore (serialize nil instead of them) the elements of a weak object (#weakSubclass:...) which aren't referenced strongly from other serialized objects.
Levente
Ach, you commited shorter and quicker ;)
Nicolas
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
Hi
if you look at the problem in wider perspective, then right question to ask, is how to deal with objects which are register themselves in global facilities, like weak finalization, startup/shutdown lists etc.
Fuel can handle globals... but I don't understand how can weak objects be accessed as global. Could you give a snippet exemplifying, please? Martin On Tue, Sep 20, 2011 at 11:03 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 21 September 2011 03:15, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
But how does Fuel "know" that the objects are not otherwise referenced? The idea of ignoring weak collections sounds a lot like a compiler optimization that sounds efficient and harmless - sometimes they cause huge problems. Think of it this way, if the objects are not otherwise referenced, sure they will be gc's soon - why are are they still around???
the main problem i think is to make fuel to know which objects are belong to your model, and which is not. because in Levente's suggestion, this makes a difference between either serializing a pointer or just replacing it with nil.
But then, once you introduce "this object belongs to my model" logic, a new question immediately arises: what if some object strongly referencing an object outside your model? So, the problem is more general than just do it selectively only for weak containers. Either you doing it uniformly (providing facilities to deal with everything which outside your domain/model somehow), or you just walk the graph and storing everything (treating all object found as ones , who belong to your model). And there is nothing special to do with weak containers.
The comments re globally registered objects are well taken too. Dolphin's proxies are a great way to handle such things: some objects delegate their serialization to a proxy that contains logic to do global registration, etc.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [ pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice@gmail.com] Sent: Tuesday, September 20, 2011 6:08 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Should a serializer do something in particular with weak references?
2011/9/21 Levente Uzonyi <leves@elte.hu>:
On Tue, 20 Sep 2011, Mariano Martinez Peck wrote:
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? what could be the impact of materializing such weak references in a new image for example?
You should ignore (serialize nil instead of them) the elements of a weak object (#weakSubclass:...) which aren't referenced strongly from other serialized objects.
Levente
Ach, you commited shorter and quicker ;)
Nicolas
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
On 21 September 2011 07:39, Martin Dias <tinchodias@gmail.com> wrote:
Hi
if you look at the problem in wider perspective, then right question to ask, is how to deal with objects which are register themselves in global facilities, like weak finalization, startup/shutdown lists etc. Fuel can handle globals... but I don't understand how can weak objects be accessed as global. Could you give a snippet exemplifying, please?
it depends who will use fuel and for what. for that reason, i think it is up to user(s) of fuel to determine what they want to serialize by having the logic which separating an objects which they want to store, and rest of the world. While in fuel itself, i don't see a possibility to make it so, that it will work for every possible use case, this is simply unrealistic.
Martin On Tue, Sep 20, 2011 at 11:03 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 21 September 2011 03:15, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
But how does Fuel "know" that the objects are not otherwise referenced? Â The idea of ignoring weak collections sounds a lot like a compiler optimization that sounds efficient and harmless - sometimes they cause huge problems. Â Think of it this way, if the objects are not otherwise referenced, sure they will be gc's soon - why are are they still around???
the main problem i think is to make fuel to know which objects are belong to your model, and which is not. because in Levente's suggestion, this makes a difference between either serializing a pointer or just replacing it with nil.
But then, once you introduce "this object belongs to my model" logic, a new question immediately arises: what if some object strongly referencing an object outside your model? So, the problem is more general than just do it selectively only for weak containers. Either you doing it uniformly (providing facilities to deal with everything which outside your domain/model somehow), or you just walk the graph and storing everything (treating all object found as ones , who belong to your model). And there is nothing special to do with weak containers.
The comments re globally registered objects are well taken too. Â Dolphin's proxies are a great way to handle such things: some objects delegate their serialization to a proxy that contains logic to do global registration, etc.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Nicolas Cellier [nicolas.cellier.aka.nice@gmail.com] Sent: Tuesday, September 20, 2011 6:08 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Should a serializer do something in particular with weak references?
2011/9/21 Levente Uzonyi <leves@elte.hu>:
On Tue, 20 Sep 2011, Mariano Martinez Peck wrote:
Hi guys. Martin and I were discussing whether we should do something in particular in Fuel with Weak references, such as WeakArray. Should we serialize it the same way we serialize all objects, or should we do something special? Â what could be the impact of materializing such weak references in a new image for example?
You should ignore (serialize nil instead of them) the elements of a weak object (#weakSubclass:...) which aren't referenced strongly from other serialized objects.
Levente
Ach, you commited shorter and quicker ;)
Nicolas
Thanks your sharing your thoughts.
-- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
participants (6)
-
Igor Stasenko -
Levente Uzonyi -
Mariano Martinez Peck -
Martin Dias -
Nicolas Cellier -
Schwab,Wilhelm K