[Pharo-project] Object serialization in Pharo
Hi folks I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes). Any sample? -- Simon
"Ma object serialization" was designed for this purpose. You simply override #maTransientVariables to answer an array of the variable names that are transient. That part of the graph will not be traversed during serialization and nil will be stored in its place. On Tue, Sep 15, 2009 at 8:31 AM, Simon Denier <Simon.Denier@inria.fr> wrote:
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
-- Â Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On 15 sept. 09, at 16:08, Chris Muller wrote:
"Ma object serialization" was designed for this purpose. You simply override #maTransientVariables to answer an array of the variable names that are transient. That part of the graph will not be traversed during serialization and nil will be stored in its place.
OK thanks, the wiki offering the doc is down for the moment, but the seaside book offers a few hints about the subject.
On Tue, Sep 15, 2009 at 8:31 AM, Simon Denier <Simon.Denier@inria.fr> wrote:
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
-- Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Simon
Simon, I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes. Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything). Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used. I have used serializers to good effect and would like to continue doing so. At the moment, I would look again, but would lean away from ReferenceStream. SIXX (an XML serializer) is a possibility. As I recall, its output was not terse (but might compress nicely??). There originally was minimal to no support for shape changes, but that has been addressed; I cannot speak for how well it has been done because I have not had time to look. It is on my todo list and might float to the surface soon, though initially I would not need to care about shape changes. HTH Bill [*] I think we recently discussed a Squeak fork that would not allow touching base classes due to packaging limitations. Not good. I am not defending the ReferenceStream example, but it would be a big problem on that system, if I understood that thread. -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Simon Denier Sent: Tuesday, September 15, 2009 8:32 AM To: Pharo Development Subject: [Pharo-project] Object serialization in Pharo Hi folks I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes). Any sample? -- Simon _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Sep 15, 2009, at 4:16 46PM, Schwab,Wilhelm K wrote:
Simon,
I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes.
Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything).
Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). Sounds alot like BOSS in VisualWorks :) The only annoying thing I've found using that, is the fact you never get prompted to update the conversion method when you modify a class which already contains a binaryReadBlockFrom: method. (Kind of hard to know if you store objects of the kind to disk if it doesn't have one already) Believe it or not, but it's quite easy to forget when doing some change, and you have no tests importing old variations of all the objects you file to disk in your project.
In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used. IIRC from looking briefly at it when reviewing some removal patch which touched it, it checks first if class has conversion method, THEN it tries to autoconvert (silently if it considers itself "successful", yay). Also, if I read the code correctly, it only checks for class structure differences, so if you've, say, switched the order of two instvars since you stored an old object to disk, it'll file in just fine, but with the data in incorrect places... In short, I imagine using ReferenceStream could lead to some _very_ frustrating debugging sessions later on in the project, consider yourself warned :)
Cheers, Henry
Ok good to know Stef On Sep 15, 2009, at 4:48 PM, Henrik Johansen wrote:
On Sep 15, 2009, at 4:16 46PM, Schwab,Wilhelm K wrote:
Simon,
I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes.
Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything).
Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). Sounds alot like BOSS in VisualWorks :) The only annoying thing I've found using that, is the fact you never get prompted to update the conversion method when you modify a class which already contains a binaryReadBlockFrom: method. (Kind of hard to know if you store objects of the kind to disk if it doesn't have one already) Believe it or not, but it's quite easy to forget when doing some change, and you have no tests importing old variations of all the objects you file to disk in your project.
In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used. IIRC from looking briefly at it when reviewing some removal patch which touched it, it checks first if class has conversion method, THEN it tries to autoconvert (silently if it considers itself "successful", yay). Also, if I read the code correctly, it only checks for class structure differences, so if you've, say, switched the order of two instvars since you stored an old object to disk, it'll file in just fine, but with the data in incorrect places... In short, I imagine using ReferenceStream could lead to some _very_ frustrating debugging sessions later on in the project, consider yourself warned :)
Cheers, Henry
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Sep 15, 2009, at 4:16 PM, Schwab,Wilhelm K wrote:
Simon,
I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes.
Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything).
Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used.
I have used serializers to good effect and would like to continue doing so. At the moment, I would look again, but would lean away from ReferenceStream.
I started to write some tests in the PharoTaskForces. If people want to join effort because before changing I would like to know what is there.
SIXX (an XML serializer) is a possibility. As I recall, its output was not terse (but might compress nicely??). There originally was minimal to no support for shape changes, but that has been addressed; I cannot speak for how well it has been done because I have not had time to look. It is on my todo list and might float to the surface soon, though initially I would not need to care about shape changes.
HTH
Bill
[*] I think we recently discussed a Squeak fork that would not allow touching base classes due to packaging limitations.
really?
Not good. I am not defending the ReferenceStream example, but it would be a big problem on that system, if I understood that thread.
I would like to fix that We need a good object serializer.
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Simon Denier Sent: Tuesday, September 15, 2009 8:32 AM To: Pharo Development Subject: [Pharo-project] Object serialization in Pharo
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
-- Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Stef, The packaging constraint had nothing to do with Pharo. See the [ANN] thread on #readStream replacing ReadStream on: (around 4 Sept). SecureSqueak is the system with the apparent limitation; it really does have that constraint, it will not be of much use to me. Unit tests for a serializer would be well-taken, but I will largely have to start over. Choosing a filer seems to be the first step. Squeak's ReferenceStream did not strike me as a good option; if you wish to argue in its favor, please do so. Otherwise, I would like to see SIXX get a good evaluation, as well as any other options that should be considered. Bill -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 10:06 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo On Sep 15, 2009, at 4:16 PM, Schwab,Wilhelm K wrote:
Simon,
I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes.
Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything).
Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used.
I have used serializers to good effect and would like to continue doing so. At the moment, I would look again, but would lean away from ReferenceStream.
I started to write some tests in the PharoTaskForces. If people want to join effort because before changing I would like to know what is there.
SIXX (an XML serializer) is a possibility. As I recall, its output was not terse (but might compress nicely??). There originally was minimal to no support for shape changes, but that has been addressed; I cannot speak for how well it has been done because I have not had time to look. It is on my todo list and might float to the surface soon, though initially I would not need to care about shape changes.
HTH
Bill
[*] I think we recently discussed a Squeak fork that would not allow touching base classes due to packaging limitations.
really?
Not good. I am not defending the ReferenceStream example, but it would be a big problem on that system, if I understood that thread.
I would like to fix that We need a good object serializer.
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Simon Denier Sent: Tuesday, September 15, 2009 8:32 AM To: Pharo Development Subject: [Pharo-project] Object serialization in Pharo
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
-- Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Stef,
The packaging constraint had nothing to do with Pharo. See the [ANN] thread on #readStream replacing ReadStream on: (around 4 Sept). SecureSqueak is the system with the apparent limitation; it really does have that constraint, it will not be of much use to me.
Unit tests for a serializer would be well-taken, but I will largely have to start over. Choosing a filer seems to be the first step. Squeak's ReferenceStream did not strike me as a good option; if you wish to argue in its favor, please do so. Otherwise, I would like to see SIXX get a good evaluation, as well as any other options that should be considered.
I'm sure that I want an XML parser in the way but why not. I started to write some tests just to learn and document. I would even have no problem to throw away everything it there is a better solution. Now what tests will let enable us to change and nothing what we change. Stef
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 10:06 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo
On Sep 15, 2009, at 4:16 PM, Schwab,Wilhelm K wrote:
Simon,
I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes.
Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything).
Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used.
I have used serializers to good effect and would like to continue doing so. At the moment, I would look again, but would lean away from ReferenceStream.
I started to write some tests in the PharoTaskForces. If people want to join effort because before changing I would like to know what is there.
SIXX (an XML serializer) is a possibility. As I recall, its output was not terse (but might compress nicely??). There originally was minimal to no support for shape changes, but that has been addressed; I cannot speak for how well it has been done because I have not had time to look. It is on my todo list and might float to the surface soon, though initially I would not need to care about shape changes.
HTH
Bill
[*] I think we recently discussed a Squeak fork that would not allow touching base classes due to packaging limitations.
really?
Not good. I am not defending the ReferenceStream example, but it would be a big problem on that system, if I understood that thread.
I would like to fix that We need a good object serializer.
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Simon Denier Sent: Tuesday, September 15, 2009 8:32 AM To: Pharo Development Subject: [Pharo-project] Object serialization in Pharo
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
-- Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Stef, Henry's warning tells me I'm not alone in being afraid of Squeak's serializer. A while ago, I had settled on SIXX as the likely best option for me, but will certainly consider other options. For the Dolphin filer, my tests mostly center on loading objects with old schemas. It was fairly easy to do because I had old data in abundance, and simply loaded some of it into new images. For Pharo, we might write tests that compile a class, serialize, make some changes to the class and reload the "old" data. The details will probably depend on the serializer(s) we select. My plan has been to start out using SIXX for network transport of data between like images (where schema changes are not a concern) and then begin to look at archival uses if that goes well. Does that answer your question? Bill -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 1:19 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo
Stef,
The packaging constraint had nothing to do with Pharo. See the [ANN] thread on #readStream replacing ReadStream on: (around 4 Sept). SecureSqueak is the system with the apparent limitation; it really does have that constraint, it will not be of much use to me.
Unit tests for a serializer would be well-taken, but I will largely have to start over. Choosing a filer seems to be the first step. Squeak's ReferenceStream did not strike me as a good option; if you wish to argue in its favor, please do so. Otherwise, I would like to see SIXX get a good evaluation, as well as any other options that should be considered.
I'm sure that I want an XML parser in the way but why not. I started to write some tests just to learn and document. I would even have no problem to throw away everything it there is a better solution. Now what tests will let enable us to change and nothing what we change. Stef
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 10:06 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo
On Sep 15, 2009, at 4:16 PM, Schwab,Wilhelm K wrote:
Simon,
I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes.
Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything).
Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used.
I have used serializers to good effect and would like to continue doing so. At the moment, I would look again, but would lean away from ReferenceStream.
I started to write some tests in the PharoTaskForces. If people want to join effort because before changing I would like to know what is there.
SIXX (an XML serializer) is a possibility. As I recall, its output was not terse (but might compress nicely??). There originally was minimal to no support for shape changes, but that has been addressed; I cannot speak for how well it has been done because I have not had time to look. It is on my todo list and might float to the surface soon, though initially I would not need to care about shape changes.
HTH
Bill
[*] I think we recently discussed a Squeak fork that would not allow touching base classes due to packaging limitations.
really?
Not good. I am not defending the ReferenceStream example, but it would be a big problem on that system, if I understood that thread.
I would like to fix that We need a good object serializer.
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Simon Denier Sent: Tuesday, September 15, 2009 8:32 AM To: Pharo Development Subject: [Pharo-project] Object serialization in Pharo
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
-- Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Stef,
Henry's warning tells me I'm not alone in being afraid of Squeak's serializer. A while ago, I had settled on SIXX as the likely best option for me, but will certainly consider other options. For the Dolphin filer, my tests mostly center on loading objects with old schemas. It was fairly easy to do because I had old data in abundance, and simply loaded some of it into new images. For Pharo, we might write tests that compile a class, serialize, make some changes to the class and reload the "old" data. The details will probably depend on the serializer(s) we select.
My plan has been to start out using SIXX for network transport of data between like images (where schema changes are not a concern) and then begin to look at archival uses if that goes well.
Does that answer your question?
ok we will evaluate SIXX. Now my point is that we should have good solutions: so if referenceStream can be fix we fix it else we trash it. Half-baked solutions just hinder progress. My goal is this community is to set the cap (fun for a lighthouse) and the cap for 1.2 and other releases. I want a neat nice robust working documented system and each step going in that direction is an important one. Stef
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 1:19 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo
Stef,
The packaging constraint had nothing to do with Pharo. See the [ANN] thread on #readStream replacing ReadStream on: (around 4 Sept). SecureSqueak is the system with the apparent limitation; it really does have that constraint, it will not be of much use to me.
Unit tests for a serializer would be well-taken, but I will largely have to start over. Choosing a filer seems to be the first step. Squeak's ReferenceStream did not strike me as a good option; if you wish to argue in its favor, please do so. Otherwise, I would like to see SIXX get a good evaluation, as well as any other options that should be considered.
I'm sure that I want an XML parser in the way but why not. I started to write some tests just to learn and document. I would even have no problem to throw away everything it there is a better solution. Now what tests will let enable us to change and nothing what we change.
Stef
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 10:06 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo
On Sep 15, 2009, at 4:16 PM, Schwab,Wilhelm K wrote:
Simon,
I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes.
Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything).
Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used.
I have used serializers to good effect and would like to continue doing so. At the moment, I would look again, but would lean away from ReferenceStream.
I started to write some tests in the PharoTaskForces. If people want to join effort because before changing I would like to know what is there.
SIXX (an XML serializer) is a possibility. As I recall, its output was not terse (but might compress nicely??). There originally was minimal to no support for shape changes, but that has been addressed; I cannot speak for how well it has been done because I have not had time to look. It is on my todo list and might float to the surface soon, though initially I would not need to care about shape changes.
HTH
Bill
[*] I think we recently discussed a Squeak fork that would not allow touching base classes due to packaging limitations.
really?
Not good. I am not defending the ReferenceStream example, but it would be a big problem on that system, if I understood that thread.
I would like to fix that We need a good object serializer.
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Simon Denier Sent: Tuesday, September 15, 2009 8:32 AM To: Pharo Development Subject: [Pharo-project] Object serialization in Pharo
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
-- Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Stef, Sounds great. ReferenceStream has some real problems, and it might be very difficult to change in ways that would not orphan existing data, if in fact there is any - projects perhaps?? It might be best to adopt an existing serializer or build a clean one of our own. With some care and a few compatibility methods, we can probably write tests that can work with multiple serializers, and perhaps even profile them at the same time. Bill -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 3:18 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo
Stef,
Henry's warning tells me I'm not alone in being afraid of Squeak's serializer. A while ago, I had settled on SIXX as the likely best option for me, but will certainly consider other options. For the Dolphin filer, my tests mostly center on loading objects with old schemas. It was fairly easy to do because I had old data in abundance, and simply loaded some of it into new images. For Pharo, we might write tests that compile a class, serialize, make some changes to the class and reload the "old" data. The details will probably depend on the serializer(s) we select.
My plan has been to start out using SIXX for network transport of data between like images (where schema changes are not a concern) and then begin to look at archival uses if that goes well.
Does that answer your question?
ok we will evaluate SIXX. Now my point is that we should have good solutions: so if referenceStream can be fix we fix it else we trash it. Half-baked solutions just hinder progress. My goal is this community is to set the cap (fun for a lighthouse) and the cap for 1.2 and other releases. I want a neat nice robust working documented system and each step going in that direction is an important one. Stef
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 1:19 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo
Stef,
The packaging constraint had nothing to do with Pharo. See the [ANN] thread on #readStream replacing ReadStream on: (around 4 Sept). SecureSqueak is the system with the apparent limitation; it really does have that constraint, it will not be of much use to me.
Unit tests for a serializer would be well-taken, but I will largely have to start over. Choosing a filer seems to be the first step. Squeak's ReferenceStream did not strike me as a good option; if you wish to argue in its favor, please do so. Otherwise, I would like to see SIXX get a good evaluation, as well as any other options that should be considered.
I'm sure that I want an XML parser in the way but why not. I started to write some tests just to learn and document. I would even have no problem to throw away everything it there is a better solution. Now what tests will let enable us to change and nothing what we change.
Stef
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Stéphane Ducasse Sent: Tuesday, September 15, 2009 10:06 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Object serialization in Pharo
On Sep 15, 2009, at 4:16 PM, Schwab,Wilhelm K wrote:
Simon,
I wish you well and will be eager to hear progress reports. Long ago, I looked at ReferenceStream (or maybe SmartReferenceStream), and did not like what I found. IIRC, the output for simple examples was much larger (of course, it might not continue to scale so unfavorably) compared that that produced by Dolphin's binary filer. Another concern was schema changes.
Dolphin has an orderly conversion process that will raise an error if it is not prepared for what it finds; Squeak seemed to want to fix it itself with the aid of the user (not gonna fly in an end-user app or a server process, which is pretty much everything).
Dolphin locates conversion methods in the class of the object being converted or in the class of a proxy for it (that sounded scary to me when I read about it, but it is exactly what one would want in practice). In contrast, Squeak puts the conversion methods in ReferenceStream. Yuk!!! Sorry, don't know what just came over me :) Of course one can always modify ReferenceStream[*], but imagine the chaos if ReferenceStream were to become widely used.
I have used serializers to good effect and would like to continue doing so. At the moment, I would look again, but would lean away from ReferenceStream.
I started to write some tests in the PharoTaskForces. If people want to join effort because before changing I would like to know what is there.
SIXX (an XML serializer) is a possibility. As I recall, its output was not terse (but might compress nicely??). There originally was minimal to no support for shape changes, but that has been addressed; I cannot speak for how well it has been done because I have not had time to look. It is on my todo list and might float to the surface soon, though initially I would not need to care about shape changes.
HTH
Bill
[*] I think we recently discussed a Squeak fork that would not allow touching base classes due to packaging limitations.
really?
Not good. I am not defending the ReferenceStream example, but it would be a big problem on that system, if I understood that thread.
I would like to fix that We need a good object serializer.
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr ] On Behalf Of Simon Denier Sent: Tuesday, September 15, 2009 8:32 AM To: Pharo Development Subject: [Pharo-project] Object serialization in Pharo
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
-- Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
El mar, 15-09-2009 a las 15:31 +0200, Simon Denier escribió:
Hi folks
I'm looking at the code and comments of ReferenceStream for object serialization and I wonder what is the best strategy to serialize a partial model. I mean, I want to store all elements found while traversing the graph of the model, but I dont want to store some attributes of those elements (i.e., transient attributes).
Any sample?
SIXX or JSON are very good options (JSON doens't serialize dates so..)
-- Simon
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Miguel Cobá http://miguel.leugim.com.mx
participants (7)
-
Chris Muller -
Henrik Johansen -
Miguel Enrique Cobá Martinez -
Schwab,Wilhelm K -
Schwab,Wilhelm K
-
Simon Denier -
Stéphane Ducasse