[Pharo-project] how should Pier file structures be serialized with Fuel?
An earlier version of Fuel did not having the substitution feature. So, I had custom mapping code to pre-process the structures before passing to Fuel for serialization, and to post-process the deserialized structures. The main purpose of the code was to nil out the PUSecurity objects, because they were linked to too many other Pier objects. I'm now eliminating the custom mapping code by using Fuel substitution. The PRFile object was handled by saving the file contents in an instance variable of the mapping object. I've eliminated the custom mapping object. Now, how should the file contents be saved/restored, using the substitution feature? I'm thinking of substituting a proxy object, which somehow "becomes" a PRFile after deserialization. Any other suggestions?
Hello! On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu <yanni@rogers.com> wrote:
An earlier version of Fuel did not having the substitution feature. So, I had custom mapping code to pre-process the structures before passing to Fuel for serialization, and to post-process the deserialized structures. The main purpose of the code was to nil out the PUSecurity objects, because they were linked to too many other Pier objects.
I'm now eliminating the custom mapping code by using Fuel substitution. The PRFile object was handled by saving the file contents in an instance variable of the mapping object. I've eliminated the custom mapping object. Now, how should the file contents be saved/restored, using the substitution feature?
I'm thinking of substituting a proxy object, which somehow "becomes" a PRFile after deserialization.
you can implement : YourPRFileProxy >> fuelAfterMaterialization self become: self proxiedPRFile Anyway, in next version would be nice to provide a faster way of doing this, without become.
Any other suggestions?
Just out of curiosity: Why are PUSecurity linked to too many other objects? Is there a bug leaking memory? Lukas On Thursday, 17 November 2011, Martin Dias <tinchodias@gmail.com> wrote:
Hello! On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu <yanni@rogers.com> wrote:
An earlier version of Fuel did not having the substitution feature. So,
I had custom mapping code to pre-process the structures before passing to Fuel for serialization, and to post-process the deserialized structures. The main purpose of the code was to nil out the PUSecurity objects, because they were linked to too many other Pier objects.
I'm now eliminating the custom mapping code by using Fuel substitution.
The PRFile object was handled by saving the file contents in an instance variable of the mapping object. I've eliminated the custom mapping object. Now, how should the file contents be saved/restored, using the substitution feature?
I'm thinking of substituting a proxy object, which somehow "becomes" a
PRFile after deserialization.
you can implement : YourPRFileProxy >> fuelAfterMaterialization self become: self proxiedPRFile
Anyway, in next version would be nice to provide a faster way of doing this, without become.
Any other suggestions?
-- Lukas Renggli www.lukas-renggli.ch
On 17/11/11 1:44 AM, Lukas Renggli wrote:
Just out of curiosity: Why are PUSecurity linked to too many other objects? Is there a bug leaking memory?
I don't think there's a memory leak. My use case here is to export a subtree of a Pier kernel, for import into another Pier kernel. [Aside - not all subtrees can be meaningfully exported, but I'm still discovering the conditions.] In the course of using Fuel (the very first version released), I found that the resulting file was big, and always the same size, no matter what structure was being exported. Looking at the de-serialized structure, I noticed the PUSecurity decoration had pulled in all the users and groups - something I did not want to be imported into the new kernel. This led me to writing a mapping object that contained only the values to be deserialized and used to reconstruct a PRStructure. Thus, the PUSecurity objects were effectively nil'ed out. Now, after upgrading to the latest Fuel, I'm trying to use the substitution feature to accomplish the same. I ended up having to use: serializer analyzer when: [:x | Array = x class and: [x anySatisfy: [:each | PUSecurity = each class ]]] substituteBy: [:x | x reject: [:each | PUSecurity = each class ]] The obvious substitution of nil for the PUSecurity object ends up putting an unexpected nil object in the children Array. I've just noticed that even with the PUSecurity object being striped before serialization, I'm still seeing that the files are big and the same size. It seems that my mapping strategy was leaving off a lot more than just the PUSecurity object.
I've just noticed that even with the PUSecurity object being striped before serialization, I'm still seeing that the files are big and the same size. It seems that my mapping strategy was leaving off a lot more than just the PUSecurity object.
You debug you can do is to put a halt in FLSerialization>>run just after #analysisStep is done. Once there you can inspect the instVar 'clusters' and see how many clusters you have and how many objects each cluster has. Cheers -- Mariano http://marianopeck.wordpress.com
On 18/11/11 5:58 PM, Mariano Martinez Peck wrote:
I've just noticed that even with the PUSecurity object being striped before serialization, I'm still seeing that the files are big and the same size. It seems that my mapping strategy was leaving off a lot more than just the PUSecurity object.
You debug you can do is to put a halt in FLSerialization>>run just after #analysisStep is done. Once there you can inspect the instVar 'clusters' and see how many clusters you have and how many objects each cluster has.
Thanks for the hint. I've not had to dive into Fuel's internals, so far. Anyhow, I've finally got the problems cleaned up. Mainly, I had to make sure that all references to the original root of the subtree, are replaced by a reference to a copy of the root. It needs to be a copy, because its parent must be set to nil, but that cannot be done to the original object. Other problems flowed from having to make a copy of the root. If you miss one reference, then that one reference will pull the entire kernel into the serialization. With the new code, I should be able to create new PRStructure or PRComponent, without having to add any custom visitor code (which I had to do with my old mapping strategy).
On Thu, Nov 17, 2011 at 3:27 AM, Martin Dias <tinchodias@gmail.com> wrote:
Hello!
On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu <yanni@rogers.com> wrote:
An earlier version of Fuel did not having the substitution feature. So, I had custom mapping code to pre-process the structures before passing to Fuel for serialization, and to post-process the deserialized structures. The main purpose of the code was to nil out the PUSecurity objects, because they were linked to too many other Pier objects.
I'm now eliminating the custom mapping code by using Fuel substitution. The PRFile object was handled by saving the file contents in an instance variable of the mapping object. I've eliminated the custom mapping object. Now, how should the file contents be saved/restored, using the substitution feature?
I'm thinking of substituting a proxy object, which somehow "becomes" a PRFile after deserialization.
you can implement :
YourPRFileProxy >> fuelAfterMaterialization self become: self proxiedPRFile
But how he replace its original PRFile of the input graph with a YourPRFileProxy instance? Wouldn't something like this make sense: PRFile >> fuelAccept: aVisitor ^ aVisitor visitSubstitution: self PRFile >> fuelSubstitution ^ YourPRFIleProxy for: self ? The problem there is that ALL PRFile will be replaced by proxies and maybe that's not what you want.
Anyway, in next version would be nice to provide a faster way of doing this, without become.
Any other suggestions?
-- Mariano http://marianopeck.wordpress.com
On 17/11/11 1:27 AM, Martin Dias wrote:
you can implement :
YourPRFileProxy >> fuelAfterMaterialization self become: self proxiedPRFile Anyway, in next version would be nice to provide a faster way of doing this, without become.
Okay #fuelAfterMaterialization - that's something I didn't know about. I think I can use that to re-arrange the content, without using #become.
On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu <yanni@rogers.com> wrote:
An earlier version of Fuel did not having the substitution feature. So, I had custom mapping code to pre-process the structures before passing to Fuel for serialization, and to post-process the deserialized structures. The main purpose of the code was to nil out the PUSecurity objects, because they were linked to too many other Pier objects.
So you have already solved this one using the substitution hook explained here: http://rmod.lille.inria.fr/web/pier/software/Fuel/Version1.7/Hooks ?
I'm now eliminating the custom mapping code by using Fuel substitution. The PRFile object was handled by saving the file contents in an instance variable of the mapping object. I've eliminated the custom mapping object. Now, how should the file contents be saved/restored, using the substitution feature?
Let me see if I understand....PRFile has a reference to a file (on disk). So what you want is to serialize not only the PRFile but also the contents of the file?
I'm thinking of substituting a proxy object, which somehow "becomes" a PRFile after deserialization.
Any other suggestions?
-- Mariano http://marianopeck.wordpress.com
On 17/11/11 11:24 AM, Mariano Martinez Peck wrote:
So you have already solved this one using the substitution hook explained here: http://rmod.lille.inria.fr/web/pier/software/Fuel/Version1.7/Hooks ?
No, the early Fuel version did not have substitution hooks. I wrote custom visitor code to do substitution during export/import. I want to now eliminate this custom code.
Let me see if I understand....PRFile has a reference to a file (on disk). So what you want is to serialize not only the PRFile but also the contents of the file?
What I want is an equivalent copy a subtree of a Pier kernel, created in another Pier kernel, even if that subtree contains a PRFile. Having the serialized subtree fit in a single file would be easiest to manipulate.
participants (4)
-
Lukas Renggli -
Mariano Martinez Peck -
Martin Dias -
Yanni Chiu