Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? -- thanks, Robert
Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com> wrote:
Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
Those are the most beautiful words I've heard today. thanks to you, brother, Robert On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com> wrote:
Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. What is the format; is it language neutral? thanks, Robert On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com> wrote:
Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com> wrote:
Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers.
What is the format; is it language neutral?
For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. Documentation: http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com> wrote:
Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
Yes, I have to do object substitutions. Thanks for the link! thanks, Robert On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com> wrote:
Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers.
What is the format; is it language neutral?
For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation: http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com> wrote:
Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
Hi Robert, Also checkout the chapter on Fuel in Pharo Enterprise book: https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la... Bye Torsten
Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr Von: "Robert Withers" <robert.w.withers@gmail.com> An: pharo-dev@lists.pharo.org Betreff: Re: [Pharo-dev] binary serialization
Yes, I have to do object substitutions. Thanks for the link!
thanks, Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com> wrote:
Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers.
What is the format; is it language neutral?
For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation: http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com> wrote:
Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
Hi Robert, As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf Let us know how it goes, On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr Von: "Robert Withers" <robert.w.withers@gmail.com> An: pharo-dev@lists.pharo.org Betreff: Re: [Pharo-dev] binary serialization
Yes, I have to do object substitutions. Thanks for the link!
thanks, Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com>
wrote:
Sven and Torsten, that's a binary serialization library! It will take
time to learn it and how to use mappers.
What is the format; is it language neutral?
For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation: http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
Yes, it is called FUEL and it is a standard part of the image. See
FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <
robert.w.withers@gmail.com> wrote:
Does Pharo have stream classes to binary de/serialize an object,
such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
-- Mariano http://marianopeck.wordpress.com
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible? thanks, Robert On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
Hi Max, Yes, I was referring to this page: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG..., though I had it backwards (and thought I might have) and got confused as the example is about a stream. I meant that this page demonstrates a way to register a substitution rule with the analyzer on serialization, rather than implement #fuelAccepts: with a mapper. I do not see a use with the analyzer on materialization. Is there such? thanks, Robert On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
Yes, I meant dynamic substitution on materialization, to use the correct terminology. thanks, Robert On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything. Cheers, On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com>
wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
Hi Mariano, This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks. #fuelAccept: is a serialization side method. If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization. I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder. No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder. Does this approach seem sound to you, from a you know architecture and design approach? thanks so much ^^ Robert On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com
wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks? We also support class renames. This is here: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s... Which kind of migration example you have in mind that would not be supported? An example would help. #fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote:
http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks? We also support class renames. This is here: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s...
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side: Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef) A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side? thanks so much ^^ Robert
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
On 14 Oct 2015, at 04:39, Robert Withers <robert.w.withers@gmail.com> wrote:
On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks?
Sounds to me like this (see FuelOutStackDebuAction): serializeTestFailureContext: aContext toFileNamed: aFilename | serializer | serializer := FLSerializer newDefault. self encodeDebugInformationOn: serializer. serializer addPostMaterializationAction: [ :materialization | Smalltalk tools debugger openOn: Processor activeProcess context: materialization root label: 'External stack' contents: nil fullView: false ]. serializer " use the sender context, generally the current context is not interesting" serialize: aContext toFileNamed: aFilename This stores a block in the serialization which is evaluated after materialization. The only requirement is that itâs a clean block (no closure!).
We also support class renames. This is here: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s...
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?
thanks so much ^^ Robert
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr > Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> > An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>> > Betreff: Re: [Pharo-dev] binary serialization > > Yes, I have to do object substitutions. Thanks for the link! > > thanks, > Robert > > On 10/13/2015 03:43 AM, Max Leske wrote: > > > >> On 13 Oct 2015, at 09:40, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote: > >> > >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers. > >> > >> What is the format; is it language neutral? > > > > For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff. > > > > Documentation: http://rmod.inria.fr/web/software/Fuel. > > > > > >> > >> thanks, > >> Robert > >> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote: > >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer. > >>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote: > >>>> > >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray? > >>>> > >>>> -- > >>>> thanks, > >>>> Robert > >>>> > >>> > >>> > >> > > > > > >
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>
BTW, there is a dedicated Fuel mailing list: pharo-fuel@lists.gforge.inria.fr Max
On 14 Oct 2015, at 09:45, Max Leske <maxleske@gmail.com> wrote:
On 14 Oct 2015, at 04:39, Robert Withers <robert.w.withers@gmail.com> wrote:
On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks?
Sounds to me like this (see FuelOutStackDebuAction):
serializeTestFailureContext: aContext toFileNamed: aFilename | serializer |
serializer := FLSerializer newDefault. self encodeDebugInformationOn: serializer. serializer addPostMaterializationAction: [ :materialization | Smalltalk tools debugger openOn: Processor activeProcess context: materialization root label: 'External stack' contents: nil fullView: false ].
serializer " use the sender context, generally the current context is not interesting" serialize: aContext toFileNamed: aFilename
This stores a block in the serialization which is evaluated after materialization. The only requirement is that itâs a clean block (no closure!).
We also support class renames. This is here: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s...
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?
thanks so much ^^ Robert
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>> Betreff: Re: [Pharo-dev] binary serialization
Yes, I have to do object substitutions. Thanks for the link!
thanks, Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Sven and Torsten, that's a binary
serialization library! It will take time to learn it and how to use mappers.
What is the format; is it language
neutral?
For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation: http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van
Caekenberghe wrote:
Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>
Good morning, Max. Thank you for the example. I got a little confused, between migrations and substitutions. My issue with no-arg blocks, I believe, is the inability to access my scope object to maintain the object tables. I'm attempting to write my own Materializer, Decoder and Materialization. At the moment, I'm just going to walk the graph, testing and do #becomes:. See how well that works when I get a test. It's really good to know about that other list. thanks so much ^^ Robert On 10/14/2015 04:15 AM, Max Leske wrote:
BTW, there is a dedicated Fuel mailing list: pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr>
Max
On 14 Oct 2015, at 09:45, Max Leske <maxleske@gmail.com <mailto:maxleske@gmail.com>> wrote:
On 14 Oct 2015, at 04:39, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com>> wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks?
Sounds to me like this (see FuelOutStackDebuAction):
serializeTestFailureContext: aContext toFileNamed: aFilename | serializer |
serializer := FLSerializer newDefault. self encodeDebugInformationOn: serializer. serializer addPostMaterializationAction: [ :materialization | Smalltalk tools debugger openOn: Processor activeProcess context: materialization root label: 'External stack' contents: nil fullView: false ].
serializer " use the sender context, generally the current context is not interesting" serialize: aContext toFileNamed: aFilename
This stores a block in the serialization which is evaluated after materialization. The only requirement is that itâs a clean block (no closure!).
We also support class renames. This is here: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s...
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?
thanks so much ^^ Robert
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr Von: "Robert Withers" <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>> Betreff: Re: [Pharo-dev] binary serialization
Yes, I have to do object substitutions. Thanks for the link!
thanks, Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Sven and Torsten, that's a binary
serialization library! It will take time to learn it and how to use mappers.
What is the format; is it language
neutral?
For quick serialization you donât need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation: http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van
Caekenberghe wrote:
Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>
Robert, As far as I can remember, the problem of substitutions at materialziation time was because... as you may have read, Fuel first serializes "objects" and then the references. At materialization, it first creates the objects and then, on a second step, sets the references betwen objects. So the problem was WHERE to place the hook, at which point in time. If we do it just after objects were created, then the substitution code would NOT have access to any of the instVars of that object. If we do it AFTER objects creation and after objects references, then there is no easy way to "substitute" which doesn't involve #become: (because the graph is already constructed) And you know that #become: is still terrible slow because it scans full memory. That will change in Spur. As for Marea and Ghost, Ghost proxies paper: https://hal.inria.fr/hal-01081236/document Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf Current repo: http://ss3.gemstone.com/ss/Marea.html And finally, my PhD thesis: https://tel.archives-ouvertes.fr/tel-00764991/document In Marea I needed custom clusters for my proxies because the serializer itself sends messages to the objects being serialized. My proxies would bring back graphs from a secondary memory. So if I was serializing a graph that had proxies already, I didn't want that. So I hooked my custom cluster for proxies that send only a few special messages to the proxy that these understand and answer rather than intercept those messages. As for how to extend Fuel for this, I recommend to check the code of Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'. I have a Marea one click here: https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0 Cheers, On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers <robert.w.withers@gmail.com
wrote:
Good morning, Max. Thank you for the example. I got a little confused, between migrations and substitutions. My issue with no-arg blocks, I believe, is the inability to access my scope object to maintain the object tables.
I'm attempting to write my own Materializer, Decoder and Materialization. At the moment, I'm just going to walk the graph, testing and do #becomes:. See how well that works when I get a test.
It's really good to know about that other list.
thanks so much ^^ Robert
On 10/14/2015 04:15 AM, Max Leske wrote:
BTW, there is a dedicated Fuel mailing list: pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr
Max
On 14 Oct 2015, at 09:45, Max Leske <maxleske@gmail.com
<mailto:maxleske@gmail.com>> wrote:
On 14 Oct 2015, at 04:39, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com>> wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks?
Sounds to me like this (see FuelOutStackDebuAction):
serializeTestFailureContext: aContext toFileNamed: aFilename | serializer |
serializer := FLSerializer newDefault. self encodeDebugInformationOn: serializer. serializer addPostMaterializationAction: [ :materialization | Smalltalk tools debugger openOn: Processor activeProcess context: materialization root label: 'External stack' contents: nil fullView: false ].
serializer " use the sender context, generally the current context is not interesting" serialize: aContext toFileNamed: aFilename
This stores a block in the serialization which is evaluated after materialization. The only requirement is that itâs a clean block (no closure!).
We also support class renames. This is here:
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s...
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?
thanks so much ^^ Robert
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
Gesendet: Dienstag, 13. Oktober 2015 um
09:44 Uhr
Von: "Robert Withers"
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>
An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>
<mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>
Betreff: Re: [Pharo-dev] binary
serialization
Yes, I have to do object substitutions.
Thanks for the link!
thanks, Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers
<robert.w.withers@gmail.com
<mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Sven and Torsten, that's a binary
serialization library! It
will take time to learn it and how to use mappers.
What is the format; is it language
neutral?
For quick serialization you donât
need to do
anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation:
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van
Caekenberghe
wrote:
Yes, it is called FUEL and it is a
standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert
Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Does Pharo have stream classes to
binary
de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>
-- Mariano http://marianopeck.wordpress.com
On 10/14/2015 11:01 AM, Mariano Martinez Peck wrote:
Robert,
As far as I can remember, the problem of substitutions at materialziation time was because... as you may have read, Fuel first serializes "objects" and then the references. At materialization, it first creates the objects and then, on a second step, sets the references betwen objects. So the problem was WHERE to place the hook, at which point in time. If we do it just after objects were created, then the substitution code would NOT have access to any of the instVars of that object. If we do it AFTER objects creation and after objects references, then there is no easy way to "substitute" which doesn't involve #become: (because the graph is already constructed) And you know that #become: is still terrible slow because it scans full memory. That will change in Spur.
The trick I learned from Gemstone is to use forwarding proxies, looking up into the FLobjectId dictionary (decoder>>objects?) when stitching the references. When you copy the proxies on substitution, it stitches normally at reference time.
As for Marea and Ghost,
Ghost proxies paper: https://hal.inria.fr/hal-01081236/document Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost
Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf Current repo: http://ss3.gemstone.com/ss/Marea.html
And finally, my PhD thesis: https://tel.archives-ouvertes.fr/tel-00764991/document
Nicely done, sir. I'll check them out, thankyou.
In Marea I needed custom clusters for my proxies because the serializer itself sends messages to the objects being serialized. My proxies would bring back graphs from a secondary memory. So if I was serializing a graph that had proxies already, I didn't want that. So I hooked my custom cluster for proxies that send only a few special messages to the proxy that these understand and answer rather than intercept those messages.
As for how to extend Fuel for this, I recommend to check the code of Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'. I have a Marea one click here: https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0
Cheers,
thanks so much ^^ Robert
On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Good morning, Max. Thank you for the example. I got a little confused, between migrations and substitutions. My issue with no-arg blocks, I believe, is the inability to access my scope object to maintain the object tables.
I'm attempting to write my own Materializer, Decoder and Materialization. At the moment, I'm just going to walk the graph, testing and do #becomes:. See how well that works when I get a test.
It's really good to know about that other list.
thanks so much ^^ Robert
On 10/14/2015 04:15 AM, Max Leske wrote:
BTW, there is a dedicated Fuel mailing list: pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr> <mailto:pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr>>
Max
On 14 Oct 2015, at 09:45, Max Leske <maxleske@gmail.com <mailto:maxleske@gmail.com> <mailto:maxleske@gmail.com <mailto:maxleske@gmail.com>>> wrote:
On 14 Oct 2015, at 04:39, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks?
Sounds to me like this (see FuelOutStackDebuAction):
serializeTestFailureContext: aContext toFileNamed: aFilename | serializer |
serializer := FLSerializer newDefault. self encodeDebugInformationOn: serializer. serializer addPostMaterializationAction: [ :materialization | Smalltalk tools debugger openOn: Processor activeProcess context: materialization root label: 'External stack' contents: nil fullView: false ].
serializer " use the sender context, generally the current context is not interesting" serialize: aContext toFileNamed: aFilename
This stores a block in the serialization which is evaluated after materialization. The only requirement is that itâs a clean block (no closure!).
We also support class renames. This is here: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s...
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?
thanks so much ^^ Robert
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
Gesendet: Dienstag, 13. Oktober 2015 um
09:44 Uhr
Von: "Robert Withers"
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>>
An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>>
Betreff: Re: [Pharo-dev] binary
serialization
Yes, I have to do object substitutions.
Thanks for the link!
thanks, Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>> wrote:
Sven and Torsten, that's a binary
serialization library! It will take time to learn it and how to use mappers.
What is the format; is it language
neutral?
For quick serialization you donât
need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation:
http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van
Caekenberghe wrote:
Yes, it is called FUEL and it is a
standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert
Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>> wrote:
Does Pharo have stream classes to
binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>
-- Mariano http://marianopeck.wordpress.com
On Wed, Oct 14, 2015 at 12:09 PM, Robert Withers <robert.w.withers@gmail.com
wrote:
On 10/14/2015 11:01 AM, Mariano Martinez Peck wrote:
Robert,
As far as I can remember, the problem of substitutions at materialziation time was because... as you may have read, Fuel first serializes "objects" and then the references. At materialization, it first creates the objects and then, on a second step, sets the references betwen objects. So the problem was WHERE to place the hook, at which point in time. If we do it just after objects were created, then the substitution code would NOT have access to any of the instVars of that object. If we do it AFTER objects creation and after objects references, then there is no easy way to "substitute" which doesn't involve #become: (because the graph is already constructed) And you know that #become: is still terrible slow because it scans full memory. That will change in Spur.
The trick I learned from Gemstone is to use forwarding proxies,
Well, Spur will/does something similar called lazy become. Basically it lets a forwarding pointer object and then takes advantage of next GC pass or whatever in order to resolve such proxy in lazy manner.
looking up into the FLobjectId dictionary (decoder>>objects?) when stitching the references. When you copy the proxies on substitution, it stitches normally at reference time.
Yes, that could work. The problem is if you need instVars of the object you want to substitute. Imagine you have a class called Client and instVar 'age'. And you want to substitute Client with instances of OldClient if 'age' is > 10. At that point in time, the reference to the instVar 10 has not yet been filled. Yet, you need to replace the object at graph construction time.
As for Marea and Ghost,
Ghost proxies paper: https://hal.inria.fr/hal-01081236/document Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost
Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf Current repo: http://ss3.gemstone.com/ss/Marea.html
And finally, my PhD thesis: https://tel.archives-ouvertes.fr/tel-00764991/document
Nicely done, sir. I'll check them out, thankyou.
Thanks, feel free to ask questions.
In Marea I needed custom clusters for my proxies because the serializer
itself sends messages to the objects being serialized. My proxies would bring back graphs from a secondary memory. So if I was serializing a graph that had proxies already, I didn't want that. So I hooked my custom cluster for proxies that send only a few special messages to the proxy that these understand and answer rather than intercept those messages.
As for how to extend Fuel for this, I recommend to check the code of Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'. I have a Marea one click here: https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0
Cheers,
thanks so much ^^ Robert
On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Good morning, Max. Thank you for the example. I got a little confused, between migrations and substitutions. My issue with no-arg blocks, I believe, is the inability to access my scope object to maintain the object tables.
I'm attempting to write my own Materializer, Decoder and Materialization. At the moment, I'm just going to walk the graph, testing and do #becomes:. See how well that works when I get a test.
It's really good to know about that other list.
thanks so much ^^ Robert
On 10/14/2015 04:15 AM, Max Leske wrote:
BTW, there is a dedicated Fuel mailing list: pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr> <mailto:pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr>>
Max
On 14 Oct 2015, at 09:45, Max Leske <maxleske@gmail.com <mailto:maxleske@gmail.com> <mailto:maxleske@gmail.com <mailto:maxleske@gmail.com>>> wrote:
On 14 Oct 2015, at 04:39, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks?
Sounds to me like this (see FuelOutStackDebuAction):
serializeTestFailureContext: aContext toFileNamed: aFilename | serializer |
serializer := FLSerializer newDefault. self encodeDebugInformationOn: serializer. serializer addPostMaterializationAction: [ :materialization | Smalltalk tools debugger openOn: Processor activeProcess context: materialization root label: 'External stack' contents: nil fullView: false ].
serializer " use the sender context, generally the current context is not interesting" serialize: aContext toFileNamed: aFilename
This stores a block in the serialization which is evaluated after materialization. The only requirement is that itâs a clean block (no closure!).
We also support class renames. This is here:
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s...
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?
thanks so much ^^ Robert
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote:
http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
Gesendet: Dienstag, 13. Oktober 2015 um
09:44 Uhr
Von: "Robert Withers"
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>>
An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>>
Betreff: Re: [Pharo-dev] binary
serialization
Yes, I have to do object substitutions.
Thanks for the link!
thanks, Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>> wrote:
Sven and Torsten, that's a binary
serialization library! It will take time to learn it and how to use mappers.
What is the format; is it language
neutral?
For quick serialization you donât
need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation:
http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van
Caekenberghe wrote:
Yes, it is called FUEL and it is a
standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert
Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>> wrote:
Does Pharo have stream classes to
binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
On 10/14/2015 11:37 AM, Mariano Martinez Peck wrote:
On Wed, Oct 14, 2015 at 12:09 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
On 10/14/2015 11:01 AM, Mariano Martinez Peck wrote:
Robert,
As far as I can remember, the problem of substitutions at materialziation time was because... as you may have read, Fuel first serializes "objects" and then the references. At materialization, it first creates the objects and then, on a second step, sets the references betwen objects. So the problem was WHERE to place the hook, at which point in time. If we do it just after objects were created, then the substitution code would NOT have access to any of the instVars of that object. If we do it AFTER objects creation and after objects references, then there is no easy way to "substitute" which doesn't involve #become: (because the graph is already constructed) And you know that #become: is still terrible slow because it scans full memory. That will change in Spur.
The trick I learned from Gemstone is to use forwarding proxies,
Well, Spur will/does something similar called lazy become. Basically it lets a forwarding pointer object and then takes advantage of next GC pass or whatever in order to resolve such proxy in lazy manner.
looking up into the FLobjectId dictionary (decoder>>objects?) when stitching the references. When you copy the proxies on substitution, it stitches normally at reference time.
Yes, that could work. The problem is if you need instVars of the object you want to substitute. Imagine you have a class called Client and instVar 'age'. And you want to substitute Client with instances of OldClient if 'age' is > 10. At that point in time, the reference to the instVar 10 has not yet been filled. Yet, you need to replace the object at graph construction time.
Ok, so a depth-first, node post-apply would allow ivars to substitute before they are accessed when the container may substitute, lazy as you point out. thanks so much ^^ Robert
As for Marea and Ghost,
Ghost proxies paper: https://hal.inria.fr/hal-01081236/document Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost
Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf Current repo: http://ss3.gemstone.com/ss/Marea.html
And finally, my PhD thesis: https://tel.archives-ouvertes.fr/tel-00764991/document
Nicely done, sir. I'll check them out, thankyou.
Thanks, feel free to ask questions.
In Marea I needed custom clusters for my proxies because the serializer itself sends messages to the objects being serialized. My proxies would bring back graphs from a secondary memory. So if I was serializing a graph that had proxies already, I didn't want that. So I hooked my custom cluster for proxies that send only a few special messages to the proxy that these understand and answer rather than intercept those messages.
As for how to extend Fuel for this, I recommend to check the code of Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'. I have a Marea one click here: https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0
Cheers,
thanks so much ^^ Robert
On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> wrote:
Good morning, Max. Thank you for the example. I got a little confused, between migrations and substitutions. My issue with no-arg blocks, I believe, is the inability to access my scope object to maintain the object tables.
I'm attempting to write my own Materializer, Decoder and Materialization. At the moment, I'm just going to walk the graph, testing and do #becomes:. See how well that works when I get a test.
It's really good to know about that other list.
thanks so much ^^ Robert
On 10/14/2015 04:15 AM, Max Leske wrote:
BTW, there is a dedicated Fuel mailing list: pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr> <mailto:pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr>> <mailto:pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr> <mailto:pharo-fuel@lists.gforge.inria.fr <mailto:pharo-fuel@lists.gforge.inria.fr>>>
Max
On 14 Oct 2015, at 09:45, Max Leske <maxleske@gmail.com <mailto:maxleske@gmail.com> <mailto:maxleske@gmail.com <mailto:maxleske@gmail.com>> <mailto:maxleske@gmail.com <mailto:maxleske@gmail.com> <mailto:maxleske@gmail.com <mailto:maxleske@gmail.com>>>> wrote:
On 14 Oct 2015, at 04:39, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>> wrote:
Hi Mariano,
This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.
What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks?
Sounds to me like this (see FuelOutStackDebuAction):
serializeTestFailureContext: aContext toFileNamed: aFilename | serializer |
serializer := FLSerializer newDefault. self encodeDebugInformationOn: serializer. serializer addPostMaterializationAction: [ :materialization | Smalltalk tools debugger openOn: Processor activeProcess context: materialization root label: 'External stack' contents: nil fullView: false ].
serializer " use the sender context, generally the current context is not interesting" serialize: aContext toFileNamed: aFilename
This stores a block in the serialization which is evaluated after materialization. The only requirement is that itâs a clean block (no closure!).
We also support class renames. This is here: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s...
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.
I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc
Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?
thanks so much ^^ Robert
No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.
Does this approach seem sound to you, from a you know architecture and design approach?
There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.
thanks so much ^^ Robert
On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingG... so if you didn't find anything of help in there there are chances there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>> wrote:
Yes, I meant dynamic substitution on materialization, to use the correct terminology.
thanks, Robert
On 10/13/2015 11:40 AM, Max Leske wrote:
On 13 Oct 2015, at 17:16, Robert Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>> wrote:
Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
I donât understand what you mean by ânon-streamâ. Could you give an example?
thanks, Robert
On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
Hi Robert,
As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel
But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it goes,
On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>>
<mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>> <mailto:astares@gmx.de <mailto:astares@gmx.de> <mailto:astares@gmx.de <mailto:astares@gmx.de>>>>>> wrote:
Hi Robert,
Also checkout the chapter on Fuel in Pharo Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la...
Bye Torsten
Gesendet: Dienstag, 13. Oktober 2015 um
09:44 Uhr
Von: "Robert Withers"
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>>>
An: pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>
<mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org> <mailto:pharo-dev@lists.pharo.org <mailto:pharo-dev@lists.pharo.org>>>>>
Betreff: Re: [Pharo-dev] binary
serialization
Yes, I have to do object substitutions.
Thanks for the link!
thanks, Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40, Robert Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>>> wrote:
Sven and Torsten, that's a binary
serialization library! It will take time to learn it and how to use mappers.
What is the format; is it language
neutral?
For quick serialization you donât
need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
Documentation:
http://rmod.inria.fr/web/software/Fuel.
thanks, Robert
On 10/13/2015 01:21 AM, Sven Van
Caekenberghe wrote:
Yes, it is called FUEL and it is a
standard part of the image. See FLSerializer and FLMaterializer.
On 13 Oct 2015, at 06:59, Robert
Withers
<robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>
<mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com> <mailto:robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>>>>>> wrote:
Does Pharo have stream classes to
binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
-- thanks, Robert
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I'm interesting in knowing more :) Especially Vat and others. Stef
On 10/22/2015 06:19 AM, stepharo wrote:
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I'm interesting in knowing more :) Especially Vat and others.
Firstly, I should credit these ideas: I leaned of Elib 9 years ago, from http://erights.org. A vat is an event-loop. This ensures single-threading with only one thread executing a stack as it schedules to activity. A vat is conceptually a stack and a queue. This is an executing stack of immediate calls and a queue of eventual sends that will be scheduled into the stack as evaluates. When you have a farReference to an object in a different vat, then you need to serialize the graph of arguments sent to the remote object with an eventual send: to queue the send in the remote vat, pending evaluation as a stack. When a graph is serialized, any PassByProxy objects sent as an argument to a remote method evaluation must be substituted on serialization with a descriptor describing the export of that far reference. This descriptor substitution is not a Fuel migration between versions of one object class nor is it a Fuel substitution, which replaces a subset of ivars, it is a true substitution, a descriptor of class FarDescriptor replaced for an arbitrary PasssByProxy object...let's say Customer <name, age, lastFourSSN>. So aCustomer <name: "Robert", age: 45, lastFourSSN: 9876> will get substituted with aFarReferenceDesc <wireId: 1, wireCount: 1>. On the receiving side, who sees this FerReference, if gets substituted again for a FarRef. aCustomer ---> farDesc ---> farRef Does this help to answer your questions? Fire away with more questions and I will try to answer. Reading about Elib on the erights site is another great source of information, sicne these guys came up with it. My thoughts in the VM are to add a pool to the stack and the queue of an event-loop. For processes/continuations what are waiting for a signal, they are not scheduled into the queue. I believe that this model is a good one for a single-image-threaded execution machinery: a scheduling event-loop. This is pure intuition. Best, Robert
Stef
Hi robert thanks for refreshing me (I read E some year ago) and I read the PhD of tom van cutsem. Now what are you doing? What is your goal? Stef Le 22/10/15 15:11, Robert Withers a écrit :
On 10/22/2015 06:19 AM, stepharo wrote:
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I'm interesting in knowing more :) Especially Vat and others.
Firstly, I should credit these ideas: I leaned of Elib 9 years ago, from http://erights.org.
A vat is an event-loop. This ensures single-threading with only one thread executing a stack as it schedules to activity. A vat is conceptually a stack and a queue. This is an executing stack of immediate calls and a queue of eventual sends that will be scheduled into the stack as evaluates. When you have a farReference to an object in a different vat, then you need to serialize the graph of arguments sent to the remote object with an eventual send: to queue the send in the remote vat, pending evaluation as a stack.
When a graph is serialized, any PassByProxy objects sent as an argument to a remote method evaluation must be substituted on serialization with a descriptor describing the export of that far reference. This descriptor substitution is not a Fuel migration between versions of one object class nor is it a Fuel substitution, which replaces a subset of ivars, it is a true substitution, a descriptor of class FarDescriptor replaced for an arbitrary PasssByProxy object...let's say Customer <name, age, lastFourSSN>. So aCustomer <name: "Robert", age: 45, lastFourSSN: 9876> will get substituted with aFarReferenceDesc <wireId: 1, wireCount: 1>. On the receiving side, who sees this FerReference, if gets substituted again for a FarRef.
aCustomer ---> farDesc ---> farRef
Does this help to answer your questions? Fire away with more questions and I will try to answer. Reading about Elib on the erights site is another great source of information, sicne these guys came up with it.
My thoughts in the VM are to add a pool to the stack and the queue of an event-loop. For processes/continuations what are waiting for a signal, they are not scheduled into the queue. I believe that this model is a good one for a single-image-threaded execution machinery: a scheduling event-loop. This is pure intuition.
Best, Robert
Stef
Hi stepharo, I was looking for tom van cutsem's PhD, since you mentioned it, but did not find it. Do you have a link? I am attaching my business plan to you, to answer your question asking what is my goal. That's high-level. Low-level, what I am doing: I require an object-capability system, so I am reconstituting SqueakElib and will be looking at VM and networking areas to fully support capabilities and piconet formation. Robert On 10/24/2015 03:43 AM, stepharo wrote:
Hi robert
thanks for refreshing me (I read E some year ago) and I read the PhD of tom van cutsem. Now what are you doing? What is your goal?
Stef
Le 22/10/15 15:11, Robert Withers a écrit :
On 10/22/2015 06:19 AM, stepharo wrote:
Which kind of migration example you have in mind that would not be supported? An example would help.
Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.
There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.
It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
I'm interesting in knowing more :) Especially Vat and others.
Firstly, I should credit these ideas: I leaned of Elib 9 years ago, from http://erights.org.
A vat is an event-loop. This ensures single-threading with only one thread executing a stack as it schedules to activity. A vat is conceptually a stack and a queue. This is an executing stack of immediate calls and a queue of eventual sends that will be scheduled into the stack as evaluates. When you have a farReference to an object in a different vat, then you need to serialize the graph of arguments sent to the remote object with an eventual send: to queue the send in the remote vat, pending evaluation as a stack.
When a graph is serialized, any PassByProxy objects sent as an argument to a remote method evaluation must be substituted on serialization with a descriptor describing the export of that far reference. This descriptor substitution is not a Fuel migration between versions of one object class nor is it a Fuel substitution, which replaces a subset of ivars, it is a true substitution, a descriptor of class FarDescriptor replaced for an arbitrary PasssByProxy object...let's say Customer <name, age, lastFourSSN>. So aCustomer <name: "Robert", age: 45, lastFourSSN: 9876> will get substituted with aFarReferenceDesc <wireId: 1, wireCount: 1>. On the receiving side, who sees this FerReference, if gets substituted again for a FarRef.
aCustomer ---> farDesc ---> farRef
Does this help to answer your questions? Fire away with more questions and I will try to answer. Reading about Elib on the erights site is another great source of information, sicne these guys came up with it.
My thoughts in the VM are to add a pool to the stack and the queue of an event-loop. For processes/continuations what are waiting for a signal, they are not scheduled into the queue. I believe that this model is a good one for a single-image-threaded execution machinery: a scheduling event-loop. This is pure intuition.
Best, Robert
Stef
Hi Robert:
On 22 Oct 2015, at 15:11, Robert Withers <robert.w.withers@gmail.com> wrote:
My thoughts in the VM are to add a pool to the stack and the queue of an event-loop. For processes/continuations what are waiting for a signal, they are not scheduled into the queue. I believe that this model is a good one for a single-image-threaded execution machinery: a scheduling event-loop. This is pure intuition.
There is no need for changing the VM, it is already doing that. When a process is blocked on a semaphore, it is not on the scheduler queue, but on the semaphore, and will only be scheduled once the semaphore has been signaled. Pharo/Smalltalk brings all the basics for implementing the model, except any form of strong guarantee for isolation between vats. But thatâs the usual restriction with libraries, so, shouldnât come as a surprise. (Also, Smalltalkâs classes really do not play well with vats. What about their state for instance?) Best regards Stefan
Best, Robert
Stef
-- Stefan Marr Johannes Kepler Universität Linz http://stefan-marr.de/research/
Hi Stefan, On 10/24/2015 04:16 AM, Stefan Marr wrote:
Hi Robert:
On 22 Oct 2015, at 15:11, Robert Withers <robert.w.withers@gmail.com> wrote:
My thoughts in the VM are to add a pool to the stack and the queue of an event-loop. For processes/continuations what are waiting for a signal, they are not scheduled into the queue. I believe that this model is a good one for a single-image-threaded execution machinery: a scheduling event-loop. This is pure intuition. There is no need for changing the VM, it is already doing that. When a process is blocked on a semaphore, it is not on the scheduler queue, but on the semaphore, and will only be scheduled once the semaphore has been signaled. With an event-loop as the core construct of the VM, hypothetically speaking from my intution, the async FFI and threeaded VM would have a structural solution. This context changes the mechanics of semaphores, I think. As well semaphores are how the VM signals into Pharo/Squeak so they would definitely change.
Pharo/Smalltalk brings all the basics for implementing the model, except any form of strong guarantee for isolation between vats. But thatâs the usual restriction with libraries, so, shouldnât come as a surprise.
(Also, Smalltalkâs classes really do not play well with vats. What about their state for instance?) Yes, indeed. Globals are a security evil. Newspeak gets this right. If Squeak went to hierarchical namespaces, changed literal lookup and eliminated Globals, it would be much more secure I think.
Robert
Best regards Stefan
Best, Robert
Stef
participants (7)
-
Mariano Martinez Peck -
Max Leske -
Robert Withers -
Stefan Marr -
stepharo -
Sven Van Caekenberghe -
Torsten Bergmann