[Pharo-project] Fuel - a fast object deployment tool

Henrik Sperre Johansen henrik.s.johansen at veloxit.no
Mon Jun 20 05:07:16 EDT 2011


On 08.12.2010 17:50, Martin Dias wrote:
> Hi all
>
> Last months I and Tristan have been working on Fuel project, an object
> binary serialization tool. The idea is that objects are much more
> times loaded than stored, therefore it is worth to spend time while
> storing in order to have faster loading and user experience. We
> present an implementation of a pickle format that is based on
> clustering similar objects.
>
> There is a summary of the project below, but more complete information
> is available here: http://rmod.lille.inria.fr/web/pier/software/Fuel
>
> The implementation still needs a lot of work to be really useful,
> optimizations should be done, but we'll be glad to get feedback of the
> community.
>
>
> = Pickle format =
>
> The pickle format and the serialization algorithm main idea, is
> explained in this slides:
>
> http://www.slideshare.net/tinchodias/fuel-serialization-in-an-example
>
>
> = Current features =
>
> - Class shape changing (when a variable has been added, or removed, or
> its index changed)
> - Serialize most of the basic objects.
> - Serialize (almost) any CompiledMethod
> - Detection of global or class variables
> - Support for cyclic object graphs
> - Tests
>
>
> = Next steps =
>
> - Improve version checking.
> - Optimize performance.
> - Serialize more kinds of objects:
> -- Class with its complete description.
> -- Method contexts
> -- Active block closures
> -- Continuation
> - Some improvements for the user:
> -- pre and post actions to be executed.
> -- easily say 'this object is singleton'.
> - Partial loading of a stored graph.
> - Fast statistics/brief info extraction of a stored graph.
> - ConfigurationOfFuel.
> - Be able to deploy materialization behavior only (independent from
> the serialization behavior)
>
>
> = Download =
>
> In a Pharo 1.1 or 1.1.1 evaluate:
>
> Gofer new
> 	squeaksource: 'Fuel';
> 	version: 'Fuel-MartinDias.74';
> 	version: 'FuelBenchmarks-MartinDias.4';
> 	load.
>
>
> = Benchmarks =
>
> You can run benchmarks executing this line (results in Transcript):
>
> FLBenchmarks newBasic run.
>
>
> Thank you!
> Martin Dias

One thing I do not see mentioned, and feel could use some attention, is 
thread safety. (aka other threads altering the graph you are serializing)
The classic answer would of course be "always run FUEL at highest 
priority", but if we ever want to move to true multi-core, that's not 
enough.

What would be neat is protecting all mutation of objects in the graph 
with a Mutex/Monitor whose critical section covers the analysis and 
serialization, i.e. blocking all other processes that wants to mutate 
objects in the graph untill serialization is complete.

Aside from the behaviour when a marked object is encountered, the 
process is the same as for immutability, as discussed here:
http://forum.world.st/immutability-and-become-Was-Re-squeak-dev-immutability-td1597511.html

You could do this image-side as part of the analyze phase, as Eliot's 
post suggests, but it's not entirely safe when:
A child -> B
B parent -> A

1 . Process1 protects A with Mutex
2. Process2 calls method on B which does:
    a) B parent: C
    b) (B's tmpRef to A) child: somethingElse *Wait for Mutex*
3. Process 1 makes B, C immutable, serializes B with C as parent.
4. Process2 changes A child.

It could of course be argued that is a programming/logic error to update 
B parent ref before child ref in A, (you'd actually have to do extra 
work to get that order), still it's a hard one to debug if it does happen.
(Note to the thread: Following the same logic, I would also say it 
should be considered an error to keep a mutable cache as part of an 
immutable object :) )

In fact, you *could* implement it with immutability, injecting handler 
contexts with behaviour as described above into existing/new processes 
for any resulting immutability errors.

Cheers,
Henry








More information about the Pharo-dev mailing list