[Pharo-dev] [Fuel] Serialize and materialize to GZip

Mariano Martinez Peck marianopeck at gmail.com
Wed Apr 30 09:04:10 EDT 2014

On Wed, Apr 30, 2014 at 6:33 AM, roberto.minelli at usi.ch <
roberto.minelli at usi.ch> wrote:

> Hi,
> Following the Fuel test FLUserGuidesTest>>#testGZip I discovered that I
> can serialize and materialize to and from GZip archives. Cool!
> I tried with some small test object and it worked like a charm. When I
> tried with a large object…
Yes, I have the exact same problem long ago. As far as I could debug at
that point in time, it was a bug in the compression plugin in the VM.
So there isn't anything easy you can do about.  See
for more details.

> Serialization
> - It does not work, the image instantly crashes
> Materialization
> - Since I was not able to serialize the large object on GZip, I serialized
> "normally" than compressed the resulting fuel file using GZipWriteStream
> class>>#compressFile:
> - When I materialize from the GZip archive, it takes forever and does not
> succeed after more than 1 minute. Note that the same non-Gzipped file takes
> a couple of seconds to be materialized.
If you serialized normally and the compressed, then to materialize you need
to first uncompress and then give fuel the decompressed version.

BTW...what I did for Fuel and compressing is a Pharo binding to a really
fast compressor/uncompressor: LZ4.


That is very fast, but it doesn't provide a streaming API. So the usage I
did was:

- Fuel serialize in memory to a bytearray
- LZ4 compress that bytearray and write compressed to file or whatever you
- LZ4 uncompress from file or from where you stored the compressed version.
- Fuel materialize from the compressed stream from previous step

this will work even for very large graphs.

There is also an experiment Max and I were doing about compressing strings
right inside Fuel:


But this is committed in the experiments repo. If you want to take a look...


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140430/a8365b30/attachment-0002.html>

More information about the Pharo-dev mailing list