[Pharo-project] Is there an adhoc header field in Fuel?
I'd like to store some metadata in a Fuel serialized file. I'd like to be able to read that metadata, without de-serializing the file. So, if there were a fixed header at the start of the file, with some "unused" fields, that would be ideal.
Hi! We don't have unused fields, but there is no problem to add your own prefix: FileStream newFileNamed: 'demoMetadata.fuel' do: [:aStream | aStream binary. aStream nextStringPut: 'metadata=5'. FLSerializer newDefault serialize: 'stuff' on: aStream ]. FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | aStream binary. metadata := aStream nextString. stuff := (FLMaterializer newDefault materializeFrom: aStream) root ]. Is this ok for you? Best regards, Martin On Thu, Feb 9, 2012 at 8:25 PM, Yanni Chiu <yanni@rogers.com> wrote:
I'd like to store some metadata in a Fuel serialized file. I'd like to be able to read that metadata, without de-serializing the file. So, if there were a fixed header at the start of the file, with some "unused" fields, that would be ideal.
On 09/02/12 10:20 PM, Martin Dias wrote:
Hi! We don't have unused fields, but there is no problem to add your own prefix: [...]
Is this ok for you?
It'll do the job. Here's what the file would have: yanni$ od -c demoMetadata.fuel 0000000 \n m e t a d a t a = 5 F U E L \0 0000020 021 017 F L U I n t 1 6 C l u s t e 0000040 r \0 \0 \0 001 \0 \0 \0 001 \0 \0 \0 \0 023 F L 0000060 B y t e S t r i n g C l u s t e 0000100 r \0 \0 \0 001 005 s t u f f \0 001 0000115 It's strange that there is a '\n' at the start. [Okay, figured it out, it's the character count.] Having "FUEL" and version number appear some where other than the start of file doesn't feel right. And, for sure, FLMaterializer will not just read the file, without some prior code to position the stream past the metadata - i.e. the file is no longer in "standard" Fuel format. Would Fuel have a problem with extra bytes at the end?
On Fri, Feb 10, 2012 at 7:01 AM, Yanni Chiu <yanni@rogers.com> wrote:
On 09/02/12 10:20 PM, Martin Dias wrote:
Hi! We don't have unused fields, but there is no problem to add your own prefix: [...]
Is this ok for you?
It'll do the job. Here's what the file would have:
yanni$ od -c demoMetadata.fuel 0000000 \n m e t a d a t a = 5 F U E L \0 0000020 021 017 F L U I n t 1 6 C l u s t e 0000040 r \0 \0 \0 001 \0 \0 \0 001 \0 \0 \0 \0 023 F L 0000060 B y t e S t r i n g C l u s t e 0000100 r \0 \0 \0 001 005 s t u f f \0 001 0000115
It's strange that there is a '\n' at the start. [Okay, figured it out, it's the character count.]
Having "FUEL" and version number appear some where other than the start of file doesn't feel right. And, for sure, FLMaterializer will not just read the file, without some prior code to position the stream past the metadata - i.e. the file is no longer in "standard" Fuel format.
Would Fuel have a problem with extra bytes at the end?
No. Fuel doesn't care. All you need to do is to send it a stream. If the stream is already at a certain position, it doesn't matter it will start from that position. Say you have stream where you write something first and then you give it to Fuel to serialize. The stream would be at some position different than zero. Say, at position 10. Fuel will start serializing for there. Say we have written 40 bytes. Then, at materialization, when you give it the stream, you have to be sure to give the stream starting at the position 10. Fuel will load EXACTLY 40 bytes and will stop. The stream can continue....fuel doesn't care and it will store exactly when it finishes reading his own bytes. BTW, this is how some databases work. The save several serialized objects in the same stream... Cheers -- Mariano http://marianopeck.wordpress.com
Mariano Martinez Peck wrote:
On Fri, Feb 10, 2012 at 7:01 AM, Yanni Chiu <yanni@rogers.com> wrote:
On 09/02/12 10:20 PM, Martin Dias wrote:
Hi! We don't have unused fields, but there is no problem to add your own prefix: [...]
Is this ok for you?
It'll do the job. Here's what the file would have:
yanni$ od -c demoMetadata.fuel 0000000 \n m e t a d a t a = 5 F U E L \0 0000020 021 017 F L U I n t 1 6 C l u s t e 0000040 r \0 \0 \0 001 \0 \0 \0 001 \0 \0 \0 \0 023 F L 0000060 B y t e S t r i n g C l u s t e 0000100 r \0 \0 \0 001 005 s t u f f \0 001 0000115
It's strange that there is a '\n' at the start. [Okay, figured it out, it's the character count.]
Having "FUEL" and version number appear some where other than the start of file doesn't feel right. And, for sure, FLMaterializer will not just read the file, without some prior code to position the stream past the metadata - i.e. the file is no longer in "standard" Fuel format.
Would Fuel have a problem with extra bytes at the end?
No. Fuel doesn't care. If you consider Fuel "the technology" you may be right. All you need to do is to send it a stream. If the stream is already at a certain position, it doesn't matter it will start from that position. Say you have stream where you write something first and then you give it to Fuel to serialize. The stream would be at some position different than zero. Say, at position 10. Fuel will start serializing for there. Say we have written 40 bytes. Then, at materialization, when you give it the stream, you have to be sure to give the stream starting at the position 10. However if you consider Fuel "the standard file format" - this would appear to be broken by people using random offsets. Fuel will load EXACTLY 40 bytes and will stop. The stream can continue....fuel doesn't care and it will store exactly when it finishes reading his own bytes.
BTW, this is how some databases work. The save several serialized objects in the same stream...
To reach for the moon... Perhaps Fuel "the standard file format" holding multiple streams might be formalized by tagging different each stream with a 'name' and/or a 'type' and/or 'deserializerMethod'. The first two of these might be '#fuel -> #fuelDeserializer' and '#meta->#textDeserializer'. The the default Fuel system can then automatically ignore 3rd-party data.
Cheers
All you need to do is to send it a stream. If the
stream is already at a certain position, it doesn't matter it will start from that position. Say you have stream where you write something first and then you give it to Fuel to serialize. The stream would be at some position different than zero. Say, at position 10. Fuel will start serializing for there. Say we have written 40 bytes. Then, at materialization, when you give it the stream, you have to be sure to give the stream starting at the position 10.
However if you consider Fuel "the standard file format" - this would appear to be broken by people using random offsets.
yes, sure. But I think its ok to fail from the serializer point of view. It is YOUR responsability to provide the correct streams.
Fuel will load EXACTLY 40 bytes and will stop. The stream can
continue....fuel doesn't care and it will store exactly when it finishes reading his own bytes.
BTW, this is how some databases work. The save several serialized objects in the same stream...
To reach for the moon... Perhaps Fuel "the standard file format" holding multiple streams might be formalized by tagging different each stream with a 'name' and/or a 'type' and/or 'deserializerMethod'. The first two of these might be '#fuel -> #fuelDeserializer' and '#meta->#textDeserializer'.
I don't understand. Fuel holding multiple streams ? why? for what?
The the default Fuel system can then automatically ignore 3rd-party data.
I want to make my point clear: Martin proposed an example that solves Yanni problem. If you serialize with a stream with a certain position, then you have to materialize it with its correct position. Dot. It is perfect that the materializer is broken if you do not give him the correct stream. Now, I DO agree that there could be another solution rather than adhoc, supported by fuel so that BY DEFAULT some data could be set and ignored at the same time...so that there is no manipulation of the srteam by the user.
Cheers
-- Mariano http://marianopeck.wordpress.com
On 10 February 2012 10:19, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
All you need to do is to send it a stream. If the stream is already at a certain position, it doesn't matter it will start from that position. Say you have stream where you write something first and then you give it to Fuel to serialize. The stream would be at some position different than zero. Say, at position 10. Fuel will start serializing for there. Say we have written 40 bytes. Â Then, at materialization, when you give it the stream, you have to be sure to give the stream starting at the position 10.
However if you consider Fuel "the standard file format" - this would appear to be broken by people using random offsets.
yes, sure. But I think its ok to fail from the serializer point of view. It is YOUR responsability to provide the correct streams.
Fuel will load EXACTLY 40 bytes and will stop. The stream can continue....fuel doesn't care and it will store exactly when it finishes reading his own bytes.
BTW, this is how some databases work. The save several serialized objects in the same stream...
To reach for the moon... Perhaps Fuel "the standard file format" holding multiple streams might be formalized by tagging different each stream with a 'name' and/or a 'type' and/or 'deserializerMethod'. Â The first two of these might be '#fuel -> #fuelDeserializer' and '#meta->#textDeserializer'.
I don't understand. Fuel holding multiple streams ? why? for what?
 The the default Fuel system can then automatically ignore 3rd-party data.
I want to make my point clear: Martin proposed an example that solves Yanni problem. If you serialize with a stream with a certain position, then you have to materialize it with its correct position. Dot. It is perfect that the materializer is broken if you do not give him the correct stream. Now, I DO agree that there could be another solution rather than adhoc, supported by fuel so that BY DEFAULT some data could be set and ignored at the same time...so that there is no manipulation of the srteam by the user.
no i think your approach is fine. you store what you need, and don't care/intervene with other parts of stream, so if user wants to add some stuff before materialized data, he free to do so. -- Best regards, Igor Stasenko.
Maybe is too much overhead, but it is also possible to serialize an object that represents the metadata as a prefix: FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream | aStream binary. metadata := Dictionary with: #meta -> 5. content := 'stuff'. FLSerializer newDefault serialize: metadata on: aStream; serialize: content on: aStream ]. FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | | aMaterializer | aStream binary. aMaterializer := FLMaterializer newDefault. metadata := (aMaterializer materializeFrom: aStream) root. content := (aMaterializer materializeFrom: aStream) root ]. Martin On Fri, Feb 10, 2012 at 8:25 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 10 February 2012 10:19, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
All you need to do is to send it a stream. If the stream is already at a certain position, it doesn't matter it will
start
from that position. Say you have stream where you write something first and then you give it to Fuel to serialize. The stream would be at some position different than zero. Say, at position 10. Fuel will start serializing for there. Say we have written 40 bytes. Then, at materialization, when you give it the stream, you have to be sure to give the stream starting at the position 10.
However if you consider Fuel "the standard file format" - this would appear to be broken by people using random offsets.
yes, sure. But I think its ok to fail from the serializer point of view. It is YOUR responsability to provide the correct streams.
Fuel will load EXACTLY 40 bytes and will stop. The stream can continue....fuel doesn't care and it will store exactly when it
finishes
reading his own bytes.
BTW, this is how some databases work. The save several serialized objects in the same stream...
To reach for the moon... Perhaps Fuel "the standard file format" holding multiple streams might be formalized by tagging different each stream with a 'name' and/or a 'type' and/or 'deserializerMethod'. The first two of these might be '#fuel -> #fuelDeserializer' and '#meta->#textDeserializer'.
I don't understand. Fuel holding multiple streams ? why? for what?
The the default Fuel system can then automatically ignore 3rd-party
data.
I want to make my point clear: Martin proposed an example that solves Yanni problem. If you serialize with a stream with a certain position, then you have to materialize it with its correct position. Dot. It is perfect that the materializer is broken if you do not give him the correct stream. Now, I DO agree that there could be another solution rather than adhoc, supported by fuel so that BY DEFAULT some data could be set and ignored at the same time...so that there is no manipulation of the srteam by the user.
no i think your approach is fine. you store what you need, and don't care/intervene with other parts of stream, so if user wants to add some stuff before materialized data, he free to do so.
-- Best regards, Igor Stasenko.
On 10 February 2012 16:24, Martin Dias <tinchodias@gmail.com> wrote:
Maybe is too much overhead, but it is also possible to serialize an object that represents the metadata as a prefix:
FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream | aStream binary. metadata := Dictionary with: #meta -> 5. content := 'stuff'. FLSerializer newDefault serialize: metadata on: aStream; serialize: content on: aStream ].
FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | | aMaterializer | aStream binary. aMaterializer := FLMaterializer newDefault. metadata := (aMaterializer materializeFrom: aStream) root. content := (aMaterializer materializeFrom: aStream) root ].
Martin
which eventually could be turned into simple: MyStuff>>serializeOn: stream serializer := FLSerializer newDefault. serializer serialize: {myStuff. myMetaStuff. myMetaMetaStuff } on: stream ]. so, you don't even need to care about stream contents :) -- Best regards, Igor Stasenko.
On 10/02/12 10:24 AM, Martin Dias wrote:
Maybe is too much overhead, but it is also possible to serialize an object that represents the metadata as a prefix:
FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream | aStream binary. metadata := Dictionary with: #meta -> 5. content := 'stuff'. FLSerializer newDefault serialize: metadata on: aStream; serialize: content on: aStream ].
FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | | aMaterializer | aStream binary. aMaterializer := FLMaterializer newDefault. metadata := (aMaterializer materializeFrom: aStream) root. content := (aMaterializer materializeFrom: aStream) root ].
I like this solution best, since it doesn't change the Fuel "file format". I added some helper methods to my code base to make this usage scenario more obvious: FLMaterializer>>nextMaterializedRootFrom: sourceStream ^ (self materializeFrom: sourceStream) root FLMaterializer>>allMaterializedRootsFrom: sourceStream | contents | contents := OrderedCollection new. [ sourceStream atEnd ] whileFalse: [ contents add: (self nextMaterializedRootFrom: sourceStream) ]. ^ contents Sample usage: FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream | | serializer | aStream binary. serializer := FLSerializer newDefault. (1 to: 5) do: [ :each | serializer serialize: (Dictionary with: each -> (each * each)) on: aStream ]. ]. FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | | aMaterializer | aStream binary. aMaterializer := FLMaterializer newDefault. aMaterializer allMaterializedRootsFrom: aStream ]. FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | | aMaterializer | aStream binary. aMaterializer := FLMaterializer newDefault. aMaterializer nextMaterializedRootFrom: aStream. aMaterializer nextMaterializedRootFrom: aStream. ].
On Fri, Feb 10, 2012 at 2:45 PM, Yanni Chiu <yanni@rogers.com> wrote:
On 10/02/12 10:24 AM, Martin Dias wrote:
Maybe is too much overhead, but it is also possible to serialize an object that represents the metadata as a prefix:
FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream | aStream binary. metadata := Dictionary with: #meta -> 5. content := 'stuff'. FLSerializer newDefault serialize: metadata on: aStream; serialize: content on: aStream ].
FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | | aMaterializer | aStream binary. aMaterializer := FLMaterializer newDefault. metadata := (aMaterializer materializeFrom: aStream) root. content := (aMaterializer materializeFrom: aStream) root ].
I like this solution best, since it doesn't change the Fuel "file format".
I added some helper methods to my code base to make this usage scenario more obvious:
FLMaterializer>>**nextMaterializedRootFrom: sourceStream ^ (self materializeFrom: sourceStream) root
FLMaterializer>>**allMaterializedRootsFrom: sourceStream | contents | contents := OrderedCollection new. [ sourceStream atEnd ] whileFalse: [ contents add: (self nextMaterializedRootFrom: sourceStream) ]. ^ contents
Sample usage:
FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream | | serializer | aStream binary. serializer := FLSerializer newDefault. (1 to: 5) do: [ :each | serializer serialize: (Dictionary with: each -> (each * each)) on: aStream ].
].
FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | | aMaterializer | aStream binary. aMaterializer := FLMaterializer newDefault. aMaterializer allMaterializedRootsFrom: aStream ].
FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream | | aMaterializer | aStream binary. aMaterializer := FLMaterializer newDefault. aMaterializer nextMaterializedRootFrom: aStream. aMaterializer nextMaterializedRootFrom: aStream. ].
Great. Maybe we should include #nextMaterializedRootFrom: to fuel. Based on metadata you might decide to not materialize the "real" content. Like: metadata := (aMaterializer nextMaterializedRootFrom: aStream). [ metadata isWhatINeed ] ifTrue: [ realContent := aMaterializer nextMaterializedRootFrom: aStream ].
On 10/02/12 1:03 PM, Martin Dias wrote:
Great. Maybe we should include #nextMaterializedRootFrom: to fuel.
That would be great. Probably best to leave off #allMaterializedRootsFrom: - it seemed necessary for completeness, but I've no actual need for it.
Based on metadata you might decide to not materialize the "real" content. Like:
metadata := (aMaterializer nextMaterializedRootFrom: aStream). [ metadata isWhatINeed ] ifTrue: [ realContent := aMaterializer nextMaterializedRootFrom: aStream ].
Yes, that's the usage scenario.
On Fri, Feb 10, 2012 at 1:19 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
All you need to do is to send it a stream. If the
stream is already at a certain position, it doesn't matter it will start from that position. Say you have stream where you write something first and then you give it to Fuel to serialize. The stream would be at some position different than zero. Say, at position 10. Fuel will start serializing for there. Say we have written 40 bytes. Then, at materialization, when you give it the stream, you have to be sure to give the stream starting at the position 10.
However if you consider Fuel "the standard file format" - this would appear to be broken by people using random offsets.
yes, sure. But I think its ok to fail from the serializer point of view. It is YOUR responsability to provide the correct streams.
Fuel will load EXACTLY 40 bytes and will stop. The stream can
continue....fuel doesn't care and it will store exactly when it finishes reading his own bytes.
BTW, this is how some databases work. The save several serialized objects in the same stream...
To reach for the moon... Perhaps Fuel "the standard file format" holding multiple streams might be formalized by tagging different each stream with a 'name' and/or a 'type' and/or 'deserializerMethod'. The first two of these might be '#fuel -> #fuelDeserializer' and '#meta->#textDeserializer'.
I don't understand. Fuel holding multiple streams ? why? for what?
e.g. for source code. Imagine storing Monticello packages in Fuel so that the source code is stored in the Fuel file and doesn't have to be written to the changes file. Instead, a more powerful SourceFilesArray (actually a source file manager) can maintain a set of source files, including Fuel files, and fetch source there-from. Hence loading is faster, and the changes file isn't polluted with code loads, containing only one's own code. e.g. for external resources. imagine loading a Fuel package that also contains some dlls that can be unpacked as required. But if you choose this route let me strongly suggest you use the zip file format. Its very useful with Monticello mcz's to be able to pick them apart using unzip.
The the default Fuel system can then automatically ignore 3rd-party data.
I want to make my point clear: Martin proposed an example that solves Yanni problem. If you serialize with a stream with a certain position, then you have to materialize it with its correct position. Dot. It is perfect that the materializer is broken if you do not give him the correct stream. Now, I DO agree that there could be another solution rather than adhoc, supported by fuel so that BY DEFAULT some data could be set and ignored at the same time...so that there is no manipulation of the srteam by the user.
Cheers
-- Mariano http://marianopeck.wordpress.com
-- best, Eliot
On 10 February 2012 19:27, Eliot Miranda <eliot.miranda@gmail.com> wrote:
But if you choose this route let me strongly suggest you use the zip file format. Â Its very useful with Monticello mcz's to be able to pick them apart using unzip.
yes but then it makes the whole thing completely orthogonal to Fuel. because in zip you can have one file to hold serialized stuff, in another your metadata and whatsoever.. i mean, you can organize your data packaging yourself, without forcing fuel to support anything related to metadata. and i think this is right direction. So. I think it is useless to try extending fuel with metadata. if people want to have metadata, they should make it by own, in the way they see fit. I think fuel should do one thing : serialize/materialize stuff. Lets not turn it into "i can do everything to please you" library. -- Best regards, Igor Stasenko.
Am 10.02.2012 um 19:51 schrieb Igor Stasenko:
On 10 February 2012 19:27, Eliot Miranda <eliot.miranda@gmail.com> wrote:
But if you choose this route let me strongly suggest you use the zip file format. Its very useful with Monticello mcz's to be able to pick them apart using unzip.
yes but then it makes the whole thing completely orthogonal to Fuel. because in zip you can have one file to hold serialized stuff, in another your metadata and whatsoever.. i mean, you can organize your data packaging yourself, without forcing fuel to support anything related to metadata.
and i think this is right direction.
So. I think it is useless to try extending fuel with metadata. if people want to have metadata, they should make it by own, in the way they see fit. I think fuel should do one thing : serialize/materialize stuff. Lets not turn it into "i can do everything to please you" library.
+1 Fuel does what it does. Having a fuel with metadata is just a customized format. In the end it is all streams. You can open a file read the metadata and then hand over the stream to fuel. I don't see any need to tweak fuel for that. Changing it to be able to read multiple roots sounds great but has the potential for e.g. doubling objects and all sorts (if I understand it correct). Norbert
Norbert Hartl wrote:
Am 10.02.2012 um 19:51 schrieb Igor Stasenko:
On 10 February 2012 19:27, Eliot Miranda <eliot.miranda@gmail.com> wrote:
But if you choose this route let me strongly suggest you use the zip file format. Its very useful with Monticello mcz's to be able to pick them apart using unzip.
yes but then it makes the whole thing completely orthogonal to Fuel. because in zip you can have one file to hold serialized stuff, in another your metadata and whatsoever.. i mean, you can organize your data packaging yourself, without forcing fuel to support anything related to metadata.
and i think this is right direction.
So. I think it is useless to try extending fuel with metadata. if people want to have metadata, they should make it by own, in the way they see fit. I think fuel should do one thing : serialize/materialize stuff. Lets not turn it into "i can do everything to please you" library.
+1
Fuel does what it does. Having a fuel with metadata is just a customized format. In the end it is all streams. You can open a file read the metadata and then hand over the stream to fuel. I don't see any need to tweak fuel for that. Changing it to be able to read multiple roots sounds great but has the potential for e.g. doubling objects and all sorts (if I understand it correct).
Norbert
Using a standard container is better resuse and design - but maybe you still want to specify exactly the format of a file ending in "*.fuel". Is it _always_ a zip file or do you look for the zipfile magic number and if that is not there revert to it being a single fuel stream. cheers, -ben
I don't understand. Fuel holding multiple streams ? why? for what?
e.g. for source code. Imagine storing Monticello packages in Fuel so that the source code is stored in the Fuel file and doesn't have to be written to the changes file. Instead, a more powerful SourceFilesArray (actually a source file manager) can maintain a set of source files, including Fuel files, and fetch source there-from. Hence loading is faster, and the changes file isn't polluted with code loads, containing only one's own code.
e.g. for external resources. imagine loading a Fuel package that also contains some dlls that can be unpacked as required.
But if you choose this route let me strongly suggest you use the zip file format. Its very useful with Monticello mcz's to be able to pick them apart using unzip.
Ok, I see. That makes sense. However, that won't be part of Fuel, but rather a tool build on top of it. Now we have the Fuel core a simple serializer. FuelMetalevel, an optional package build on top of Fuel core to be able to correctly serialize classes, methods, traits, etc. FuelPackageLoader, an optional package build on top of FuelMetalevel to experiment import/export of packages in a binary way. So....continue with that infrastructure, if there would be a Monticello integration with Fuel, then such package should take care about serializing different things in different streams. But so far, for Fuel core, and I think the best approach is to let is simple as it is now.
The the default Fuel system can then automatically ignore 3rd-party data.
I want to make my point clear: Martin proposed an example that solves Yanni problem. If you serialize with a stream with a certain position, then you have to materialize it with its correct position. Dot. It is perfect that the materializer is broken if you do not give him the correct stream. Now, I DO agree that there could be another solution rather than adhoc, supported by fuel so that BY DEFAULT some data could be set and ignored at the same time...so that there is no manipulation of the srteam by the user.
Cheers
-- Mariano http://marianopeck.wordpress.com
-- best, Eliot
-- Mariano http://marianopeck.wordpress.com
this is a good idea. On Feb 10, 2012, at 12:25 AM, Yanni Chiu wrote:
I'd like to store some metadata in a Fuel serialized file. I'd like to be able to read that metadata, without de-serializing the file. So, if there were a fixed header at the start of the file, with some "unused" fields, that would be ideal.
On 10/02/12 3:45 PM, Stéphane Ducasse wrote:
this is a good idea.
Hmmm. I think the .zip file approach is the "correct/better" approach to take to my metadata problem. This discussion did turn up a feature of Fuel that was not obvious to me before - namely, that you can read/write multiple materializations on a single stream.
On Sat, Feb 11, 2012 at 2:32 AM, Yanni Chiu <yanni@rogers.com> wrote:
On 10/02/12 3:45 PM, Stéphane Ducasse wrote:
this is a good idea.
Hmmm. I think the .zip file approach is the "correct/better" approach to take to my metadata problem.
This discussion did turn up a feature of Fuel that was not obvious to me before - namely, that you can read/write multiple materializations on a single stream.
I think that then this deserves at least a page in the documentation, right? -- Mariano http://marianopeck.wordpress.com
I think that then this deserves at least a page in the documentation,
right?
+1 I'll add something.
-- Mariano http://marianopeck.wordpress.com
participants (8)
-
Ben Coman -
Eliot Miranda -
Igor Stasenko -
Mariano Martinez Peck -
Martin Dias -
Norbert Hartl -
Stéphane Ducasse -
Yanni Chiu