- Pharo-dev - lists.pharo.org

Re: [Pharo-project] Fuel serialization - feature/bug

by Mariano Martinez Peck

On Tue, Jun 14, 2011 at 8:12 AM, Yanni Chiu <yanni(a)rogers.com> wrote: > On 13/06/11 6:56 PM, Yanni Chiu wrote: > >> On 13/06/11 5:49 PM, Mariano Martinez Peck wrote: >> >>> >>> mmmmmm so you are using a stream to serialize and such stream is also >>> being refenced from the object to serialize? >>> >> >> Yes, that's it - except that the stream is an FSWriteStream on a >> FSMemoryStore "file", which I think is an issue here. The PRKernel is >> holding an instance of PRFuelPersistency, which holds an FSReference. >> That FSReference happens to refer to a memory-based "file". >> > > The code is at http://ss3.gemstone.com/ss/pierfuel.html > > You'll first need a Pier image, with the following loaded: > > (ConfigurationOfFilesystem project version: '2.0.1-baseline') load > > (ConfigurationOfFuel project version: '1.4') load > OR > (ConfigurationOfFuel project version: '1.5-baseline') load > > If you run PRFuelPersistencyTest, then you'll see the stack trace. I saw > the same problem with 1.4 and with 1.5-baseline. > > In the 1.5-baseline, I added #fuelIgnoredInstanceVariableNames to > PRFuelPersistency and FSMemoryStore (to ignore 'directory' and 'root', > respectively) but the same problem remained. So I think it's a bug. > > Yes, We think you have spot a bug :) Martin is working on that. We will let you know. Thanks > Despite the test failing, the PRFuelPersistency works as expected to > save/restore a Pier kernel to/from a file. The file is 30 times smaller than > the corresponding (uncompressed) SIXX file. > > > and speed? could you measure serialization / materialization time ? Thanks a lot, -- Mariano http://marianopeck.wordpress.com

June 14, 2011

Re: [Pharo-project] Fuel serialization - feature/bug

by Mariano Martinez Peck

On Tue, Jun 14, 2011 at 12:56 AM, Yanni Chiu <yanni(a)rogers.com> wrote: > On 13/06/11 5:49 PM, Mariano Martinez Peck wrote: > >> >> Why the following is not enought? >> >> (FLSerializer on: myStream) serialize: myPRKernel. >> >> and >> >> (FLMaterializer on: myStream) materialize >> > > Yes, that's pretty much the code. Except that it's in a test case that uses > FSMemoryStore. The tests run a lot faster when writing to memory, instead of > to disk. Yes, but if you want to serialize in memory you can also use for example: myStream := (MultiByteBinaryOrTextStream on: ''). (FLSerializer on: myStream) serialize: myPRKernel. ... myStream reset. (FLMaterializer on: myStream) materialize That would use a stream in memory. And even more, you can serialize in memory with Fuel this way: testSerializationAndMaterializationInMemory | byteArray materializedObject | byteArray := FLSerializer serializeInMemory: 'mariano'. materializedObject := FLMaterializer materializeFromByteArray: byteArray. self assert: materializedObject equals: 'mariano'. > > > The FSMemoryStore holds the "file" bytes in a ByteArray. During >> serialization, I got a stack trace with KeyNotFound due to >> FLSerializer>>nextPutReferenceTo: when the object is the "file" >> ByteArray. >> >> Since this ByteArray is where the serialization is being written to, >> >> >> mmmmmm so you are using a stream to serialize and such stream is also >> being refenced from the object to serialize? >> > > Yes, that's it - except that the stream is an FSWriteStream on a > FSMemoryStore "file", which I think is an issue here. The PRKernel is > holding an instance of PRFuelPersistency, which holds an FSReference. That > FSReference happens to refer to a memory-based "file". > > > -- Mariano http://marianopeck.wordpress.com

June 14, 2011

Re: [Pharo-project] Toward MVP

by StÃ©phane Ducasse

code looks good to me. > > It's not big deal, but neither should be creating it. > > Start with a model: > > EventLogger>>initialize > super initialize. > quickList := ListModel new list:#( 'Start' 'BP down' 'BP up' 'End' '***SNAFU***' ); yourself. > fileSequence := ... "the log" > > Add the attached code. Among the important features are transparent handling of the selection index for the list and having something (following Dolphin, call them presenters) between the model and views (morphs). Models should not know or care about their selection index; the morph/view should do that (in Pharo, the ListPresenter is the natural home for it), and can do so in a way that "just works" for most purposes. > > The text presenters have been more trouble than I expected, but they bring with them some very important concepts. Dolphin's ViewResource is looking for a home (whether it will find one is open for bets), and the volley of events between model, view and presenter is starting to appear. Among the important tests (not yet realized as unit tests - sorry) is one model with multiple views; change the model and the views update. > > Dolphin's layout managers are very powerful, and while the pieces to make them exist in Pharo, I have yet to master their use. A common trap is to use top-level views when something composable is possible and desirable. I am not yet sure what to do about that. > > Bill >

June 14, 2011

Re: [Pharo-project] Fuel version 1.4 uses #isBinary, and SIXX uses #tab:

by Martin Dias

On Mon, Jun 13, 2011 at 7:00 PM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > > > On Mon, Jun 13, 2011 at 11:52 PM, Yanni Chiu <yanni(a)rogers.com> wrote: > >> FLBufferedWriteStream>>sizeBuffer: calls #isBinary, which is not currently >> implemented by FSWriteStream - probably because #isBinary is not specified >> by ANSI. Is there a reason why the stream is not always binary? >> >> > I think no and I think you are correct. Fuel is binary, and > FLBufferedWriteStream is only for Fuel, so we remove that #isBinary. Martin, > do you agree? > Yes, I agree. You know more than me about the buffered stream. The only objection I see is the open issue we have about supporting an alternative text format in Fuel... How can I put a String in a FSWriteStream? > > >> Also, when using SIXX with FSWriteStream, I had to add the non-ANSI #tab: >> method to FSWriteStream. There's probably a dozen calls to #tab: in the SIXX >> code. Should #tab: be added to FSWriteStream somehow? Maybe a non-ANSI >> streams extension package? >> >> >> > Esteban Lorenzano wanted also to use Colin's FileSystem in Fuel instead of > the regular streams and that's why we have created the package > FuelFileSystem. So you can add all the methods you want to FSWriteStream > under a category *FuelFileSystem. I can give you commit access to Fuel repo > if you want. > > I still don't understand why you cannot directly serialize/materialize the > PRKernel instance. > > -- > Mariano > http://marianopeck.wordpress.com > >

June 14, 2011

Re: [Pharo-project] Fuel serialization - feature/bug

by Yanni Chiu

On 13/06/11 6:56 PM, Yanni Chiu wrote: > On 13/06/11 5:49 PM, Mariano Martinez Peck wrote: >> >> mmmmmm so you are using a stream to serialize and such stream is also >> being refenced from the object to serialize? > > Yes, that's it - except that the stream is an FSWriteStream on a > FSMemoryStore "file", which I think is an issue here. The PRKernel is > holding an instance of PRFuelPersistency, which holds an FSReference. > That FSReference happens to refer to a memory-based "file". The code is at http://ss3.gemstone.com/ss/pierfuel.html You'll first need a Pier image, with the following loaded: (ConfigurationOfFilesystem project version: '2.0.1-baseline') load (ConfigurationOfFuel project version: '1.4') load OR (ConfigurationOfFuel project version: '1.5-baseline') load If you run PRFuelPersistencyTest, then you'll see the stack trace. I saw the same problem with 1.4 and with 1.5-baseline. In the 1.5-baseline, I added #fuelIgnoredInstanceVariableNames to PRFuelPersistency and FSMemoryStore (to ignore 'directory' and 'root', respectively) but the same problem remained. So I think it's a bug. Despite the test failing, the PRFuelPersistency works as expected to save/restore a Pier kernel to/from a file. The file is 30 times smaller than the corresponding (uncompressed) SIXX file.

June 14, 2011

Re: [Pharo-project] Fuel version 1.4 uses #isBinary, and SIXX uses #tab:

by Yanni Chiu

On 13/06/11 6:00 PM, Mariano Martinez Peck wrote: > > Esteban Lorenzano wanted also to use Colin's FileSystem in Fuel instead > of the regular streams and that's why we have created the package > FuelFileSystem. So you can add all the methods you want to > FSWriteStream under a category *FuelFileSystem. I can give you commit > access to Fuel repo if you want. Since FileSystem is still under development, I'd hope that FuelFileSystem would be unnecessary - useful protocol would be added to FileSystem directly (or its usage would be avoided by Fuel). The only issue I've had so far is the #isBinary call. > I still don't understand why you cannot directly serialize/materialize > the PRKernel instance. That does work, so I'm surprised about the test case too. One test case does a single snapshot, which works. The other test case does three snapshots, and fails when trying to do the second snapshot (due to the KeyNotFound error described in the other post). I'm almost done, so I'll push the code somewhere soon.

June 13, 2011

Re: [Pharo-project] Fuel serialization - feature/bug

by Yanni Chiu

On 13/06/11 5:49 PM, Mariano Martinez Peck wrote: > > Why the following is not enought? > > (FLSerializer on: myStream) serialize: myPRKernel. > > and > > (FLMaterializer on: myStream) materialize Yes, that's pretty much the code. Except that it's in a test case that uses FSMemoryStore. The tests run a lot faster when writing to memory, instead of to disk. > The FSMemoryStore holds the "file" bytes in a ByteArray. During > serialization, I got a stack trace with KeyNotFound due to > FLSerializer>>nextPutReferenceTo: when the object is the "file" > ByteArray. > > Since this ByteArray is where the serialization is being written to, > > > mmmmmm so you are using a stream to serialize and such stream is also > being refenced from the object to serialize? Yes, that's it - except that the stream is an FSWriteStream on a FSMemoryStore "file", which I think is an issue here. The PRKernel is holding an instance of PRFuelPersistency, which holds an FSReference. That FSReference happens to refer to a memory-based "file".

June 13, 2011

Re: [Pharo-project] Fuel version 1.4 uses #isBinary, and SIXX uses #tab:

by Mariano Martinez Peck

On Mon, Jun 13, 2011 at 11:52 PM, Yanni Chiu <yanni(a)rogers.com> wrote: > FLBufferedWriteStream>>sizeBuffer: calls #isBinary, which is not currently > implemented by FSWriteStream - probably because #isBinary is not specified > by ANSI. Is there a reason why the stream is not always binary? > > I think no and I think you are correct. Fuel is binary, and FLBufferedWriteStream is only for Fuel, so we remove that #isBinary. Martin, do you agree? > Also, when using SIXX with FSWriteStream, I had to add the non-ANSI #tab: > method to FSWriteStream. There's probably a dozen calls to #tab: in the SIXX > code. Should #tab: be added to FSWriteStream somehow? Maybe a non-ANSI > streams extension package? > > > Esteban Lorenzano wanted also to use Colin's FileSystem in Fuel instead of the regular streams and that's why we have created the package FuelFileSystem. So you can add all the methods you want to FSWriteStream under a category *FuelFileSystem. I can give you commit access to Fuel repo if you want. I still don't understand why you cannot directly serialize/materialize the PRKernel instance. -- Mariano http://marianopeck.wordpress.com

June 13, 2011

[Pharo-project] Fuel version 1.4 uses #isBinary, and SIXX uses #tab:

by Yanni Chiu

FLBufferedWriteStream>>sizeBuffer: calls #isBinary, which is not currently implemented by FSWriteStream - probably because #isBinary is not specified by ANSI. Is there a reason why the stream is not always binary? Also, when using SIXX with FSWriteStream, I had to add the non-ANSI #tab: method to FSWriteStream. There's probably a dozen calls to #tab: in the SIXX code. Should #tab: be added to FSWriteStream somehow? Maybe a non-ANSI streams extension package?

June 13, 2011

Re: [Pharo-project] Fuel serialization - feature/bug

by Mariano Martinez Peck

On Mon, Jun 13, 2011 at 11:28 PM, Yanni Chiu <yanni(a)rogers.com> wrote: > In my test case for serializing a Pier kernel with Fuel, a FSMemoryStore > (from the FileSystem package) is used. I really didn't understand, sorry :( Why the following is not enought? (FLSerializer on: myStream) serialize: myPRKernel. and (FLMaterializer on: myStream) materialize > The FSMemoryStore holds the "file" bytes in a ByteArray. During > serialization, I got a stack trace with KeyNotFound due to > FLSerializer>>nextPutReferenceTo: when the object is the "file" ByteArray. > > Since this ByteArray is where the serialization is being written to, mmmmmm so you are using a stream to serialize and such stream is also being refenced from the object to serialize? > then it's hash value is changing, so Fuel serialization cannot find it in > the instancesIndex. > > I think this is expected behaviour, and the fix is to not serialize certain > inst vars. I was using the "stable" 1.4 version. Do I need to change to the > 1.5 baseline to get the #fuelIgnoredInstanceVariableNames feature? > > -- > Yanni > > > -- Mariano http://marianopeck.wordpress.com

June 13, 2011