[Pharo-project] Moving data into new package/image version
I'm porting some apps that were using simple image-based persistence (ReferenceStream) from 1.3 to 1.4. The first thing that I ran into was FS class name changes (e.g. FSReference -> FileReference). Since the class wasn't present, I got an errorKeyNotFound [1]. Naively tried: Smalltalk globals at: #FSReference put: FileReference; at: #FSFilesystem put: FileSystem; at: #FSAbsolutePath put: AbsolutePath; at: #FSUnixStore put: UnixStore. But got "primitive #primGetPosition: in MultiByteFileStream failed" from ReferenceStream>>next I have a few ideas, but has anyone gone successfully through this type of process before? What's my best bet? I've been following Fuel and STON (which is quite nice for parsing json from REST APIs b.t.w.), but I'm not clear if either would be helpful here. What do you suggest? Thanks, Sean [1] Stack trace (heavily snipped) SystemDictionary(Dictionary)>>errorKeyNotFound: Receiver: a SystemDictionary(lots of globals) Arguments and temporary variables: aKey: #FSReference Receiver's instance variables: tally: 2920 array: an Array(#MemoryHandle->MemoryHandle nil nil #BlockClosuresTestCase->Blo...etc... cachedClassNames: nil cachedNonClassNames: nil ... ReferenceStream(DataStream)>>readInstance Receiver: a ReferenceStream Arguments and temporary variables: instSize: 2 aSymbol: #FSReference refPosn: 50 anObject: nil newClass: nil Receiver's instance variables: byteStream: MultiByteFileStream: '/Volumes/Sean''s Universe/sean/.jenkins/jobs/...etc... topCall: nil basePos: 0 references: an IdentityDictionary() objects: an IdentityDictionary(0->#(nil) 5->a Resources 10->#Resources 21->an O...etc... currentReference: 55 fwdRefEnds: an IdentityDictionary() blockers: an IdentityDictionary() skipping: an IdentitySet() -- View this message in context: http://forum.world.st/Moving-data-into-new-package-image-version-tp4628820.h... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On Sat, May 12, 2012 at 3:03 PM, Sean P. DeNigris <sean@clipperadams.com>wrote:
I'm porting some apps that were using simple image-based persistence (ReferenceStream) from 1.3 to 1.4.
The first thing that I ran into was FS class name changes (e.g. FSReference -> FileReference). Since the class wasn't present, I got an errorKeyNotFound [1]. Naively tried:
Smalltalk globals at: #FSReference put: FileReference; at: #FSFilesystem put: FileSystem; at: #FSAbsolutePath put: AbsolutePath; at: #FSUnixStore put: UnixStore.
But got "primitive #primGetPosition: in MultiByteFileStream failed" from ReferenceStream>>next
I have a few ideas, but has anyone gone successfully through this type of process before? What's my best bet? I've been following Fuel and STON (which is quite nice for parsing json from REST APIs b.t.w.), but I'm not clear if either would be helpful here.
Sorry, I don't have ideas of how to solve that. However, I can only say that Fuel *does* suppor that type of problem (if I understood the problem correctly). The following test demonstrates an example: FLMigrationTest >> testClassRename | pointClass aPoint resultPoint pointClassName | pointClass := self newClassWithInstanceVariableNames: 'x y'. pointClassName := pointClass name. aPoint := pointClass new. aPoint instVarNamed: 'x' put: 7. aPoint instVarNamed: 'y' put: 11. self serialize: aPoint. pointClass renameSilently: (pointClassName, 'Renamed') asSymbol. self materializer migrateClassNamed: pointClassName toClass: pointClass. resultPoint := self materialized. self assert: (resultPoint instVarNamed: 'x') = 7. self assert: (resultPoint instVarNamed: 'y') = 11.
What do you suggest?
Thanks, Sean
[1] Stack trace (heavily snipped) SystemDictionary(Dictionary)>>errorKeyNotFound: Receiver: a SystemDictionary(lots of globals) Arguments and temporary variables: aKey: #FSReference Receiver's instance variables: tally: 2920 array: an Array(#MemoryHandle->MemoryHandle nil nil #BlockClosuresTestCase->Blo...etc... cachedClassNames: nil cachedNonClassNames: nil
...
ReferenceStream(DataStream)>>readInstance Receiver: a ReferenceStream Arguments and temporary variables: instSize: 2 aSymbol: #FSReference refPosn: 50 anObject: nil newClass: nil Receiver's instance variables: byteStream: MultiByteFileStream: '/Volumes/Sean''s Universe/sean/.jenkins/jobs/...etc... topCall: nil basePos: 0 references: an IdentityDictionary() objects: an IdentityDictionary(0->#(nil) 5->a Resources 10->#Resources 21->an O...etc... currentReference: 55 fwdRefEnds: an IdentityDictionary() blockers: an IdentityDictionary() skipping: an IdentitySet()
-- View this message in context: http://forum.world.st/Moving-data-into-new-package-image-version-tp4628820.h... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
Mariano Martinez Peck wrote
Sorry, I don't have ideas of how to solve that.
I ended up using OB to rename the classes in the old image to match the new class names in the new image, then serialized with ReferenceStream, then deserialized in the new image. Mariano Martinez Peck wrote
I can only say that Fuel *does* suppor that type of problem Would Fuel be better than ReferenceStream for persistence? I thought the use case was saving rarely, but loading often; where serialization is balanced between the two.
-- View this message in context: http://forum.world.st/Moving-data-into-new-package-image-version-tp4628820p4... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On Wed, May 16, 2012 at 10:37 PM, Sean P. DeNigris <sean@clipperadams.com>wrote:
Mariano Martinez Peck wrote
Sorry, I don't have ideas of how to solve that.
I ended up using OB to rename the classes in the old image to match the new class names in the new image, then serialized with ReferenceStream, then deserialized in the new image.
Good idea!
Mariano Martinez Peck wrote
I can only say that Fuel *does* suppor that type of problem Would Fuel be better than ReferenceStream for persistence?
Yes, much better.
I thought the use case was saving rarely, but loading often; where serialization is balanced between the two.
Well, even if that's true, that doesn't mean that Fuel serialization is slow. In fact, it is between 5x and 10x faster than ReferenceStream ;) and like 30x when loading. (I should check the paper for the numbers..i don't remember them). Fuel is *much* faster than ReferenceStream while also providing more features (as the one you have just experimented).
-- View this message in context: http://forum.world.st/Moving-data-into-new-package-image-version-tp4628820p4... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
A bit tangent to your problem, Sean. Just want to tell you trick which i use: Rule #1. never make more than 1 reference to any class external to your package in your code. an exception is kernel classes. which means, that you are allowed to do 'Set new' or 'Array new', in any place of your code but never 'SomeFooClass new' more than once. If your code having such dependency, localize it in single method: fooClass ^ FooClass and then use 'self fooClass' everywhere.. Like that, if you will figure out one day, that sand moves under your feets, you'll have to change only single method. :) -- Best regards, Igor Stasenko.
Igor Stasenko wrote
Rule #1. never make more than 1 reference to any class external to your package in your code. an exception is kernel classes.
Thanks, Igor. Of course I'm lazy and don't always do this, and in this case, I was treating FS as part of the kernel, I guess a little too prematurely. I sometimes extend this policy even to some kernel classes both because Pharo is progressing so quickly and it can help with testing... Sean -- View this message in context: http://forum.world.st/Moving-data-into-new-package-image-version-tp4628820p4... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On 16/05/12 4:37 PM, Sean P. DeNigris wrote:
Mariano Martinez Peck wrote
Sorry, I don't have ideas of how to solve that.
I ended up using OB to rename the classes in the old image to match the new class names in the new image, then serialized with ReferenceStream, then deserialized in the new image.
When I don't have the old image anymore, I create a subclass in the new image (e.g. FileReference subclass: FSReference), do the import, then fix up the class references. Mariano - I think the class rename feature is not available in Fuel-1.7 which is the current stable release. Is there a target date for 1.8 to become stable?
On Thu, May 17, 2012 at 9:37 PM, Yanni Chiu <yanni@rogers.com> wrote:
On 16/05/12 4:37 PM, Sean P. DeNigris wrote:
Mariano Martinez Peck wrote
Sorry, I don't have ideas of how to solve that.
I ended up using OB to rename the classes in the old image to match the new class names in the new image, then serialized with ReferenceStream, then deserialized in the new image.
When I don't have the old image anymore, I create a subclass in the new image (e.g. FileReference subclass: FSReference), do the import, then fix up the class references.
wow..how many good ideas :)
Mariano - I think the class rename feature is not available in Fuel-1.7 which is the current stable release.
That's correct.
Is there a target date for 1.8 to become stable?
this week :) And we will include it in Pharo 2.0 also. -- Mariano http://marianopeck.wordpress.com
Yanni Chiu wrote
I create a subclass in the new image (e.g. FileReference subclass: FSReference), do the import, then fix up the class references.
That could be very useful. How do you "fix up the class references"? What methods should I look at? -- View this message in context: http://forum.world.st/Moving-data-into-new-package-image-version-tp4628820p4... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On 17/05/12 3:51 PM, Sean P. DeNigris wrote:
That could be very useful. How do you "fix up the class references"? What methods should I look at?
Brute force: find all references to the old class (select class name, cmd-shift-N). Go through the list of methods, paste the new class name, accept the change. This works for code. If you have instances of the class, then I guess even more brute force is needed. Or, you could go back to your original method, using class re-naming. (Remember that this technique started with not having an old image, but only having the exported objects).
participants (5)
-
Francois Stephany -
Igor Stasenko -
Mariano Martinez Peck -
Sean P. DeNigris -
Yanni Chiu