Hello I have found a way round this problem, but it seems messy and uncomfortable, so I thought I should ask what is the right way to do it. I have just changed from using Moose 5.0 (Pharo 3.0, update: #30862) to Moose 5.1 (Pharo 4.0, update: #40613). I had a rather complex dictionary structure in a Playground in 5.0, which I wanted to reproduce exactly in the 5.1 image. Looking around, Fuel looked like the obvious tool to save and restore the structure. It saved OK in 5.0, but when I tried to re-materialize it in 5.1 I got an error message: 'FLBadVersion: Materialization error. Unexpected stream vers' (can't see the rest of the message). I could see from the debugger output that Fuel was looking for version 194 and seeing 193. Using a brute force approach, I hacked FLSerializer class >> currentVersion in the Moose 5.0 image to return 194 instead 193, then tried again, and of course everything worked OK. My question is what should I have done? How can I move data from one image to another of a later version? If Fuel has no backward compatibility, is there another tool I can use? Thanks for any advice Peter Kenny
On 28 May 2015, at 18:18, PBKResearch <peter@pbkresearch.co.uk> wrote:
Hello
I have found a way round this problem, but it seems messy and uncomfortable, so I thought I should ask what is the right way to do it.
I have just changed from using Moose 5.0 (Pharo 3.0, update: #30862) to Moose 5.1 (Pharo 4.0, update: #40613). I had a rather complex dictionary structure in a Playground in 5.0, which I wanted to reproduce exactly in the 5.1 image. Looking around, Fuel looked like the obvious tool to save and restore the structure. It saved OK in 5.0, but when I tried to re-materialize it in 5.1 I got an error message: âFLBadVersion: Materialization error. Unexpected stream versâ (canât see the rest of the message). I could see from the debugger output that Fuel was looking for version 194 and seeing 193. Using a brute force approach, I hacked FLSerializer class >> currentVersion in the Moose 5.0 image to return 194 instead 193, then tried again, and of course everything worked OK.
My question is what should I have done? How can I move data from one image to another of a later version? If Fuel has no backward compatibility, is there another tool I can use?
Depending on what the data is, STON can be quite useful. https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu...
Thanks for any advice
Peter Kenny
Hi, STON is the "document format" for a project I'm making about data narratives and visualization on Pharo. It has worked pretty fine to move information between images without any problem. I would recommend you to take a look on it. Cheers, Offray El 28/05/15 a las 11:32, Sven Van Caekenberghe escribió:
On 28 May 2015, at 18:18, PBKResearch <peter@pbkresearch.co.uk> wrote:
Hello
I have found a way round this problem, but it seems messy and uncomfortable, so I thought I should ask what is the right way to do it.
I have just changed from using Moose 5.0 (Pharo 3.0, update: #30862) to Moose 5.1 (Pharo 4.0, update: #40613). I had a rather complex dictionary structure in a Playground in 5.0, which I wanted to reproduce exactly in the 5.1 image. Looking around, Fuel looked like the obvious tool to save and restore the structure. It saved OK in 5.0, but when I tried to re-materialize it in 5.1 I got an error message: âFLBadVersion: Materialization error. Unexpected stream versâ (canât see the rest of the message). I could see from the debugger output that Fuel was looking for version 194 and seeing 193. Using a brute force approach, I hacked FLSerializer class >> currentVersion in the Moose 5.0 image to return 194 instead 193, then tried again, and of course everything worked OK.
My question is what should I have done? How can I move data from one image to another of a later version? If Fuel has no backward compatibility, is there another tool I can use?
Depending on what the data is, STON can be quite useful.
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu...
Thanks for any advice
Peter Kenny
Peter Kenny wrote
My question is what should I have done?
There is usually an overlap between the Fuel versions loadable in consecutive Pharo versions, so you have to either (via Metacello) upgrade the Fuel in your source image, or downgrade in the target image, so that you can save and move the data without conflict. ----- Cheers, Sean -- View this message in context: http://forum.world.st/How-to-move-data-between-Pharo-versions-tp4829102p4829... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Sean Thanks for this. At the back of my mind in asking was whether I want to use Fuel more extensively as a way of storing object structures long term on disk. If I move to Pharo 5 later on, will that have a new version of Fuel which will refuse to read my saved files, so that I will have to convert all my files to the new version? As far as I can see, I could not do that using the device you suggest; I would have to have one version of Fuel which could materialize files of version 193 and re-serialize to version 194. I see that both FLSerializer and FLMaterializer have class side methods called #currentVersion; could I set those to give different version numbers, and do the conversion that way? Alternatively, I could use Sven's suggestion of STON, provided that will not go the way of version changes without backward compatibility. I suppose a lot will depend on the efficiency of the two approaches, both in terms of storage space on disk and in conversion times. Thanks to both for the suggestions. I have food for thought and ideas for further experiment. Peter Kenny -----Original Message----- From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of Sean P. DeNigris Sent: 28 May 2015 17:39 To: pharo-users@lists.pharo.org Subject: Re: [Pharo-users] How to move data between Pharo versions Peter Kenny wrote
My question is what should I have done?
There is usually an overlap between the Fuel versions loadable in consecutive Pharo versions, so you have to either (via Metacello) upgrade the Fuel in your source image, or downgrade in the target image, so that you can save and move the data without conflict. ----- Cheers, Sean -- View this message in context: http://forum.world.st/How-to-move-data-between-Pharo-versions-tp4829102p4829 115.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Peter Kenny wrote
I will have to convert all my files to the new version? As far as I can see, I could not do that using the device you suggest
The thing is that it's only the serialization that is tied to a Fuel version, so the workflow is (and I've done this to port my data from Pharo 3.0 to Pharo 4.0 and then to Pharo 5.0): 1. In the older image with the data, upgrade Fuel to the latest version that will load in that Pharo `ConfigurationOfFuel project stableVersion load` 2. Serialize the data If the Fuel version in #1 is the one that comes with the newer Pharo you're porting to, you're done. You can now materialize in the newer Pharo version. Otherwise... 3. In the newer Pharo a. Downgrade Fuel e.g `(ConfigurationOfFuel project version: versionStringFromNumberOneAbove) load` b. Materialize the data c. Upgrade Fuel back to the normal version for the newer Pharo (replace version string from 3.a. with the original version) d. Serialize the data Now you can load the data into the newer Pharo. Peter Kenny wrote
could I set those to give different version numbers
The problem is that the data format may not be compatible between the versions. Mariano would be able to give a better answer, but I would be afraid. What if it seems to work but corrupts your data?! Does the workflow I detailed make sense and cover your needs? ----- Cheers, Sean -- View this message in context: http://forum.world.st/How-to-move-data-between-Pharo-versions-tp4829102p4829... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
On Fri, May 29, 2015 at 7:17 AM, Sean P. DeNigris <sean@clipperadams.com> wrote:
Peter Kenny wrote
I will have to convert all my files to the new version? As far as I can see, I could not do that using the device you suggest
The thing is that it's only the serialization that is tied to a Fuel version, so the workflow is (and I've done this to port my data from Pharo 3.0 to Pharo 4.0 and then to Pharo 5.0):
1. In the older image with the data, upgrade Fuel to the latest version that will load in that Pharo `ConfigurationOfFuel project stableVersion load`
This does not cater for "storing object structures long term on disk." I guess the workaround is that the image used to store the objects needs to archive near the fuel files. But you'd need to be careful not to save it once you upgraded fuel versions.
2. Serialize the data If the Fuel version in #1 is the one that comes with the newer Pharo you're porting to, you're done. You can now materialize in the newer Pharo version. Otherwise... 3. In the newer Pharo a. Downgrade Fuel e.g `(ConfigurationOfFuel project version: versionStringFromNumberOneAbove) load` b. Materialize the data c. Upgrade Fuel back to the normal version for the newer Pharo (replace version string from 3.a. with the original version) d. Serialize the data
Some use cases may require multiple downgrades/upgrades, which could be annoying. An interesting idea might be for a new release of Fuel to rename the previous version of as OldFuel that is released in its own package. Then "some OldFuel version" could exist in parallel with the "current version." The "current version" might even be smart enough to identify the OldFuel version needed and load it into the image and continue materializing the data. A CI program might run through a range of OldFuel versions to track which remain working on future versions of Pharo. cheers -ben
Now you can load the data into the newer Pharo.
Peter Kenny wrote
could I set those to give different version numbers
The problem is that the data format may not be compatible between the versions. Mariano would be able to give a better answer, but I would be afraid. What if it seems to work but corrupts your data?!
Does the workflow I detailed make sense and cover your needs?
----- Cheers, Sean -- View this message in context: http://forum.world.st/How-to-move-data-between-Pharo-versions-tp4829102p4829... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
This thread may help you: http://forum.world.st/Fuel-version-td4823170.html Cheers, On Thu, May 28, 2015 at 9:19 PM, Ben Coman <btc@openinworld.com> wrote:
On Fri, May 29, 2015 at 7:17 AM, Sean P. DeNigris <sean@clipperadams.com> wrote:
Peter Kenny wrote
I will have to convert all my files to the new version? As far as I can see, I could not do that using the device you suggest
The thing is that it's only the serialization that is tied to a Fuel version, so the workflow is (and I've done this to port my data from Pharo 3.0 to Pharo 4.0 and then to Pharo 5.0):
1. In the older image with the data, upgrade Fuel to the latest version that will load in that Pharo `ConfigurationOfFuel project stableVersion load`
This does not cater for "storing object structures long term on disk." I guess the workaround is that the image used to store the objects needs to archive near the fuel files. But you'd need to be careful not to save it once you upgraded fuel versions.
2. Serialize the data If the Fuel version in #1 is the one that comes with the newer Pharo you're porting to, you're done. You can now materialize in the newer Pharo version. Otherwise... 3. In the newer Pharo a. Downgrade Fuel e.g `(ConfigurationOfFuel project version: versionStringFromNumberOneAbove) load` b. Materialize the data c. Upgrade Fuel back to the normal version for the newer Pharo (replace version string from 3.a. with the original version) d. Serialize the data
Some use cases may require multiple downgrades/upgrades, which could be annoying. An interesting idea might be for a new release of Fuel to rename the previous version of as OldFuel that is released in its own package. Then "some OldFuel version" could exist in parallel with the "current version." The "current version" might even be smart enough to identify the OldFuel version needed and load it into the image and continue materializing the data. A CI program might run through a range of OldFuel versions to track which remain working on future versions of Pharo. cheers -ben
Now you can load the data into the newer Pharo.
Peter Kenny wrote
could I set those to give different version numbers
The problem is that the data format may not be compatible between the versions. Mariano would be able to give a better answer, but I would be afraid. What if it seems to work but corrupts your data?!
Does the workflow I detailed make sense and cover your needs?
----- Cheers, Sean -- View this message in context:
http://forum.world.st/How-to-move-data-between-Pharo-versions-tp4829102p4829...
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
Hello all Thanks for the comments. I should have made myself clearer in my second post (moral: donât post after midnight!). The original query related to a simple case, which I had solved, but which alerted me to the problem of incompatible versions in Fuel. The next stage of my work (still in planning, hence some vagueness in description) will involve storing objects long term on disk, presumably in serialized form, and I am still considering which serializer to use. I shall be creating large numbers of domain objects â many thousands, possibly running to millions. The instance variables of the objects will be standard Smalltalk objects â strings, arrays, ordered collections, dictionaries â and possibly other domain objects. Each object will be identified by a unique string. In any particular exercise I want to have available in the image only the objects relevant to that exercise. The proposal is to have a database on disk, possibly using PunQLite or Voyage, where the keys are the object identifiers and the database entries are serialized versions of the objects. As references to the domain objects are encountered in the input, the relevant object will be materialized from the database in the image. If I use Fuel to serialize, and if I decide to move to Pharo 5, I may have the problem of converting all the entries in the database to a new version of Fuel. If I use STON, I think there should not be a problem of version changes. I have as yet no idea whether there is any advantage for either serializer in terms of efficiency, either space or time, but at the moment I am inclined to start with STON. If the serializing and materializing are put in a separate section, it should be possible to switch later without too much effort, if it seems desirable. If there is any obvious idiocy in these proposals, please let me know. Otherwise, thanks to all for your helpful comments. Peter Kenny From: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] On Behalf Of Mariano Martinez Peck Sent: 29 May 2015 02:19 To: Any question about pharo is welcome Subject: Re: [Pharo-users] How to move data between Pharo versions This thread may help you: http://forum.world.st/Fuel-version-td4823170.html Cheers, On Thu, May 28, 2015 at 9:19 PM, Ben Coman <btc@openinworld.com <mailto:btc@openinworld.com> > wrote: On Fri, May 29, 2015 at 7:17 AM, Sean P. DeNigris <sean@clipperadams.com <mailto:sean@clipperadams.com> > wrote:
Peter Kenny wrote
I will have to convert all my files to the new version? As far as I can see, I could not do that using the device you suggest
The thing is that it's only the serialization that is tied to a Fuel version, so the workflow is (and I've done this to port my data from Pharo 3.0 to Pharo 4.0 and then to Pharo 5.0):
1. In the older image with the data, upgrade Fuel to the latest version that will load in that Pharo `ConfigurationOfFuel project stableVersion load`
This does not cater for "storing object structures long term on disk." I guess the workaround is that the image used to store the objects needs to archive near the fuel files. But you'd need to be careful not to save it once you upgraded fuel versions.
2. Serialize the data If the Fuel version in #1 is the one that comes with the newer Pharo you're porting to, you're done. You can now materialize in the newer Pharo version. Otherwise... 3. In the newer Pharo a. Downgrade Fuel e.g `(ConfigurationOfFuel project version: versionStringFromNumberOneAbove) load` b. Materialize the data c. Upgrade Fuel back to the normal version for the newer Pharo (replace version string from 3.a. with the original version) d. Serialize the data
Some use cases may require multiple downgrades/upgrades, which could be annoying. An interesting idea might be for a new release of Fuel to rename the previous version of as OldFuel that is released in its own package. Then "some OldFuel version" could exist in parallel with the "current version." The "current version" might even be smart enough to identify the OldFuel version needed and load it into the image and continue materializing the data. A CI program might run through a range of OldFuel versions to track which remain working on future versions of Pharo. cheers -ben
Now you can load the data into the newer Pharo.
Peter Kenny wrote
could I set those to give different version numbers
The problem is that the data format may not be compatible between the versions. Mariano would be able to give a better answer, but I would be afraid. What if it seems to work but corrupts your data?!
Does the workflow I detailed make sense and cover your needs?
----- Cheers, Sean -- View this message in context: http://forum.world.st/How-to-move-data-between-Pharo-versions-tp4829102p4829... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
-- Mariano http://marianopeck.wordpress.com
participants (6)
-
Ben Coman -
Mariano Martinez Peck -
Offray Vladimir Luna Cárdenas -
PBKResearch -
Sean P. DeNigris -
Sven Van Caekenberghe