Hi Sean, You can debug the answer of #serialize:, which has information. If you suspect of certain objects you could configure the serializer before serialization with #when:substituteBy: to filter them or something like that. Martin On Fri, Oct 14, 2016 at 1:56 PM, Eliot Miranda <eliot.miranda@gmail.com> wrote:

Hi Sean,

On Fri, Oct 14, 2016 at 9:26 AM, Sean P. DeNigris <sean@clipperadams.com> wrote:

...

I serialized an object graph with lots of Chinese characters in Pharo 4.0. I have been able to rematerialize it back into Pharo 4.0. However, when I try to materialize into a Pharo 5.0 image, I get "PrimitiveFailed: primitive #value: in Character class failed" from `Character class>>#value: 1310876005`. The only difference immediately apparent is the VM (both via Launcher; details below).

Spur allows 30-bit unsigned character codes, so the max character code is 1073741823, or 16r3FFFFFFF. So your character value is out of range. Character class>>#value: reds better commentary. The limit cannot be raised; there are only 30 bits available for immediate character values. Is that character code legal unicode?

More generally, I'm interested in debugging techniques for Fuel. How would

...

I drill down and isolate the problem?

Thanks!

5.0 VM: Virtual Machine --------------- /Applications/Pharo.app/Contents/MacOS/Pharo CoInterpreter VMMaker.oscog-eem.1855 uuid: d8e4a3c2-a3bf-4adc-b224-8012903a1ef4 May 4 2016 StackToRegisterMappingCogit VMMaker.oscog-eem.1855 uuid: d8e4a3c2-a3bf-4adc-b224-8012903a1ef4 May 4 2016 https://github.com/pharo-project/pharo-vm.git Commit: b8ec25a570d7539653e1d793e97609adb509aaed Date: 2016-05-04 11:14:22 +0200 By: Esteban Lorenzano <estebanlm@gmail.com> Jenkins build #589

Mac Cocoa Cog 5.8b12 21-Sep-10 >1B0534FA-246C-47C5-AB29-7A76C81CCDCB< VMMaker versionString https://github.com/pharo-project/pharo-vm.git Commit: b8ec25a570d7539653e1d793e97609adb509aaed Date: 2016-05-04 11:14:22 +0200 By: Esteban Lorenzano <estebanlm@gmail.com> Jenkins build #589 CoInterpreter VMMaker.oscog-eem.1855 uuid: d8e4a3c2-a3bf-4adc-b224-8012903a1ef4 May 4 2016 StackToRegisterMappingCogit VMMaker.oscog-eem.1855 uuid: d8e4a3c2-a3bf-4adc-b224-8012903a1ef4 May 4 2016

VS.

4.0 VM: Virtual Machine --------------- /Applications/Pharo.app/Contents/Resources/vm/40/Pharo.app/ Contents/MacOS/Pharo NBCoInterpreter NativeBoost-CogPlugin-EstebanLorenzano.21 uuid: 4d9b9bdf-2dfa-4c0b-99eb-5b110dadc697 Apr 2 2015 NBCogit NativeBoost-CogPlugin-EstebanLorenzano.21 uuid: 4d9b9bdf-2dfa-4c0b-99eb-5b110dadc697 Apr 2 2015 https://github.com/pharo-project/pharo-vm.git Commit: 32d18ba0f2db9bee7f3bdbf16bdb24fe4801cfc5 Date: 2015-03-24 11:08:14 +0100 By: Esteban Lorenzano <estebanlm@gmail.com> Jenkins build #14904

Mac Cocoa Cog 5.8b12 21-Sep-10 >1B0534FA-246C-47C5-AB29-7A76C81CCDCB< VMMaker versionString https://github.com/pharo-project/pharo-vm.git Commit: 32d18ba0f2db9bee7f3bdbf16bdb24fe4801cfc5 Date: 2015-03-24 11:08:14 +0100 By: Esteban Lorenzano <estebanlm@gmail.com> Jenkins build #14904 NBCoInterpreter NativeBoost-CogPlugin-EstebanLorenzano.21 uuid: 4d9b9bdf-2dfa-4c0b-99eb-5b110dadc697 Apr 2 2015 NBCogit NativeBoost-CogPlugin-EstebanLorenzano.21 uuid: 4d9b9bdf-2dfa-4c0b-99eb-5b110dadc697 Apr 2 2015

----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM- Bug-for-Chinese-Characters-tp4918883.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

-- _,,,^..^,,,_ best, Eliot

AFAIU, Fuel is not capable working across versions, let alone over the non-spur/spur divide. It's a let down, but it is by design (to optimise for absolute speed).

On 15 Oct 2016, at 05:46, Sean P. DeNigris <sean@clipperadams.com> wrote:

Eliot Miranda-2 wrote

...

Is that character code legal unicode?

No, but that turned out to be a red herring!

This simple experiment seems to show a fundamental problem when serializing in 4.0 (non-spur) and materializing in 5.0 (spur):

FileStream forceNewFileNamed: filename do: [:aFile | FLSerializer newDefault serialize: ($e) on: aFile binary ] .

FLMaterializer materializeFromFileNamed: filename.

Doing the serialization and materialization in the same 4.0 image works, but materializing in a 5.0 image throws an error (a different one this time - "MessageNotUnderstood: receiver of ""adaptToNumber:andSend:"" is nil")

----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM-Bug-for-Chinese-Characters-tp4918883p4918909.h... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Sean, fuel relies on the fact that the environment for materialization is the same as where it has been serialized. So classes and methods need to be mostly the same. A non-spur to spur change is likely to make it fail. These changes are rare but sonetimes the need to be done. If you want to move data between different environments you are better off using STON. Norbert

Am 15.10.2016 um 23:28 schrieb Sean P. DeNigris <sean@clipperadams.com>:

Sven Van Caekenberghe-2 wrote

...

AFAIU, Fuel is not capable working across versions, let alone over the non-spur/spur divide. It's a let down, but it is by design (to optimise for absolute speed).

Lack of portability between Fuel versions is an annoyance, but is easy to overcome by updating Fuel on the source image to the same version as the target image before porting. But I don't see an obvious workaround for the non-spur -> spur breakdown. How can we leave users in a situation where it seems impossible to migrate data between Pharo versions?!

----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM-Bug-for-Chinese-Characters-tp4918883p4918943.h... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

On 16 Oct 2016, at 23:02, Sean P. DeNigris <sean@clipperadams.com> wrote:

NorbertHartl wrote

...

These changes are rare

Thankfully :)

NorbertHartl wrote

...

you are better off using STON

But I thought STON does not preserve references. That wouldn't work for all but the simplest models, no?

Of course it does, it wouldn't be very good if it didn't ;-) See #testReferenceSharing and #testReferenceCycle for example

----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM-Bug-for-Chinese-Characters-tp4918883p4919011.h... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

I am in the other boat for reading OrderPreservingDictionary. Phil On Mon, Oct 17, 2016 at 3:43 AM, Sean P. DeNigris <sean@clipperadams.com> wrote:

Sven Van Caekenberghe-2 wrote

...

Of course it does, it wouldn't be very good if it didn't ;-)

Great!

I fell down on "STONWriterError: custom #stonOn: implementation needed for variable/indexable class" for AbsolutePath. Is there an example of how to handle this? I read the paper and googled, but didn't see one...

----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM- Bug-for-Chinese-Characters-tp4918883p4919031.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

On 17 Oct 2016, at 09:17, Sven Van Caekenberghe <sven@stfx.eu> wrote:

...

On 17 Oct 2016, at 07:36, phil@highoctane.be wrote:

I am in the other boat for reading OrderPreservingDictionary.

Both OrderedDictionary and OrderedIdentityDictionary are handled well, see #testOrderedDictionary.

I believe OrderPreservingDictionary is not a standard Pharo part, it is part of PharoExtras. But it is in my main image too.

I think OrderedDictionary is just a copy of OrderPreservingDictionary, then improved. The idea was that we did not want to break peoples code that relied on OrderPreservingDictionary and thus used another (and better) name. Marcus

Sven Van Caekenberghe-2 wrote

Hmm, Path (the superclass) is handled, it has custom #stonOn: and #fromSton: implementations. Do you have them in your image ?

They are not included in the stable version from either the Pharo 4.0 MetaC Browser, or Pharo 5.0 Catalog browser, but I copy pasted them from the latest STON package. Thanks. ----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM-Bug-for-Chinese-Characters-tp4918883p4919099.h... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

On 17 Oct 2016, at 21:14, Sean P. DeNigris <sean@clipperadams.com> wrote:

Sean P. DeNigris wrote

...

I copy pasted them from the latest STON package. Thanks.

Ugh.

FileLocator home / 'Downloads' / 'myfile.ston' writeStreamDo: [ :s | STON put: myDB onStream: s].

and then

FileLocator home / 'Downloads' / 'myfile.ston' readStreamDo: [ :s | STON fromStream: s ]

resulted in: "STONReaderError: At character 978274: unexpected property name type"

Any thoughts on how to debug?

That obviously depends on what 'myDB' contains, the basic call is OK though: FileLocator home / 'Downloads' / 'myfile.ston' writeStreamDo: [ :s | STON put: STONTestUser dummy onStream: s ]. FileLocator home / 'Downloads' / 'myfile.ston' readStreamDo: [ :s | STON fromStream: s ]. The error you get basically means that something non-simple was used as property name. You could try doing: FileLocator home / 'Downloads' / 'myfile.ston' readStreamDo: [ :s | (STON reader on: s) allowComplexMapKeys: true; next ].

From the class comment of STONReader

- allowComplexMapKeys <Boolean> default is false if true, any object is allowed as map key if false, only strings, symbols and numbers are allowed as map keys

----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM-Bug-for-Chinese-Characters-tp4918883p4919109.h... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

On 18 Oct 2016, at 03:36, Sean P. DeNigris <sean@clipperadams.com> wrote:

Sven Van Caekenberghe-2 wrote

...

You could try doing... allowComplexMapKeys: true ...

Hmm, if the default writing may use complex keys, shouldn't the default reading do the same? I found it jarring to write using the defaults, and then have the default read fail!

Yes, you are right, the default should be true, I can't remember why I did not yet do it, it feels as if I am forgetting something, don't see all the consequences.

----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM-Bug-for-Chinese-Characters-tp4918883p4919141.h... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Sven Van Caekenberghe-2 wrote

I decided to... simplify implementation

Thanks for following up! ----- Cheers, Sean -- View this message in context: http://forum.world.st/Spur-VM-Bug-for-Chinese-Characters-tp4918883p4919990.h... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.