STON materialization corrupts a dictionary if the keys are references
Hi, I ran into a problem with STON that seems to corrupt on materialization. Consider the following script: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ c := OrderedCollection with: UUID new. d := Dictionary with: c first -> nil. o := { c . d }. s := STON toStringPretty: o. o2 := STON reader allowComplexMapKeys: true; on: s readStream; next. o2 second keys. "an Array(an UUID('571bbfb7-78b9-430c-9aad-3e0088a25f4c'))" o2 second printString. "KeyNotFound: key an UUID(''571bbfb7-78b9-430c-9aad-3e0088a25f4c'') not found in Dictionary" o2 second at: o2 second keys first. "the same error" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ So even though it contains the key it fails to access it. Interestingly enough if I serialize and materialize the "broken" dictionary again it suddenly works again. Thanks, Peter
Hi Peter, Strange indeed. It did not fail for me in 5.0 with the standard STON version, but it does fail in 4.0 with the latest STON. Asking the bogus dictionary to #rehash seems to fix the problem. I will investigate and turn you example into a unit test. Sven
On 05 Apr 2016, at 11:14, Peter Uhnák <i.uhnak@gmail.com> wrote:
Hi,
I ran into a problem with STON that seems to corrupt on materialization.
Consider the following script:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ c := OrderedCollection with: UUID new. d := Dictionary with: c first -> nil. o := { c . d }.
s := STON toStringPretty: o.
o2 := STON reader allowComplexMapKeys: true; on: s readStream; next.
o2 second keys. "an Array(an UUID('571bbfb7-78b9-430c-9aad-3e0088a25f4c'))" o2 second printString. "KeyNotFound: key an UUID(''571bbfb7-78b9-430c-9aad-3e0088a25f4c'') not found in Dictionary" o2 second at: o2 second keys first. "the same error" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
So even though it contains the key it fails to access it.
Interestingly enough if I serialize and materialize the "broken" dictionary again it suddenly works again.
Thanks, Peter
This is Pharo #50666 Moose 6.0 build #1437 STON-Core-SvenVanCaekenberghe.67 (I also loaded STON-Core-SvenVanCaekenberghe.70 but the result is the same). One thing to note: it doesn't fail always⦠for me it seems that about every ten or so attempts it succeeds. Peter On Tue, Apr 5, 2016 at 11:41 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi Peter,
Strange indeed. It did not fail for me in 5.0 with the standard STON version, but it does fail in 4.0 with the latest STON. Asking the bogus dictionary to #rehash seems to fix the problem. I will investigate and turn you example into a unit test.
Sven
On 05 Apr 2016, at 11:14, Peter Uhnák <i.uhnak@gmail.com> wrote:
Hi,
I ran into a problem with STON that seems to corrupt on materialization.
Consider the following script:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ c := OrderedCollection with: UUID new. d := Dictionary with: c first -> nil. o := { c . d }.
s := STON toStringPretty: o.
o2 := STON reader allowComplexMapKeys: true; on: s readStream; next.
o2 second keys. "an Array(an UUID('571bbfb7-78b9-430c-9aad-3e0088a25f4c'))" o2 second printString. "KeyNotFound: key an UUID(''571bbfb7-78b9-430c-9aad-3e0088a25f4c'') not found in Dictionary" o2 second at: o2 second keys first. "the same error" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
So even though it contains the key it fails to access it.
Interestingly enough if I serialize and materialize the "broken" dictionary again it suddenly works again.
Thanks, Peter
On 05 Apr 2016, at 11:50, Peter Uhnák <i.uhnak@gmail.com> wrote:
This is Pharo #50666 Moose 6.0 build #1437 STON-Core-SvenVanCaekenberghe.67
(I also loaded STON-Core-SvenVanCaekenberghe.70 but the result is the same).
One thing to note: it doesn't fail always⦠for me it seems that about every ten or so attempts it succeeds.
Like I said, it is a hashing issue, sometimes it will be correct by accident. I hope you did not have to much trouble with this bug, I guess it must have been hard to chase. Is it urgent ? I probably can give you a quick fix, but I would like to think a bit more about this, since rehashing each materialised dictionary seems expensive.
Peter
On Tue, Apr 5, 2016 at 11:41 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote: Hi Peter,
Strange indeed. It did not fail for me in 5.0 with the standard STON version, but it does fail in 4.0 with the latest STON. Asking the bogus dictionary to #rehash seems to fix the problem. I will investigate and turn you example into a unit test.
Sven
On 05 Apr 2016, at 11:14, Peter Uhnák <i.uhnak@gmail.com> wrote:
Hi,
I ran into a problem with STON that seems to corrupt on materialization.
Consider the following script:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ c := OrderedCollection with: UUID new. d := Dictionary with: c first -> nil. o := { c . d }.
s := STON toStringPretty: o.
o2 := STON reader allowComplexMapKeys: true; on: s readStream; next.
o2 second keys. "an Array(an UUID('571bbfb7-78b9-430c-9aad-3e0088a25f4c'))" o2 second printString. "KeyNotFound: key an UUID(''571bbfb7-78b9-430c-9aad-3e0088a25f4c'') not found in Dictionary" o2 second at: o2 second keys first. "the same error" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
So even though it contains the key it fails to access it.
Interestingly enough if I serialize and materialize the "broken" dictionary again it suddenly works again.
Thanks, Peter
Like I said, it is a hashing issue, sometimes it will be correct by accident.
I hope you did not have to much trouble with this bug, I guess it must have been hard to chase.
The hardest thing often is to suspect the system or external libraries, but it wasn't that bad. Is it urgent ?
I can rehash it for now (I have a trivial case), but it should be fixed eventually (it wouldn't be so easy to rehash more complex structures). Thanks, Peter
participants (2)
-
Peter Uhnák -
Sven Van Caekenberghe