Hi Eliot, On Sat, Jan 20, 2018 at 09:19:04AM +0100, Alistair Grant wrote:
Hi Eliot,
On 19 January 2018 at 23:04, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Alistair, Hi Cl??ment,
On Fri, Jan 19, 2018 at 12:53 PM, Alistair Grant <akgrant0710@gmail.com> wrote:
Hi Cl??ment,
On 19 January 2018 at 17:21, Alistair Grant <akgrant0710@gmail.com> wrote:
Hi Cl??ment,
On 19 January 2018 at 17:04, Cl??ment Bera <bera.clement@gmail.com> wrote:
Does not seem to be related to prim 105.
I suspect that the problem is the same compactor bug I've been trying to reproduce all week, and have just fixed. Could you try and reproduce with a VM built from the latest commit?
Happy to, but I'm out all day today, so it will be tomorrow or Monday.
Cheers, Alistair (on the run...)
I've tested this with 2 images and 3 VMs in all 6 combinations: - "Old VM": commit date: Wed Jan 10 23:39:30 2018 -0800, gcc 4.8.5 - "New VM": commit date: Sat Jan 20 13:52:26 2018 +0100, gcc 4.8.5 - "GCC 5 VM": commit date: Sat Jan 20 13:52:26 2018 +0100, gcc 5.4.0 - Clean image: commit id: b28d466f - Work image: commit id: eb0a6fb1 The gcc 5 is only there because I was playing with it. The results may be useful, or completely misleading. :-) Each time I ran "5 timesRepeat: [ self test4 ]" with the halts replaced with a count increment. test4 is the method provided in Cyrille's original message. Result summary: - Old VM + Work image: 5, 5, 5, 0, 0 - Old VM + Clean image: 5, 5, 0, 0, 0 - New VM + Work image: 5, 0, 5, 5, 5 - New VM + Clean image: 0, 0, 1, 5, 5 - GCC 5 + Work image: 0, 0, 0, 0, 0 - GCC 5 + Clean image: 0, 0, 0, 0, 0 Old VM: 5.0-201801110739 Thursday 11 January 09:30:12 CET 2018 gcc 4.8.5 [Production Spur VM] CoInterpreter VMMaker.oscog-eem.2302 uuid: 55ec8f63-cdbe-4e79-8f22-48fdea585b88 Jan 11 2018 StackToRegisterMappingCogit VMMaker.oscog-eem.2302 uuid: 55ec8f63-cdbe-4e79-8f22-48fdea585b88 Jan 11 2018 VM: 201801110739 alistair@alistair-xps13:snap/pharo-snap/pharo-vm/opensmalltalk-vm $ Date: Wed Jan 10 23:39:30 2018 -0800 $ Plugins: 201801110739 alistair@alistair-xps13:snap/pharo-snap/pharo-vm/opensmalltalk-vm $ Linux b07d7880072c 4.13.0-26-generic #29~16.04.2-Ubuntu SMP Tue Jan 9 22:00:44 UTC 2018 i686 i686 i686 GNU/Linux plugin path: /snap/pharo7/x1/usr/bin/pharo-vm32/ [default: /snap/pharo7/x1/usr/bin/pharo-vm32/] New VM: 5.0-201801201252 Saturday 20 January 21:24:16 CET 2018 gcc 4.8.5 [Production Spur VM] CoInterpreter VMMaker.oscog-eem.2320 uuid: e2692e35-5fc8-4623-95d0-b445b3329f75 Jan 20 2018 StackToRegisterMappingCogit VMMaker.oscog-eem.2320 uuid: e2692e35-5fc8-4623-95d0-b445b3329f75 Jan 20 2018 VM: 201801201252 alistair@d62ce50f4930:snap/pharo-snap/pharo-vm/opensmalltalk-vm $ Date: Sat Jan 20 13:52:26 2018 +0100 $ Plugins: 201801201252 alistair@d62ce50f4930:snap/pharo-snap/pharo-vm/opensmalltalk-vm $ Linux 73cbbaa49451 4.13.0-26-generic #29~16.04.2-Ubuntu SMP Tue Jan 9 22:00:44 UTC 2018 i686 i686 i686 GNU/Linux plugin path: /snap/pharo7/x1/usr/bin/pharo-vm32/ [default: /snap/pharo7/x1/usr/bin/pharo-vm32/] GCC 5 VM: 5.0-201801201252 Sun Jan 21 14:41:41 UTC 2018 gcc 5.4.0 [Production Spur VM] CoInterpreter VMMaker.oscog-eem.2320 uuid: e2692e35-5fc8-4623-95d0-b445b3329f75 Jan 21 2018 StackToRegisterMappingCogit VMMaker.oscog-eem.2320 uuid: e2692e35-5fc8-4623-95d0-b445b3329f75 Jan 21 2018 VM: 201801201252 alistair@alistair-xps13:squeak/opensmalltalk-vm $ Date: Sat Jan 20 13:52:26 2018 +0100 $ Plugins: 201801201252 alistair@alistair-xps13:squeak/opensmalltalk-vm $ Linux ec9d95d2105a 4.13.0-26-generic #29~16.04.2-Ubuntu SMP Tue Jan 9 22:00:44 UTC 2018 i686 i686 i686 GNU/Linux plugin path: /home/alistair/squeak/opensmalltalk-vm/products/phcogspurlinuxht/lib/pharo/5.0-201801201252 [default: /home/alistair/squeak/opensmalltalk-vm/products/phcogspurlinuxht/lib/pharo/5.0-201801201252/] HTH, Alistair
Some details: The SpurPlanningCompactor works by using the fact that all Spur objects have room for a forwarding pointer. The compactor make three passes:
- the first pass through memory works out where objects will go, replacig their first fields with where they will go, and saving their first fields in a buffer (savedFirstFieldsSpace). - the second pass scans all pointer objects, replacing their fields with where the objects referenced will go (following the forwarding pointers), and also relocates any pointer fields in savedFirstFieldsSpace - the final pass slides objects down, restoring their relocated first fields
The buffer used for savedFirstFieldsSpace determines how many passes are used. The system will either use eden (which is empty when compaction occurs) or a large free chunk or allocate a new segment, depending on whatever yields the largest space. So in the right circumstances eden will be used and more than one pass required.
The bug was that when multiple passes are used the compactor forgot to unmark the corpse left behind when the object was moved. Instead of the corpse being changed into free space it was retained, but its first field would be that of the forwarding pointer to its new location, not the actual first field. So on 32-bits a ByteArray that should have been collected would have its first 4 bytes appear to be invalid, and on 64-bits its first 8 bytes. Because the heap on 64-bits can grow larger it could be that the bug shows itself much less frequently than on 32-bits. When compaction can be completed in a single pass all corpses are correctly collected, so most of the time the bug is hidden.
This is the commit: commit 0fe1e1ea108e53501a0e728736048062c83a66ce Author: Eliot Miranda <eliot.miranda@gmail.com> Date: Fri Jan 19 13:17:57 2018 -0800
CogVM source as per VMMaker.oscog-eem.2320
Spur: Fix a bad bug in SpurPlnningCompactor. unmarkObjectsFromFirstFreeObject, used when the compactor requires more than one pass due to insufficient savedFirstFieldsSpace, expects the corpse of a moved object to be unmarked, but copyAndUnmarkObject:to:bytes:firstField: only unmarked the target. Unmarking the corpse before the copy unmarks both. This fixes a crash with ReleaseBuilder class>>saveAsNewRelease when non-use of cacheDuring: creates lots of files, enough to push the system into the multi-pass regime.