Re: [Pharo-dev] [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] Desired size of Eden parameter is not respected (#476)
Hi Cyril,
On Feb 20, 2020, at 2:03 AM, CyrilFerlicot <notifications@github.com> wrote:
I want to customize the GC parameters for images that will probably grow to be very large.
One of the most important performance-wise is the size of the Eden. IIUR, the size cannot be changed at runtime so there are two VM parameters. The parameter 44 gives the current size of the Eden while parameter 45 should be used to set the desired size of the Eden.
What I did was that I executed this:
$VM Pharo.image --no-default-preferences eval --save "Smalltalk vm parameterAt: 45 put: 67108864. Smalltalk vm parameterAt: 25 put: 33554432. Smalltalk vm parameterAt: 24 put: 67108864. Smalltalk vm parameterAt: 55 put: 0.7. 'GC tunned'" $VM Pharo.image In the image that opens I then get this:
Smalltalk vm parameterAt: 44. "49546912" Smalltalk vm parameterAt: 45. "67108864" There is quite a large margin between the two.
Alas this discrepancy is historical and semantic. In the old V3 memory manager there was only old space and eden, and one could set the size of eden dynamically because it was simply âthe region at the end of old spaceâ, marked by a base/limit paid of pointers (IIRC; Iâm on my phone so I might have details wrong). In Spur eden is actually one of three regions of new space, the other two being the two survivor spaces, past survivor space and future survivor space (past space and future space for short). Past and future space are the same size and swap after each scavenge, which empties eden and past space, copying survivors into future space, tenuring overflow to old space and then making future space the new past space. The ratio of each survivor space to eden is 1 to 5, i.e. eden is 5/7 of new space and the survivor spaces are 2/7 of new space (currently this ratio cannot be changed). What we really want to do is set the size of new space but the parameter historically referred to v3âs eden. So Spur interprets parameter 45 as setting the desired size of (all of) new space but accurately reports the size of eden for parameter 44. (49546912 / 67108864) â (5/7). So the vm parameter methods should state that vm parameter 45 in v3 sets the desired size of eden and in Spur sets the desired size of new space, while in both, parameter 44 reports the size of eden. And yes, this is a bit of a mess. IIRC there is another parameter that reports the size of new space, but it may report the occupancy of new space. HTH _,,,^..^,,,_ (phone)
I got the VM via: https://files.pharo.org/get-files/pharo64-mac-stable.zip
Image ----- /Users/cyrilferlicot/Downloads/Pharo/Pharo.image Pharo8.0.0 Build information: Pharo-8.0.0+build.1128.sha.9f6475d88dda7d83acdeeda794df35d304cf620d (64 Bit) Unnamed
Virtual Machine --------------- /Users/cyrilferlicot/Downloads/Pharo/pharo-mac-vm/Pharo.app/Contents/MacOS/Pharo CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb 7 2019 StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb 7 2019 VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346 Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git
Mac OS X built on Feb 7 2019 00:01:47 UTC Compiler: 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31) VMMaker versionString VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346 Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb 7 2019 StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb 7 2019
Virtual Machine Commandline Options ----------------------------------- none
Virtual Machine Parameters -------------------------- #1 1297188640 end (v3)/size(Spur) of old-space (0-based, read-only) #2 8941328 end (v3)/size(Spur) of young/new-space (read-only) #3 1365909504 end (v3)/size(Spur) of heap (read-only) #4 nil nil (was allocationCount (read-only)) #5 nil nil (was allocations between GCs (read-write) #6 0 survivor count tenuring threshold (read-write) #7 4 full GCs since startup (read-only) #8 9937 total milliseconds in full GCs since startup (read-only) #9 71 incremental GCs (SqueakV3) or scavenges (Spur) since startup (read-only) #10 77 total milliseconds in incremental GCs (SqueakV3) or scavenges (Spur) since startup (read-only) #11 0 tenures of surving objects since startup (read-only) #12 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #13 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #14 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #15 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #16 1053071685 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #17 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #18 3361 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #19 47160742 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #20 3759644577117121 utc microseconds at VM start-up (actually at time initialization, which precedes image load). #21 348 root table size (read-only) #22 0 root table overflows since startup (read-only) #23 0 bytes of extra memory to reserve for VM buffers, plugins, etc (stored in image file header). #24 33554432 memory threshold above which shrinking object memory (rw) #25 16777216 memory headroom when growing object memory (rw) #26 2 interruptChecksEveryNms - force an ioProcessEvents every N milliseconds (rw) 27 number of times mark loop iterated for current IGC/FGC (read-only) includes ALL marking #28 0 number of times sweep loop iterated for current IGC/FGC (read-only) #29 0 number of times make forward loop iterated for current IGC/FGC (read-only) 30 number of times compact move loop iterated for current IGC/FGC (read-only) #31 2 number of grow memory requests (read-only) #32 0 number of shrink memory requests (read-only) #33 343 number of root table entries used for current IGC/FGC (read-only) #34 3420031008 number of allocations done before current IGC/FGC (read-only) #35 39534 number of survivor objects after current IGC/FGC (read-only) #36 475582454 millisecond clock when current IGC/FGC completed (read-only) #37 0 number of marked objects for Roots of the world, not including Root Table entries for current IGC/FGC (read-only) #38 0 milliseconds taken by current IGC (read-only) #39 0 Number of finalization signals for Weak Objects pending when current IGC/FGC completed (read-only) #40 8 BytesPerOop for this image #41 68021 imageFormatVersion for the VM #42 50 number of stack pages in use #43 0 desired number of stack pages (stored in image file header, max 65535) #44 49546912 size of eden, in bytes #45 67108864 desired size of eden, in bytes (stored in image file header) #46 1433600 machine code zone size, in bytes (Cog only; otherwise nil) #47 0 desired machine code zone size (stored in image file header; Cog only; otherwise nil) #48 0 various header flags. See getCogVMFlags. #49 256 max size the image promises to grow the external semaphore table to (0 sets to default, which is 256 as of writing) #50 nil nil; reserved for VM parameters that persist in the image (such as eden above) #51 nil nil; reserved for VM parameters that persist in the image (such as eden above) #52 524288 root table capacity #53 3 number of segments (Spur only; otherwise nil) #54 21608328 total size of free old space (Spur only, otherwise nil) #55 0.33333298563957214 ratio of growth and image size at or above which a GC will be performed post scavenge #56 266089 number of process switches since startup (read-only) #57 50406 number of ioProcessEvents calls since startup (read-only) #58 518425 number of ForceInterruptCheck calls since startup (read-only) #59 86052 number of check event calls since startup (read-only) #60 504221 number of stack page overflows since startup (read-only) #61 0 number of stack page divorces since startup (read-only) 62 compiled code compactions since startup (read-only; Cog only; otherwise nil) #63 26 total milliseconds in compiled code compactions since startup (read-only; Cog only; otherwise nil) #64 3035 the number of methods that currently have jitted machine-code #65 3 whether the VM supports a certain feature, MULTIPLE_BYTECODE_SETS is bit 0, IMMTABILITY is bit 1 #66 4096 the byte size of a stack page #67 0 the max allowed size of old space (Spur only; nil otherwise; 0 implies no limit except that of the underlying platform) #68 15.455696202531646 the average number of live stack pages when scanned by GC (at scavenge/gc/become et al) #69 21 the maximum number of live stack pages when scanned by GC (at scavenge/gc/become et al) #70 1 the vmProxyMajorVersion (the interpreterProxy VM_MAJOR_VERSION) #71 15 the vmProxyMinorVersion (the interpreterProxy VM_MINOR_VERSION)
Operating System/Hardware ------------------------- Mac OS 1014.6 x64 â You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.
Having control over eden and survivor spaces independently would be nice. I have in the real world in another smalltalk that uses Ungar's memory model solved a nasty bit of earlier tenuring by tweaking the survivor space size 'just right'. .... John M. McIntosh. Corporate Smalltalk Consulting Ltd https://www.linkedin.com/in/smalltalk âââââââ Original Message âââââââ On Thursday, February 20, 2020 3:07 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Cyril,
On Feb 20, 2020, at 2:03 AM, CyrilFerlicot <notifications@github.com> wrote:
I want to customize the GC parameters for images that will probably grow to be very large.
One of the most important performance-wise is the size of the Eden. IIUR, the size cannot be changed at runtime so there are two VM parameters. The parameter 44 gives the current size of the Eden while parameter 45 should be used to set the desired size of the Eden.
What I did was that I executed this:
$
VM
Pharo
.image
--
no
-
default
-
preferences eval
--
save
"
Smalltalk vm parameterAt: 45 put: 67108864. Smalltalk vm parameterAt: 25 put: 33554432. Smalltalk vm parameterAt: 24 put: 67108864. Smalltalk vm parameterAt: 55 put: 0.7. 'GC tunned'
"
$
VM
Pharo
.image
In the image that opens I then get this:
Smalltalk vm parameterAt: 44. "49546912" Smalltalk vm parameterAt: 45. "67108864"
There is quite a large margin between the two.
Alas this discrepancy is historical and semantic. In the old V3 memory manager there was only old space and eden, and one could set the size of eden dynamically because it was simply âthe region at the end of old spaceâ, marked by a base/limit paid of pointers (IIRC; Iâm on my phone so I might have details wrong).
In Spur eden is actually one of three regions of new space, the other two being the two survivor spaces, past survivor space and future survivor space (past space and future space for short). Past and future space are the same size and swap after each scavenge, which empties eden and past space, copying survivors into future space, tenuring overflow to old space and then making future space the new past space. The ratio of each survivor space to eden is 1 to 5, i.e. eden is 5/7 of new space and the survivor spaces are 2/7 of new space (currently this ratio cannot be changed).
What we really want to do is set the size of new space but the parameter historically referred to v3âs eden. So Spur interprets parameter 45 as setting the desired size of (all of) new space but accurately reports the size of eden for parameter 44. (49546912 / 67108864) â (5/7).
So the vm parameter methods should state that vm parameter 45 in v3 sets the desired size of eden and in Spur sets the desired size of new space, while in both, parameter 44 reports the size of eden. And yes, this is a bit of a mess. IIRC there is another parameter that reports the size of new space, but it may report the occupancy of new space.
HTH
_,,,^..^,,,_ (phone)
I got the VM via: https://files.pharo.org/get-files/pharo64-mac-stable.zip
Image ----- /Users/cyrilferlicot/Downloads/Pharo/Pharo.image Pharo8.0.0 Build information: Pharo-8.0.0+build.1128.sha.9f6475d88dda7d83acdeeda794df35d304cf620d (64 Bit) Unnamed
Virtual Machine --------------- /Users/cyrilferlicot/Downloads/Pharo/pharo-mac-vm/Pharo.app/Contents/MacOS/Pharo CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb 7 2019 StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb 7 2019 VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346 Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git
Mac OS X built on Feb 7 2019 00:01:47 UTC Compiler: 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31) VMMaker versionString VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346 Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb 7 2019 StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb 7 2019
Virtual Machine Commandline Options ----------------------------------- none
Virtual Machine Parameters -------------------------- #1 1297188640 end (v3)/size(Spur) of old-space (0-based, read-only) #2 8941328 end (v3)/size(Spur) of young/new-space (read-only) #3 1365909504 end (v3)/size(Spur) of heap (read-only) #4 nil nil (was allocationCount (read-only)) #5 nil nil (was allocations between GCs (read-write) #6 0 survivor count tenuring threshold (read-write) #7 4 full GCs since startup (read-only) #8 9937 total milliseconds in full GCs since startup (read-only) #9 71 incremental GCs (SqueakV3) or scavenges (Spur) since startup (read-only) #10 77 total milliseconds in incremental GCs (SqueakV3) or scavenges (Spur) since startup (read-only) #11 0 tenures of surving objects since startup (read-only) #12 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #13 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #14 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #15 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #16 1053071685 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #17 0 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #18 3361 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #19 47160742 12-20 were specific to ikp's JITTER VM, now 12-19 are open for use #20 3759644577117121 utc microseconds at VM start-up (actually at time initialization, which precedes image load). #21 348 root table size (read-only) #22 0 root table overflows since startup (read-only) #23 0 bytes of extra memory to reserve for VM buffers, plugins, etc (stored in image file header). #24 33554432 memory threshold above which shrinking object memory (rw) #25 16777216 memory headroom when growing object memory (rw) #26 2 interruptChecksEveryNms - force an ioProcessEvents every N milliseconds (rw) 27 number of times mark loop iterated for current IGC/FGC (read-only) includes ALL marking #28 0 number of times sweep loop iterated for current IGC/FGC (read-only) #29 0 number of times make forward loop iterated for current IGC/FGC (read-only) 30 number of times compact move loop iterated for current IGC/FGC (read-only) #31 2 number of grow memory requests (read-only) #32 0 number of shrink memory requests (read-only) #33 343 number of root table entries used for current IGC/FGC (read-only) #34 3420031008 number of allocations done before current IGC/FGC (read-only) #35 39534 number of survivor objects after current IGC/FGC (read-only) #36 475582454 millisecond clock when current IGC/FGC completed (read-only) #37 0 number of marked objects for Roots of the world, not including Root Table entries for current IGC/FGC (read-only) #38 0 milliseconds taken by current IGC (read-only) #39 0 Number of finalization signals for Weak Objects pending when current IGC/FGC completed (read-only) #40 8 BytesPerOop for this image #41 68021 imageFormatVersion for the VM #42 50 number of stack pages in use #43 0 desired number of stack pages (stored in image file header, max 65535) #44 49546912 size of eden, in bytes #45 67108864 desired size of eden, in bytes (stored in image file header) #46 1433600 machine code zone size, in bytes (Cog only; otherwise nil) #47 0 desired machine code zone size (stored in image file header; Cog only; otherwise nil) #48 0 various header flags. See getCogVMFlags. #49 256 max size the image promises to grow the external semaphore table to (0 sets to default, which is 256 as of writing) #50 nil nil; reserved for VM parameters that persist in the image (such as eden above) #51 nil nil; reserved for VM parameters that persist in the image (such as eden above) #52 524288 root table capacity #53 3 number of segments (Spur only; otherwise nil) #54 21608328 total size of free old space (Spur only, otherwise nil) #55 0.33333298563957214 ratio of growth and image size at or above which a GC will be performed post scavenge #56 266089 number of process switches since startup (read-only) #57 50406 number of ioProcessEvents calls since startup (read-only) #58 518425 number of ForceInterruptCheck calls since startup (read-only) #59 86052 number of check event calls since startup (read-only) #60 504221 number of stack page overflows since startup (read-only) #61 0 number of stack page divorces since startup (read-only) 62 compiled code compactions since startup (read-only; Cog only; otherwise nil) #63 26 total milliseconds in compiled code compactions since startup (read-only; Cog only; otherwise nil) #64 3035 the number of methods that currently have jitted machine-code #65 3 whether the VM supports a certain feature, MULTIPLE_BYTECODE_SETS is bit 0, IMMTABILITY is bit 1 #66 4096 the byte size of a stack page #67 0 the max allowed size of old space (Spur only; nil otherwise; 0 implies no limit except that of the underlying platform) #68 15.455696202531646 the average number of live stack pages when scanned by GC (at scavenge/gc/become et al) #69 21 the maximum number of live stack pages when scanned by GC (at scavenge/gc/become et al) #70 1 the vmProxyMajorVersion (the interpreterProxy VM_MAJOR_VERSION) #71 15 the vmProxyMinorVersion (the interpreterProxy VM_MINOR_VERSION)
Operating System/Hardware ------------------------- Mac OS 1014.6 x64
â You are receiving this because you are subscribed to this thread. Reply to this email directly, [view it on GitHub](https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/476?email_source=no...), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AIJPEW2MDHUE5PC3MS7S3CLRDZ...).
participants (2)
-
Eliot Miranda -
John M McIntosh