About Image Size Pharo9->Pharo11
Pharo9: 65,8 MB
Pharo10: 57,4 MB
Pharo11: 56,5 MB
The space saving come from three things:
- Code cleanup Pharo9-Pharo10
=========================
e.g. removal of support in Kernel and Compiler of old style blocks and bytecodes,
to mention one smaller (in term of image size) but deep cleanup.
Of course, we can do more here. There is still a lot of dead code (and duplicated
things) in the image!
- Global Literal Sharing
===================
Literals are now, when the method is compiled, set to be read-only, recursively in the case of arrays.
This allows us to unify (share one instance) not only in the same method, but globally over all methods.
As the compiler can not do a global literal table when compiling a single method (too slow), this is implemented
as an additional pass, called as part of the build process.
If you load a lot of code, it might be an idea to do that as part of your build again:
ImageCleaner shareLiterals
- Improvement of code structure objects
===============================
e.g, in Pharo10 we found two empty OrderedCollections per class that where not needed.
Further work on this idea is where the improvement in Pharo11 comes from, we merged these three changes:
-
Class>>#classPool: empty dictionary for all classes, but not many define class vars #11172
https://github.com/pharo-project/pharo/issues/11172 -
[Memory] sharedPools OrderedCollections waste memory #10957
https://github.com/pharo-project/pharo/issues/10957 -
[Memory] AllProtocol uses a lot of memory #10963
https://github.com/pharo-project/pharo/issues/10963
The classPools/sharedPool improves the size of the bootstrapped image not to0 much as the boostrap creates
classes with nil there, but for newly loaded code this will be more visible.
This direction has some more fairly simple next steps:
-
[Memory] commentSourcePointer is not needed for MetaClasses
https://github.com/pharo-project/pharo/issues/10958 -
[Memory] Every class and meta class has both a ProtocolOrganizer and a ClassOrganization #10959
https://github.com/pharo-project/pharo/issues/10959
... and of course there are many more things to find with just looking a bit.
(And it is interesting starting point to think about better language support for making long living but rarely changed
data structures more memory efficient)
If you want to get an impression of which classes use image space, "SpaceTally printSpaceAnalysis"
prints a per-class analysis.
Marcus
Great work, Marcus.
On 11 May 2022, at 09:00, Marcus Denker marcus.denker@inria.fr wrote:
Pharo9: 65,8 MB
Pharo10: 57,4 MB
Pharo11: 56,5 MB
The space saving come from three things:
- Code cleanup Pharo9-Pharo10
=========================
e.g. removal of support in Kernel and Compiler of old style blocks and bytecodes,
to mention one smaller (in term of image size) but deep cleanup.
Of course, we can do more here. There is still a lot of dead code (and duplicated
things) in the image!
- Global Literal Sharing
===================
Literals are now, when the method is compiled, set to be read-only, recursively in the case of arrays.
This allows us to unify (share one instance) not only in the same method, but globally over all methods.
As the compiler can not do a global literal table when compiling a single method (too slow), this is implemented
as an additional pass, called as part of the build process.
If you load a lot of code, it might be an idea to do that as part of your build again:
ImageCleaner shareLiterals
- Improvement of code structure objects
===============================
e.g, in Pharo10 we found two empty OrderedCollections per class that where not needed.
Further work on this idea is where the improvement in Pharo11 comes from, we merged these three changes:
-
Class>>#classPool: empty dictionary for all classes, but not many define class vars #11172
https://github.com/pharo-project/pharo/issues/11172 -
[Memory] sharedPools OrderedCollections waste memory #10957
https://github.com/pharo-project/pharo/issues/10957 -
[Memory] AllProtocol uses a lot of memory #10963
https://github.com/pharo-project/pharo/issues/10963
The classPools/sharedPool improves the size of the bootstrapped image not to0 much as the boostrap creates
classes with nil there, but for newly loaded code this will be more visible.
This direction has some more fairly simple next steps:
-
[Memory] commentSourcePointer is not needed for MetaClasses
https://github.com/pharo-project/pharo/issues/10958 -
[Memory] Every class and meta class has both a ProtocolOrganizer and a ClassOrganization #10959
https://github.com/pharo-project/pharo/issues/10959
... and of course there are many more things to find with just looking a bit.
(And it is interesting starting point to think about better language support for making long living but rarely changed
data structures more memory efficient)
If you want to get an impression of which classes use image space, "SpaceTally printSpaceAnalysis"
prints a per-class analysis.
Marcus
This is gold. This is the work nobody usually likes to do but is THE apsect for sustainability of a system.
Like it!!!
thanks,
Norbert
Am 11.05.2022 um 09:00 schrieb Marcus Denker marcus.denker@inria.fr:
Pharo9: 65,8 MB
Pharo10: 57,4 MB
Pharo11: 56,5 MB
The space saving come from three things:
- Code cleanup Pharo9-Pharo10
=========================
e.g. removal of support in Kernel and Compiler of old style blocks and bytecodes,
to mention one smaller (in term of image size) but deep cleanup.
Of course, we can do more here. There is still a lot of dead code (and duplicated
things) in the image!
- Global Literal Sharing
===================
Literals are now, when the method is compiled, set to be read-only, recursively in the case of arrays.
This allows us to unify (share one instance) not only in the same method, but globally over all methods.
As the compiler can not do a global literal table when compiling a single method (too slow), this is implemented
as an additional pass, called as part of the build process.
If you load a lot of code, it might be an idea to do that as part of your build again:
ImageCleaner shareLiterals
- Improvement of code structure objects
===============================
e.g, in Pharo10 we found two empty OrderedCollections per class that where not needed.
Further work on this idea is where the improvement in Pharo11 comes from, we merged these three changes:
-
Class>>#classPool: empty dictionary for all classes, but not many define class vars #11172
https://github.com/pharo-project/pharo/issues/11172 -
[Memory] sharedPools OrderedCollections waste memory #10957
https://github.com/pharo-project/pharo/issues/10957 -
[Memory] AllProtocol uses a lot of memory #10963
https://github.com/pharo-project/pharo/issues/10963
The classPools/sharedPool improves the size of the bootstrapped image not to0 much as the boostrap creates
classes with nil there, but for newly loaded code this will be more visible.
This direction has some more fairly simple next steps:
-
[Memory] commentSourcePointer is not needed for MetaClasses
https://github.com/pharo-project/pharo/issues/10958 -
[Memory] Every class and meta class has both a ProtocolOrganizer and a ClassOrganization #10959
https://github.com/pharo-project/pharo/issues/10959
... and of course there are many more things to find with just looking a bit.
(And it is interesting starting point to think about better language support for making long living but rarely changed
data structures more memory efficient)
If you want to get an impression of which classes use image space, "SpaceTally printSpaceAnalysis"
prints a per-class analysis.
Marcus
On 11 May 2022, at 09:00, Marcus Denker marcus.denker@inria.fr wrote:
Pharo9: 65,8 MB
Pharo10: 57,4 MB
Pharo11: 56,5 MB
Another small step:
Pharo11: 55,4 MB
Reset SelectorTable on #cleanUp #11235
https://github.com/pharo-project/pharo/pull/11235
- avoid interning selectors on compile, just do it on install
(which means it will not be done for DoIts and FFI methods)
- This allows to reset it on #cleanUp
- Which is done on the CI as part of the build.
- trivial: simplify #internSelector: a little (no temp needed)
Not perfect, of course, as the SelectorTable is created on the first compile
and then only removed if you call the reset method (e.g. via “Symbol cleanUp”).
But on the plus side, besides a smaller distributed image size, even e.g.
a pre-build worker image should never allocate this and be a bit smaller during execution.
Marcus