Large images reasons [WAS] Re: [Pharo-users] Pharo 2.0 with Seaside + DBXTalk + GlorpDBX + Magritte 3 + TWBS is getting slower and slower
OK, this was was my experiment.... Image fresh with all my app and dependencies loades: 30MB After using it for some days/weeks: 160MB. SpaceTally new printSpaceAnalysis showed: Class code space # instances inst space percent inst average size ByteString 2785 413144 116244078 69.90 281.36 Array 3712 181772 8466668 5.10 46.58 ByteArray 8574 1319 8186802 4.90 6206.82 Bitmap 3653 303 6656340 4.00 21968.12 CompiledMethod 22467 90554 5685374 3.40 62.78 After executing ImageCleaner cleanUpForRelease: 36MB Then...I searched which part of #cleanUpForRelease: was making the difference, and it was: Smalltalk cleanUp: true except: #() confirming: false. So now it was time to know WHICH class did the diference, so I modified #cleanUp: aggressive except: exclusions confirming: aBool in these lines: "Run the cleanup code" classes do:[:aClass| Transcript show: 'Image size before cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. aClass cleanUp: aggressive. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. Transcript show: 'Image size after cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. ] displayingProgress: [:aClass| 'Cleaning up in ', aClass name]. I then opened a Transcript, and evaluated Smalltalk cleanUp: true except: #() confirming: false. I went to prepare Mate, and when I come back, the result was, of course: "Image size after cleaning MCFileBasedRepository : 39744008" That clean up ends up doing: flushAllCaches self allSubInstancesDo: [:ea | ea flushCache] So it sends #flushCache to all instances of MCHttpRepository and MCFileBasedRepository. Now what I wanted to see if it there was a particular repo that could take most of the space (like package-cache). And indeed, it was...I modified #flushCaches to: flushAllCaches | file | file := 'repos.txt' asFileReference writeStream text. self allSubInstancesDo: [:each | file nextPutAll: 'Image size before cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString; cr. each flushCache. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. file nextPutAll: 'Image size after cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString;cr. ]. file flush; close. And then I looked in the 'repos.txt' file. My package cache repo cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB. So...cleaning cache of just 3 repos frees approx 100MB. The question is....can we flush the cache safely? If they are called "cache", then I guess yes, we can. Thoughts? Thanks, On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
Guys, I have images also with seaside, magritte, glorp, postgresV2, etc and it is also around 200MB. I will try to do some research today and let you know.
Cheers,
On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Jul 30, 2013, at 1:49 PM, "phil@highoctane.be" <phil@highoctane.be> wrote:
the changes file contained passwords and I replaced the text. So offsets may be wrong due to that.
Yes, the first thing I wanted to do is to recompile everything. Does not work.
Memorymonitor is not doing fanct stuff. It just counts instances.
Yes, but maybe it holds on to these instances?
Marcus
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
I'll give a shot at trying that on my 300 megs thing! Learning ahead! On Wednesday, July 31, 2013, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
OK, this was was my experiment....
Image fresh with all my app and dependencies loades: 30MB After using it for some days/weeks: 160MB. SpaceTally new printSpaceAnalysis showed:
Class code space # instances inst space percent inst average size ByteString 2785 413144 116244078 69.90 281.36 Array 3712 181772 8466668 5.10 46.58 ByteArray 8574 1319 8186802 4.90 6206.82 Bitmap 3653 303 6656340 4.00 21968.12 CompiledMethod 22467 90554 5685374 3.40 62.78
After executing ImageCleaner cleanUpForRelease: 36MB Then...I searched which part of #cleanUpForRelease: was making the difference, and it was: Smalltalk cleanUp: true except: #() confirming: false. So now it was time to know WHICH class did the diference, so I modified #cleanUp: aggressive except: exclusions confirming: aBool in these lines: "Run the cleanup code" classes do:[:aClass| Transcript show: 'Image size before cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. aClass cleanUp: aggressive. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. Transcript show: 'Image size after cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. ] displayingProgress: [:aClass| 'Cleaning up in ', aClass name]. I then opened a Transcript, and evaluated Smalltalk cleanUp: true except: #() confirming: false. I went to prepare Mate, and when I come back, the result was, of course: "Image size after cleaning MCFileBasedRepository : 39744008" That clean up ends up doing: flushAllCaches self allSubInstancesDo: [:ea | ea flushCache] So it sends #flushCache to all instances of MCHttpRepository and MCFileBasedRepository. Now what I wanted to see if it there was a particular repo that could take most of the space (like package-cache). And indeed, it was...I modified #flushCaches to: flushAllCaches | file | file := 'repos.txt' asFileReference writeStream text. self allSubInstancesDo: [:each | file nextPutAll: 'Image size before cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString; cr. each flushCache. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. file nextPutAll: 'Image size after cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString;cr. ]. file flush; close. And then I looked in the 'repos.txt' file. My package cache repo cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB. So...cleaning cache of just 3 repos frees approx 100MB. The question is....can we flush the cache safely? If they are called "cache", then I guess yes, we can. Thoughts? Thanks,
On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
Guys, I have images also with seaside, magritte, glorp, postgresV2, etc
and it is also around 200MB.
I will try to do some research today and let you know. Cheers,
On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On Jul 30, 2013, at 1:49 PM, "phil@highoctane.be" <phil@highoctane.be>
wrote:
the changes file contained passwords and I replaced the text. So
offsets may be wrong due to that.
Yes, the first thing I wanted to do is to recompile everything. Does not work.
Memorymonitor is not doing fanct stuff. It just counts instances.
Yes, but maybe it holds on to these instances?
Marcus
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- --- Philippe Back Dramatic Performance Improvements Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 Mail:phil@highoctane.be | Web: http://philippeback.eu Blog: http://philippeback.be | Twitter: @philippeback Youtube: http://www.youtube.com/user/philippeback/videos High Octane SPRL rue cour Boisacq 101 | 1301 Bierges | Belgium Featured on the Software Process and Measurement Cast http://spamcast.libsyn.com Sparx Systems Enterprise Architect and Ability Engineering EADocX Value Added Reseller
Well, here is what I do get: (I guess some stuff scrolled up too far in the transcript) Image size before cleaning HashedCollection : 320691776----SNAPSHOT----an Array(1 August 2013 12:21:29 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119932535 Image size after cleaning HashedCollection : 323444020Image size before cleaning RecentMessageList : 323444020----SNAPSHOT----an Array(1 August 2013 12:21:46 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119936948 Image size after cleaning RecentMessageList : 323492924Image size before cleaning ChangeSet : 323492924----SNAPSHOT----an Array(1 August 2013 12:22:01 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119937094 Image size after cleaning ChangeSet : 323557656Image size before cleaning SystemOrganizer : 323557656----SNAPSHOT----an Array(1 August 2013 12:22:16 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119937240 Image size after cleaning SystemOrganizer : 323621024Image size before cleaning MCDefinition : 323621024----SNAPSHOT----an Array(1 August 2013 12:22:31 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119937386 Image size after cleaning MCDefinition : 323685312Image size before cleaning Behavior : 323685312----SNAPSHOT----an Array(1 August 2013 12:22:46 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119937532 Image size after cleaning Behavior : 323749188Image size before cleaning Workspace : 323749188----SNAPSHOT----an Array(1 August 2013 12:23:01 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119937678 Image size after cleaning Workspace : 323797016Image size before cleaning AppRegistry : 323797016----SNAPSHOT----an Array(1 August 2013 12:23:16 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119937824 Image size after cleaning AppRegistry : 323860980Image size before cleaning MCFileBasedRepository : 323860980----SNAPSHOT----an Array(1 August 2013 12:23:34 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119937970 Image size after cleaning MCFileBasedRepository : 323924988Image size before cleaning ClassOrganizer : 323924988----SNAPSHOT----an Array(1 August 2013 12:23:51 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119938116 Image size after cleaning ClassOrganizer : 323989128Image size before cleaning FreeTypeCache : 323989128----SNAPSHOT----an Array(1 August 2013 12:24:06 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119938262 Image size after cleaning FreeTypeCache : 324053128Image size before cleaning NOCCompletionTable : 324053128----SNAPSHOT----an Array(1 August 2013 12:24:21 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119938408 Image size after cleaning NOCCompletionTable : 324117108Image size before cleaning NaturalLanguageTranslator : 324117108----SNAPSHOT----an Array(1 August 2013 12:24:36 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119938554 Image size after cleaning NaturalLanguageTranslator : 324181356Image size before cleaning ScrollBar : 324181356----SNAPSHOT----an Array(1 August 2013 12:24:52 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119938700 Image size after cleaning ScrollBar : 324241728Image size before cleaning FileServices : 324241728----SNAPSHOT----an Array(1 August 2013 12:25:07 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119938846 Image size after cleaning FileServices : 324311268Image size before cleaning EventManager : 324311268----SNAPSHOT----an Array(1 August 2013 12:25:23 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119938992 Image size after cleaning EventManager : 324375192Image size before cleaning PasteUpMorph : 324375192----SNAPSHOT----an Array(1 August 2013 12:25:38 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119939138 Image size after cleaning PasteUpMorph : 324439164Image size before cleaning BitBlt : 324439164----SNAPSHOT----an Array(1 August 2013 12:25:54 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119939284 Image size after cleaning BitBlt : 324156860Image size before cleaning GradientFillStyle : 324156860----SNAPSHOT----an Array(1 August 2013 12:26:11 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119939430 Image size after cleaning GradientFillStyle : 324389584Image size before cleaning NaturalLanguageFormTranslator : 324389584----SNAPSHOT----an Array(1 August 2013 12:26:26 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119939576 Image size after cleaning NaturalLanguageFormTranslator : 324460080Image size before cleaning ProcessBrowser : 324460080----SNAPSHOT----an Array(1 August 2013 12:26:40 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119939722 Image size after cleaning ProcessBrowser : 324524292Image size before cleaning StrikeFont : 324524292----SNAPSHOT----an Array(1 August 2013 12:26:56 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119939868 Image size after cleaning StrikeFont : 324588200Image size before cleaning MCDataStream : 324588200----SNAPSHOT----an Array(1 August 2013 12:27:11 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119940014 Image size after cleaning MCDataStream : 324652584Image size before cleaning CompiledMethod : 324652584----SNAPSHOT----an Array(1 August 2013 12:27:27 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119940160 Image size after cleaning CompiledMethod : 324716520Image size before cleaning SmalltalkImage : 324716520----SNAPSHOT----an Array(1 August 2013 12:27:49 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119940306 Image size after cleaning SmalltalkImage : 324764312Image size before cleaning Symbol : 324764312----SNAPSHOT----an Array(1 August 2013 12:28:07 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119940452 Image size after cleaning Symbol : 324787420Image size before cleaning Browser : 324787420----SNAPSHOT----an Array(1 August 2013 12:28:23 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119940598 Image size after cleaning Browser : 324852504Image size before cleaning MCMethodDefinition : 324852504----SNAPSHOT----an Array(1 August 2013 12:28:39 am) Pharo-DripfeedIt-Magritte3-OpenDBX_20130729_1154-Prod_Cleaning.image priorSource: 119940744 Image size after cleaning MCMethodDefinition : 324916888 The image actually grew bigger... Now trying the flush caches thing. Phil On Wed, Jul 31, 2013 at 8:53 PM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
OK, this was was my experiment....
Image fresh with all my app and dependencies loades: 30MB
After using it for some days/weeks: 160MB.
SpaceTally new printSpaceAnalysis showed:
Class code space # instances inst space percent inst average size ByteString 2785 413144 116244078 69.90 281.36 Array 3712 181772 8466668 5.10 46.58 ByteArray 8574 1319 8186802 4.90 6206.82 Bitmap 3653 303 6656340 4.00 21968.12 CompiledMethod 22467 90554 5685374 3.40 62.78
After executing ImageCleaner cleanUpForRelease: 36MB
Then...I searched which part of #cleanUpForRelease: was making the difference, and it was:
Smalltalk cleanUp: true except: #() confirming: false.
So now it was time to know WHICH class did the diference, so I modified #cleanUp: aggressive except: exclusions confirming: aBool
in these lines:
"Run the cleanup code" classes do:[:aClass| Transcript show: 'Image size before cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. aClass cleanUp: aggressive. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. Transcript show: 'Image size after cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. ] displayingProgress: [:aClass| 'Cleaning up in ', aClass name]. I then opened a Transcript, and evaluated
Smalltalk cleanUp: true except: #() confirming: false.
I went to prepare Mate, and when I come back, the result was, of course:
"Image size after cleaning MCFileBasedRepository : 39744008"
That clean up ends up doing:
flushAllCaches self allSubInstancesDo: [:ea | ea flushCache] So it sends #flushCache to all instances of MCHttpRepository and MCFileBasedRepository.
Now what I wanted to see if it there was a particular repo that could take most of the space (like package-cache). And indeed, it was...I modified #flushCaches to:
flushAllCaches | file | file := 'repos.txt' asFileReference writeStream text. self allSubInstancesDo: [:each | file nextPutAll: 'Image size before cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString; cr. each flushCache. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. file nextPutAll: 'Image size after cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString;cr. ]. file flush; close. And then I looked in the 'repos.txt' file. My package cache repo cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB. So...cleaning cache of just 3 repos frees approx 100MB.
The question is....can we flush the cache safely? If they are called "cache", then I guess yes, we can.
Thoughts?
Thanks,
On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
Guys, I have images also with seaside, magritte, glorp, postgresV2, etc and it is also around 200MB. I will try to do some research today and let you know.
Cheers,
On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Jul 30, 2013, at 1:49 PM, "phil@highoctane.be" <phil@highoctane.be> wrote:
the changes file contained passwords and I replaced the text. So offsets may be wrong due to that.
Yes, the first thing I wanted to do is to recompile everything. Does not work.
Memorymonitor is not doing fanct stuff. It just counts instances.
Yes, but maybe it holds on to these instances?
Marcus
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
Doing the flushCaches... Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324017564 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324137232 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324137232 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324195720 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324195720 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324255300 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324255300 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324314868 Definitely not going down... Phil On Wed, Jul 31, 2013 at 8:53 PM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
OK, this was was my experiment....
Image fresh with all my app and dependencies loades: 30MB
After using it for some days/weeks: 160MB.
SpaceTally new printSpaceAnalysis showed:
Class code space # instances inst space percent inst average size ByteString 2785 413144 116244078 69.90 281.36 Array 3712 181772 8466668 5.10 46.58 ByteArray 8574 1319 8186802 4.90 6206.82 Bitmap 3653 303 6656340 4.00 21968.12 CompiledMethod 22467 90554 5685374 3.40 62.78
After executing ImageCleaner cleanUpForRelease: 36MB
Then...I searched which part of #cleanUpForRelease: was making the difference, and it was:
Smalltalk cleanUp: true except: #() confirming: false.
So now it was time to know WHICH class did the diference, so I modified #cleanUp: aggressive except: exclusions confirming: aBool
in these lines:
"Run the cleanup code" classes do:[:aClass| Transcript show: 'Image size before cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. aClass cleanUp: aggressive. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. Transcript show: 'Image size after cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. ] displayingProgress: [:aClass| 'Cleaning up in ', aClass name]. I then opened a Transcript, and evaluated
Smalltalk cleanUp: true except: #() confirming: false.
I went to prepare Mate, and when I come back, the result was, of course:
"Image size after cleaning MCFileBasedRepository : 39744008"
That clean up ends up doing:
flushAllCaches self allSubInstancesDo: [:ea | ea flushCache] So it sends #flushCache to all instances of MCHttpRepository and MCFileBasedRepository.
Now what I wanted to see if it there was a particular repo that could take most of the space (like package-cache). And indeed, it was...I modified #flushCaches to:
flushAllCaches | file | file := 'repos.txt' asFileReference writeStream text. self allSubInstancesDo: [:each | file nextPutAll: 'Image size before cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString; cr. each flushCache. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. file nextPutAll: 'Image size after cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString;cr. ]. file flush; close. And then I looked in the 'repos.txt' file. My package cache repo cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB. So...cleaning cache of just 3 repos frees approx 100MB.
The question is....can we flush the cache safely? If they are called "cache", then I guess yes, we can.
Thoughts?
Thanks,
On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
Guys, I have images also with seaside, magritte, glorp, postgresV2, etc and it is also around 200MB. I will try to do some research today and let you know.
Cheers,
On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Jul 30, 2013, at 1:49 PM, "phil@highoctane.be" <phil@highoctane.be> wrote:
the changes file contained passwords and I replaced the text. So offsets may be wrong due to that.
Yes, the first thing I wanted to do is to recompile everything. Does not work.
Memorymonitor is not doing fanct stuff. It just counts instances.
Yes, but maybe it holds on to these instances?
Marcus
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
Phil, and the cleanUpForRelease in general does not help at all? On Wed, Jul 31, 2013 at 8:30 PM, phil@highoctane.be <phil@highoctane.be>wrote:
Doing the flushCaches...
Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324017564 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324137232 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324137232 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324195720 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324195720 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324255300 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324255300 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324314868
Definitely not going down...
Phil
On Wed, Jul 31, 2013 at 8:53 PM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
OK, this was was my experiment....
Image fresh with all my app and dependencies loades: 30MB
After using it for some days/weeks: 160MB.
SpaceTally new printSpaceAnalysis showed:
Class code space # instances inst space percent inst average size ByteString 2785 413144 116244078 69.90 281.36 Array 3712 181772 8466668 5.10 46.58 ByteArray 8574 1319 8186802 4.90 6206.82 Bitmap 3653 303 6656340 4.00 21968.12 CompiledMethod 22467 90554 5685374 3.40 62.78
After executing ImageCleaner cleanUpForRelease: 36MB
Then...I searched which part of #cleanUpForRelease: was making the difference, and it was:
Smalltalk cleanUp: true except: #() confirming: false.
So now it was time to know WHICH class did the diference, so I modified #cleanUp: aggressive except: exclusions confirming: aBool
in these lines:
"Run the cleanup code" classes do:[:aClass| Transcript show: 'Image size before cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. aClass cleanUp: aggressive. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. Transcript show: 'Image size after cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. ] displayingProgress: [:aClass| 'Cleaning up in ', aClass name]. I then opened a Transcript, and evaluated
Smalltalk cleanUp: true except: #() confirming: false.
I went to prepare Mate, and when I come back, the result was, of course:
"Image size after cleaning MCFileBasedRepository : 39744008"
That clean up ends up doing:
flushAllCaches self allSubInstancesDo: [:ea | ea flushCache] So it sends #flushCache to all instances of MCHttpRepository and MCFileBasedRepository.
Now what I wanted to see if it there was a particular repo that could take most of the space (like package-cache). And indeed, it was...I modified #flushCaches to:
flushAllCaches | file | file := 'repos.txt' asFileReference writeStream text. self allSubInstancesDo: [:each | file nextPutAll: 'Image size before cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString; cr. each flushCache. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. file nextPutAll: 'Image size after cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString;cr. ]. file flush; close. And then I looked in the 'repos.txt' file. My package cache repo cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB. So...cleaning cache of just 3 repos frees approx 100MB.
The question is....can we flush the cache safely? If they are called "cache", then I guess yes, we can.
Thoughts?
Thanks,
On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
Guys, I have images also with seaside, magritte, glorp, postgresV2, etc and it is also around 200MB. I will try to do some research today and let you know.
Cheers,
On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Jul 30, 2013, at 1:49 PM, "phil@highoctane.be" <phil@highoctane.be> wrote:
the changes file contained passwords and I replaced the text. So offsets may be wrong due to that.
Yes, the first thing I wanted to do is to recompile everything. Does not work.
Memorymonitor is not doing fanct stuff. It just counts instances.
Yes, but maybe it holds on to these instances?
Marcus
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
On Aug 1, 2013, at 1:31 AM, phil@highoctane.be wrote:
Doing the flushCaches...
Image size before cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324017564 Image size after cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324137232 Image size before cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324137232 Image size after cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324195720 Image size before cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324195720 Image size after cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324255300 Image size before cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324255300 Image size after cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324314868
Definitely not going down...
No, we already saw that in your case the reason are many,many instances of MorphExtension without the Morphs hanging around. (which should not happen as MorphExtensions are only referenced by the Morph they extend). Your case is not a case of caches, there is a real problem somewhere. Marcus
I did a little run on those MorphExtensions and got the output file in annex. Cursor wait showWhile: [ (FileSystem disk workingDirectory / 'MorphExtensions.txt') asFileReference writeStreamDo: [ :s | MorphExtension allInstancesDo: [ :each | each printOn: s. s lf ] ] ]. Looks like there are a lot of Announcers. Phil On Thu, Aug 1, 2013 at 9:35 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Aug 1, 2013, at 1:31 AM, phil@highoctane.be wrote:
Doing the flushCaches...
Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324017564 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324137232 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324137232 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324195720 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324195720 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324255300 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324255300 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324314868
Definitely not going down...
No, we already saw that in your case the reason are many,many instances of MorphExtension without the Morphs hanging around. (which should not happen as MorphExtensions are only referenced by the Morph they extend).
Your case is not a case of caches, there is a real problem somewhere.
Marcus
Yes, I think the announcers of morph might be the cause of the problem... On Thu, Aug 1, 2013 at 1:09 PM, phil@highoctane.be <phil@highoctane.be>wrote:
I did a little run on those MorphExtensions and got the output file in annex.
Cursor wait showWhile: [ (FileSystem disk workingDirectory / 'MorphExtensions.txt') asFileReference writeStreamDo: [ :s | MorphExtension allInstancesDo: [ :each | each printOn: s. s lf ] ] ].
Looks like there are a lot of Announcers.
Phil
On Thu, Aug 1, 2013 at 9:35 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Aug 1, 2013, at 1:31 AM, phil@highoctane.be wrote:
Doing the flushCaches...
Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324017564 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324137232 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324137232 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324195720 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324195720 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324255300 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324255300 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324314868
Definitely not going down...
No, we already saw that in your case the reason are many,many instances of MorphExtension without the Morphs hanging around. (which should not happen as MorphExtensions are only referenced by the Morph they extend).
Your case is not a case of caches, there is a real problem somewhere.
Marcus
-- -- Marcus Denker -- denker@acm.org http://www.marcusdenker.de
Ok, we are looking at that with marcus. So, one thing that may be causing problems is that there are some usages of #onAnnouncement: anAnnouncement do: aBlock. And since we don't have ephermerons, the usage of blocks in announcements is leaking references. Also, looking at the usages of onAnnouncement:do:, we found NECController>>setEditor:, which is called for every keystroke in when using autocompletion :D. Maybe the cause is around here? On Thu, Aug 1, 2013 at 1:40 PM, Marcus Denker <marcus.denker@inria.fr>wrote:
Yes, I think the announcers of morph might be the cause of the problem...
On Thu, Aug 1, 2013 at 1:09 PM, phil@highoctane.be <phil@highoctane.be>wrote:
I did a little run on those MorphExtensions and got the output file in annex.
Cursor wait showWhile: [ (FileSystem disk workingDirectory / 'MorphExtensions.txt') asFileReference writeStreamDo: [ :s | MorphExtension allInstancesDo: [ :each | each printOn: s. s lf ] ] ].
Looks like there are a lot of Announcers.
Phil
On Thu, Aug 1, 2013 at 9:35 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Aug 1, 2013, at 1:31 AM, phil@highoctane.be wrote:
Doing the flushCaches...
Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324017564 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324137232 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324137232 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324195720 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324195720 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324255300 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324255300 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324314868
Definitely not going down...
No, we already saw that in your case the reason are many,many instances of MorphExtension without the Morphs hanging around. (which should not happen as MorphExtensions are only referenced by the Morph they extend).
Your case is not a case of caches, there is a real problem somewhere.
Marcus
-- -- Marcus Denker -- denker@acm.org http://www.marcusdenker.de
I got a pretty large number with these announcements... provided nothing special is running at the moment... Announcement allSubInstances size. 14120 And: (Announcement allSubInstances collect: [ :a | a class ]) asSet a Set(WindowOpened WindowLabelled WindowClosed KMCompleteMatch TestSuiteEnded ClassRemoved) And ann:=(Announcement allSubInstances collect: [ :a | a class ]) asSet. ann do: [ :each | Transcript nextPutAll: each className; nextPut: $:; nextPutAll: each allInstances size printString;cr]. Transcript flush. gives: WindowOpened class:WindowOpened class:953 WindowOpened class:956 WindowLabelled class:11837 WindowClosed class:1345 KMCompleteMatch class:6 TestSuiteEnded class:1 ClassRemoved class:1 Well, 11837 for WindowLabelled is quite large. Hope it gives a hint. Phil
Continuing on the topic, it looks like that every single window that I ever opened is remembered by the Announcements... Cursor wait showWhile: [ (FileSystem disk workingDirectory / 'WindowLabelled-labels.txt') asFileReference writeStreamDo: [ :s | WindowLabelled allInstances do: [ :each | each label printOn: s. s lf ]. ] ]. The attached file is what I got. Phil On Thu, Aug 1, 2013 at 3:47 PM, Guillermo Polito <guillermopolito@gmail.com>wrote:
Ok, we are looking at that with marcus.
So, one thing that may be causing problems is that there are some usages of #onAnnouncement: anAnnouncement do: aBlock. And since we don't have ephermerons, the usage of blocks in announcements is leaking references.
Also, looking at the usages of onAnnouncement:do:, we found NECController>>setEditor:, which is called for every keystroke in when using autocompletion :D. Maybe the cause is around here?
On Thu, Aug 1, 2013 at 1:40 PM, Marcus Denker <marcus.denker@inria.fr>wrote:
Yes, I think the announcers of morph might be the cause of the problem...
On Thu, Aug 1, 2013 at 1:09 PM, phil@highoctane.be <phil@highoctane.be>wrote:
I did a little run on those MorphExtensions and got the output file in annex.
Cursor wait showWhile: [ (FileSystem disk workingDirectory / 'MorphExtensions.txt') asFileReference writeStreamDo: [ :s | MorphExtension allInstancesDo: [ :each | each printOn: s. s lf ] ] ].
Looks like there are a lot of Announcers.
Phil
On Thu, Aug 1, 2013 at 9:35 AM, Marcus Denker <marcus.denker@inria.fr>wrote:
On Aug 1, 2013, at 1:31 AM, phil@highoctane.be wrote:
Doing the flushCaches...
Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324017564 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324137232 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324137232 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324195720 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324195720 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324255300 Image size before cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324255300 Image size after cleaning a MCSmalltalkhubRepository( http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324314868
Definitely not going down...
No, we already saw that in your case the reason are many,many instances of MorphExtension without the Morphs hanging around. (which should not happen as MorphExtensions are only referenced by the Morph they extend).
Your case is not a case of caches, there is a real problem somewhere.
Marcus
-- -- Marcus Denker -- denker@acm.org http://www.marcusdenker.de
indeed this is what marcus thought. So we found a nice bug. Stef On Aug 1, 2013, at 7:47 PM, phil@highoctane.be wrote:
Continuing on the topic, it looks like that every single window that I ever opened is remembered by the Announcements...
Cursor wait showWhile: [ (FileSystem disk workingDirectory / 'WindowLabelled-labels.txt') asFileReference writeStreamDo: [ :s |
WindowLabelled allInstances do: [ :each | each label printOn: s. s lf ].
] ].
The attached file is what I got.
Phil
On Thu, Aug 1, 2013 at 3:47 PM, Guillermo Polito <guillermopolito@gmail.com> wrote: Ok, we are looking at that with marcus.
So, one thing that may be causing problems is that there are some usages of #onAnnouncement: anAnnouncement do: aBlock. And since we don't have ephermerons, the usage of blocks in announcements is leaking references.
Also, looking at the usages of onAnnouncement:do:, we found NECController>>setEditor:, which is called for every keystroke in when using autocompletion :D. Maybe the cause is around here?
On Thu, Aug 1, 2013 at 1:40 PM, Marcus Denker <marcus.denker@inria.fr> wrote: Yes, I think the announcers of morph might be the cause of the problem...
On Thu, Aug 1, 2013 at 1:09 PM, phil@highoctane.be <phil@highoctane.be> wrote: I did a little run on those MorphExtensions and got the output file in annex.
Cursor wait showWhile: [ (FileSystem disk workingDirectory / 'MorphExtensions.txt') asFileReference writeStreamDo: [ :s | MorphExtension allInstancesDo: [ :each | each printOn: s. s lf ] ] ].
Looks like there are a lot of Announcers.
Phil
On Thu, Aug 1, 2013 at 9:35 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On Aug 1, 2013, at 1:31 AM, phil@highoctane.be wrote:
Doing the flushCaches...
Image size before cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324017564 Image size after cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/estebanlm/EclipsePack/main/) : 324137232 Image size before cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324137232 Image size after cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/DripfeedIt/main/) : 324195720 Image size before cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324195720 Image size after cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/HOWebStack/main/) : 324255300 Image size before cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324255300 Image size after cleaning a MCSmalltalkhubRepository(http://smalltalkhub.com/mc/philippeback/HOExperiments/main/) : 324314868
Definitely not going down...
No, we already saw that in your case the reason are many,many instances of MorphExtension without the Morphs hanging around. (which should not happen as MorphExtensions are only referenced by the Morph they extend).
Your case is not a case of caches, there is a real problem somewhere.
Marcus
-- -- Marcus Denker -- denker@acm.org http://www.marcusdenker.de
<WindowLabelled-labels.txt.gz>
On Aug 1, 2013, at 9:04 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
indeed this is what marcus thought. So we found a nice bug.
I added a tracker entry: https://pharo.fogbugz.com/f/cases/11291/Announcer-for-Morphs-should-be-Weak (I will be on holidays starting tomorrow afternoon, so no action on this from me for a whileâ¦) Marcus
since years marcus is telling that MC storing ancestor information is doomed but we do not have something to really replace it. Stef On Jul 31, 2013, at 8:53 PM, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
OK, this was was my experiment....
Image fresh with all my app and dependencies loades: 30MB
After using it for some days/weeks: 160MB.
SpaceTally new printSpaceAnalysis showed:
Class code space # instances inst space percent inst average size ByteString 2785 413144 116244078 69.90 281.36 Array 3712 181772 8466668 5.10 46.58 ByteArray 8574 1319 8186802 4.90 6206.82 Bitmap 3653 303 6656340 4.00 21968.12 CompiledMethod 22467 90554 5685374 3.40 62.78
After executing ImageCleaner cleanUpForRelease: 36MB
Then...I searched which part of #cleanUpForRelease: was making the difference, and it was:
Smalltalk cleanUp: true except: #() confirming: false.
So now it was time to know WHICH class did the diference, so I modified #cleanUp: aggressive except: exclusions confirming: aBool
in these lines:
"Run the cleanup code" classes do:[:aClass| Transcript show: 'Image size before cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. aClass cleanUp: aggressive. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. Transcript show: 'Image size after cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. ] displayingProgress: [:aClass| 'Cleaning up in ', aClass name].
I then opened a Transcript, and evaluated
Smalltalk cleanUp: true except: #() confirming: false.
I went to prepare Mate, and when I come back, the result was, of course:
"Image size after cleaning MCFileBasedRepository : 39744008"
That clean up ends up doing:
flushAllCaches self allSubInstancesDo: [:ea | ea flushCache]
So it sends #flushCache to all instances of MCHttpRepository and MCFileBasedRepository.
Now what I wanted to see if it there was a particular repo that could take most of the space (like package-cache). And indeed, it was...I modified #flushCaches to:
flushAllCaches | file | file := 'repos.txt' asFileReference writeStream text. self allSubInstancesDo: [:each |
file nextPutAll: 'Image size before cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString; cr. each flushCache. 3 timesRepeat: [Smalltalk garbageCollect]. Smalltalk snapshot: true andQuit: false. file nextPutAll: 'Image size after cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString;cr.
]. file flush; close.
And then I looked in the 'repos.txt' file. My package cache repo cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB. So...cleaning cache of just 3 repos frees approx 100MB.
The question is....can we flush the cache safely? If they are called "cache", then I guess yes, we can.
Thoughts?
Thanks,
On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck <marianopeck@gmail.com> wrote: Guys, I have images also with seaside, magritte, glorp, postgresV2, etc and it is also around 200MB. I will try to do some research today and let you know.
Cheers,
On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On Jul 30, 2013, at 1:49 PM, "phil@highoctane.be" <phil@highoctane.be> wrote:
the changes file contained passwords and I replaced the text. So offsets may be wrong due to that.
Yes, the first thing I wanted to do is to recompile everything. Does not work.
Memorymonitor is not doing fanct stuff. It just counts instances.
Yes, but maybe it holds on to these instances?
Marcus
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
On Aug 1, 2013, at 8:56 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
since years marcus is telling that MC storing ancestor information is doomed but we do not have something to really replace it.
This is yet another thing. Monticello keep all history data of all package in image, this is not a cache. When you delete it, your package loses it's history and you can not merge anymore. MCVersionInfo allInstances do: [ :each | each instVarNamed: 'ancestors' put: nil ]. We did that on Pharo2 release and it saved 8MB (nearly a third of the image). Marcus
Hi Marcus, On Thu, Aug 1, 2013 at 12:33 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On Aug 1, 2013, at 8:56 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
since years marcus is telling that MC storing ancestor information is doomed but we do not have something to really replace it.
This is yet another thing.
Monticello keep all history data of all package in image, this is not a cache. When you delete it, your package loses it's history and you can not merge anymore.
MCVersionInfo allInstances do: [ :each | each instVarNamed: 'ancestors' put: nil ].
When I looked at this I noticed that there was lots of duplicate entries in the ancestry list. I *think* this happens on merge. I think one ends up with each of the ancestries of the merged packages after a merge, i.e. when merging near the head one effectively doubles the ancestry. I think Monticello should unique the ancestry, either after a merge, or better still, before saving. This I think will shrink the space taken by ancestry significantly. I have some code handy if you'd like, but its not a hard thing to do.
We did that on Pharo2 release and it saved 8MB (nearly a third of the image).
Marcus
-- best, Eliot
On Aug 3, 2013, at 2:13 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Marcus,
On Thu, Aug 1, 2013 at 12:33 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On Aug 1, 2013, at 8:56 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
since years marcus is telling that MC storing ancestor information is doomed but we do not have something to really replace it.
This is yet another thing.
Monticello keep all history data of all package in image, this is not a cache. When you delete it, your package loses it's history and you can not merge anymore.
MCVersionInfo allInstances do: [ :each | each instVarNamed: 'ancestors' put: nil ].
When I looked at this I noticed that there was lots of duplicate entries in the ancestry list. I *think* this happens on merge. I think one ends up with each of the ancestries of the merged packages after a merge, i.e. when merging near the head one effectively doubles the ancestry. I think Monticello should unique the ancestry, either after a merge, or better still, before saving. This I think will shrink the space taken by ancestry significantly.
I have some code handy if you'd like, but its not a hard thing to do.
Can you send it so that we have a look? Tx
We did that on Pharo2 release and it saved 8MB (nearly a third of the image).
Marcus
-- best, Eliot
On Aug 3, 2013, at 2:13 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Marcus,
On Thu, Aug 1, 2013 at 12:33 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On Aug 1, 2013, at 8:56 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
since years marcus is telling that MC storing ancestor information is doomed but we do not have something to really replace it.
This is yet another thing.
Monticello keep all history data of all package in image, this is not a cache. When you delete it, your package loses it's history and you can not merge anymore.
MCVersionInfo allInstances do: [ :each | each instVarNamed: 'ancestors' put: nil ].
When I looked at this I noticed that there was lots of duplicate entries in the ancestry list. I *think* this happens on merge. I think one ends up with each of the ancestries of the merged packages after a merge, i.e. when merging near the head one effectively doubles the ancestry. I think Monticello should unique the ancestry, either after a merge, or better still, before saving. This I think will shrink the space taken by ancestry significantly.
Ahh⦠that explains why it wastes so much memory.
I have some code handy if you'd like, but its not a hard thing to do.
If you have it I would be interested⦠I have added the information to https://pharo.fogbugz.com/f/cases/2711 Marcus
participants (6)
-
Eliot Miranda -
Guillermo Polito -
Marcus Denker -
Mariano Martinez Peck -
phil＠highoctane.be -
Stéphane Ducasse