2010/9/21 Henrik Sperre Johansen <henrik.s.johansen@veloxit.no>

On 21.09.2010 20:07, Mariano Martinez Peck wrote:

On Tue, Sep 21, 2010 at 7:55 PM, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:

�On 21.09.2010 19:10, Mariano Martinez Peck wrote:

Hi folks. I wasn't aware of the class SparseLargeTable but it seems it has two important instances "SparseLargeTable allInstances size -> 2"

They are the class side variables #DecimalProperty and #GeneralCategory in Unicode.

Actually,

(Unicode classPool at: 'GeneralCategory') size �-> 917632

(Unicode classPool at: 'DecimalProperty') size -> 917632

So...collections of almost 1 million elements.

Now a couple of questions:

1) Is this normal/expected? � In my calculus both arrays are like 3mb of memory

Look at the implementation, there's a reason they're called sparse.
If my math is correct (crosses fingers), DecimalProperty is ~15KB, GeneralCategory ~70KB

So...there must be a problem with my code, I was doing something like SpaceTally does:

spaceForUsedInstancesOf: aClass withInstanceCount: instCount
�� "Answer the number of bytes consumed by all instances of the given class, including their object headers."

�� | isCompact instVarBytes bytesPerElement headerBytes total |
�� instCount = 0 ifTrue: [^ 0].
�� isCompact := aClass indexIfCompact > 0.
�� instVarBytes := aClass instSize * 4.
�� aClass isVariable
�� ifTrue: [
�� bytesPerElement := aClass isBytes ifTrue: [1] ifFalse: [4].
�� total := 0.
�� aClass allInstancesDo: [:inst | | contentBytes |
�� contentBytes := instVarBytes + (inst size * bytesPerElement).

There you go. Use basicSize, size is overridden by f.ex. HashedCollections and LargeSparseTables.

:)� Thanks Henry. Now� SpaceTally new spaceForInstancesOf: SparseLargeTable withInstanceCount: 2� ->> 18748

So....I guess I should commit the fix for SpaceTally >> spaceForInstancesOf: aClass withInstanceCount: instCount

shouldn't I ?

In addition, I modified SpaceTally >> printSpaceAnalysis: threshold on: aStream

from this:

printSpaceAnalysis: threshold on: aStream
�� "SpaceTally new printSpaceAnalysis: 1 on:(FileStream forceNewFileNamed: 'STspace.text')"

�� "sd-This method should be rewrote to be more coherent within the rest of the class
�� ie using preAllocate and spaceForInstanceOf:"

�� "If threshold > 0, then only those classes with more than that number
�� of instances will be shown, and they will be sorted by total instance space.
�� If threshold = 0, then all classes will appear, sorted by name."

�� | totalCodeSpace totalInstCount totalInstSpace n totalPercent |
�� Smalltalk garbageCollect.
�� totalCodeSpace := totalInstCount := totalInstSpace := n := 0.
�� results := OrderedCollection new: Smalltalk classNames size.
�� 'Taking statistics...'
�� displayProgressAt: Sensor cursorPoint
�� from: 0 to: Smalltalk classNames size
�� during: [:bar |
�� (Smalltalk globals allClasses) do:
�� [:cl | | codeSpace instCount instSpace eltSize | codeSpace := cl spaceUsed.
�� bar value: (n := n+1).
�� Smalltalk garbageCollectMost.
�� instCount := cl instanceCount.
�� instSpace := (cl indexIfCompact > 0 ifTrue: [4] ifFalse: [8])*instCount. "Object headers"
�� cl isVariable
�� ifTrue: [eltSize := cl isBytes ifTrue: [1] ifFalse: [4].
�� cl allInstancesDo: [:x | instSpace := instSpace + (x basicSize*eltSize)]]
�� ifFalse: [instSpace := instSpace + (cl instSize*instCount*4)].
�� results add: (SpaceTallyItem analyzedClassName: cl name codeSize: codeSpace instanceCount:� instCount spaceForInstances: instSpace).
�� totalCodeSpace := totalCodeSpace + codeSpace.
�� totalInstCount := totalInstCount + instCount.
�� totalInstSpace := totalInstSpace + instSpace]].
�� totalPercent := 0.0.

�� "aStream timeStamp."
�� aStream
�� nextPutAll: ('Class' padded: #right to: 30 with: $ );
�� nextPutAll: ('code space' padded: #left to: 12 with: $ );
�� nextPutAll: ('# instances' padded: #left to: 12 with: $ );
�� nextPutAll: ('inst space' padded: #left to: 12 with: $ );
�� nextPutAll: ('percent' padded: #left to: 8 with: $ ); cr.

�� threshold > 0 ifTrue: [
�� "If inst count threshold > 0, then sort by space"
�� results := (results select: [:s | s instanceCount >= threshold or: [s spaceForInstances > (totalInstSpace // 500)]])
�� asSortedCollection: [:s :s2 | s spaceForInstances > s2 spaceForInstances]].

�� results do: [:s | | percent |
�� aStream
�� nextPutAll: (s analyzedClassName padded: #right to: 30 with: $ );
�� nextPutAll: (s codeSize printString padded: #left to: 12 with: $ );
�� nextPutAll: (s instanceCount printString padded: #left to: 12 with: $ );
�� nextPutAll: (s spaceForInstances printString padded: #left to: 14 with: $ ).
�� percent := s spaceForInstances*100.0/totalInstSpace roundTo: 0.1.
�� totalPercent := totalPercent + percent.
�� percent >= 0.1 ifTrue: [
�� aStream nextPutAll: (percent printString padded: #left to: 8 with: $ )].
�� aStream cr].

�� aStream
�� cr; nextPutAll: ('Total' padded: #right to: 30 with: $ );
�� nextPutAll: (totalCodeSpace printString padded: #left to: 12 with: $ );
�� nextPutAll: (totalInstCount printString padded: #left to: 12 with: $ );
�� nextPutAll: (totalInstSpace printString padded: #left to: 14 with: $ );
�� nextPutAll: ((totalPercent roundTo: 0.1) printString padded: #left to: 8 with: $ ).

�

To this:

printSpaceAnalysis: threshold on: aStream
�� "SpaceTally new printSpaceAnalysis: 1 on:(FileStream forceNewFileNamed: 'STspace.text')"

�� "sd-This method should be rewrote to be more coherent within the rest of the class
�� ie using preAllocate and spaceForInstanceOf:"

�� "If threshold > 0, then only those classes with more than that number
�� of instances will be shown, and they will be sorted by total instance space.
�� If threshold = 0, then all classes will appear, sorted by name."

�� | totalCodeSpace totalInstCount totalInstSpace n totalPercent |
�� Smalltalk garbageCollect.
�� totalCodeSpace := totalInstCount := totalInstSpace := n := 0.
�� results := OrderedCollection new: Smalltalk classNames size.
�� 'Taking statistics...'
�� displayProgressAt: Sensor cursorPoint
�� from: 0 to: Smalltalk classNames size
�� during: [:bar |
�� (Smalltalk globals allClasses) do:
�� [:cl | | codeSpace instCount instSpace eltSize | codeSpace := cl spaceUsed.
�� bar value: (n := n+1).
�� Smalltalk garbageCollectMost.
�� instCount := cl instanceCount.
�� instSpace := self spaceForInstancesOf: cl withInstanceCount: instCount.
�� results add: (SpaceTallyItem analyzedClassName: cl name codeSize: codeSpace instanceCount:� instCount spaceForInstances: instSpace).
�� totalCodeSpace := totalCodeSpace + codeSpace.
�� totalInstCount := totalInstCount + instCount.
�� totalInstSpace := totalInstSpace + instSpace]].
�� totalPercent := 0.0.

�� "aStream timeStamp."
�� aStream
�� nextPutAll: ('Class' padded: #right to: 30 with: $ );
�� nextPutAll: ('code space' padded: #left to: 12 with: $ );
�� nextPutAll: ('# instances' padded: #left to: 12 with: $ );
�� nextPutAll: ('inst space' padded: #left to: 12 with: $ );
�� nextPutAll: ('percent' padded: #left to: 8 with: $ ); cr.

�� threshold > 0 ifTrue: [
�� "If inst count threshold > 0, then sort by space"
�� results := (results select: [:s | s instanceCount >= threshold or: [s spaceForInstances > (totalInstSpace // 500)]])
�� asSortedCollection: [:s :s2 | s spaceForInstances > s2 spaceForInstances]].

�� results do: [:s | | percent |
�� aStream
�� nextPutAll: (s analyzedClassName padded: #right to: 30 with: $ );
�� nextPutAll: (s codeSize printString padded: #left to: 12 with: $ );
�� nextPutAll: (s instanceCount printString padded: #left to: 12 with: $ );
�� nextPutAll: (s spaceForInstances printString padded: #left to: 14 with: $ ).
�� percent := s spaceForInstances*100.0/totalInstSpace roundTo: 0.1.
�� totalPercent := totalPercent + percent.
�� percent >= 0.1 ifTrue: [
�� aStream nextPutAll: (percent printString padded: #left to: 8 with: $ )].
�� aStream cr].

�� aStream
�� cr; nextPutAll: ('Total' padded: #right to: 30 with: $ );
�� nextPutAll: (totalCodeSpace printString padded: #left to: 12 with: $ );
�� nextPutAll: (totalInstCount printString padded: #left to: 12 with: $ );
�� nextPutAll: (totalInstSpace printString padded: #left to: 14 with: $ );
�� nextPutAll: ((totalPercent roundTo: 0.1) printString padded: #left to: 8 with: $ ).

So....you agree with both changes?

Thanks!

Mariano

Cheers,
Henry

_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project