On Wed, Mar 9, 2016 at 3:08 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:

Hi eliot

Do not worry. Now would it be possible to have a test or something like that that would help us to - document this part of the system - check that the system is in a coherent state

Stef

Yes, watch this space. I should have the work done by tomorrow. And the 64-bit system will be a tiny bit smaller and faster :-) On 09 Mar 2016, at 10:26, Eliot Miranda <eliot.miranda@gmail.com> wrote:

Um, under /no/ circumstances do what I'm suggesting below. It will screw up 320bit images completely. I've just turned odd SmallIntegers into SmallFloat54's in 32-bit Squeak. I hope I won't destroy too many people's images. This is perhaps my worst blunder.

On Tue, Mar 8, 2016 at 10:38 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:

...

Hi Pavel,

On Tue, Mar 8, 2016 at 1:40 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

...

Hi,

there is a strange array of size 1024 in the Pharo image that is not eaten by garbage collector even if no object points to it. I guess it is related to Spur.

((Array allInstances select: [ :i | i size = 1024 ]) select: [ :a | a second = SmallInteger ]) pointersTo

What is it and is it accessible somehow?

It is indeed the first class table page. It looks like an old bug. The oldest Spur image I can start up has the problem. The way that class table pages are hidden is that they have a class index which is not that of Array. Array is at index 52 in the class table (51 zero-relative, Array identityHash = 51), but it is also at index 17 (16 zero-relative). Array allInstances looks for objects whose class index is 51, hence not seeing the class table pages, which all (but the first) have class index 16.

Luckily we can fix the problem easily, and we can fix another problem at the same time, which is that SmallFloat64 has the wrong hash.

The class table is two-level; a (hidden) array of Arrays. Classes are in the class table at their identityHashes (so that to instantiate a class one copies the class's identity hash into the classIndex of the new instance instead of having to search the class table). But when SmallFloat64 was created in the 32-bit image I forgot to give it the right hash and store it in the class table. (there are no SmallFloat64 instances in a 32-bit image, only in 64-bits).

So we can first change SmallFloat64's hash to 3, which is what the Spur 64-bit VM and image require, and enter it into the first class table page, and then we can make the first class table page disappear.

| firstClassTablePage clone | "The first class table page is the first Array instance. Of course this is a bug." firstClassTablePage := Array someInstance.

"It should contain only nil and classes." self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e isBehavior]]).

"And its first 17 elements should only be the immediate classes and Array, since Array is used as the class pun for class table pages, with index 17 (16 zero-relative)" self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet.

"It just so happens that the second Array is the specialSelectors" self assert: Array someInstance nextInstance == Smalltalk specialSelectors.

"Store SmallFloat64 in the first class table page at index 4 (3 zero-relative)." firstClassTablePage at: 4 put: SmallFloat64.

"Use the secret set identity hash primitive to set SmallFloat64's identityHash to 3." SmallFloat64 tryPrimitive: 161 withArgs: #(3).

"Now create an object that looks like Array class, but has identityHash 16." "Take a shallow copy of Array class." clone := Array shallowCopy. "Change it into an Array." Array adoptInstance: clone. "Set its identityHash to 16." clone tryPrimitive: 161 withArgs: #(16). "Use the adoptInstance: primitive to ``set the class of the firstClassTablePage to the cone''. or, more accurately, to set the firstClassTablePage's class index to 16." clone tryPrimitive: 160 withArgs: {firstClassTablePage}

With the above done, we can check that everything is ok." self assert: SmallFloat64 identityHash = 3. self assert: Array someInstance == Smalltalk specialSelectors. "A class table page is 1024 slots, contains only nil or behaviours, and contains at least one behaviour (is not all nils). There shouldn't be any that we can find." self assert: (self systemNavigation allObjects select: [:o| o isArray and: [o size = 1024 and: [(o allSatisfy: [:e| e == nil or: [e isBehavior]]) and: [o anySatisfy: [:e| e isBehavior]]]]]) size = 0

I recommend you create an update map. Then create a version of kernel with a post-load action, written something like this:

(Array someInstance size = 1014 and: [(Array someInstance allSatisfy: [:e| e == nil or: [e isBehavior]]) and: [(firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet]]) ifTrue: [| firstClassTablePage clone | firstClassTablePage := Array someInstance. self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e isBehavior]]). self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet. firstClassTablePage at: 4 put: SmallFloat64. SmallFloat64 tryPrimitive: 161 withArgs: #(3). clone := Array shallowCopy. Array adoptInstance: clone. clone tryPrimitive: 161 withArgs: #(16). clone tryPrimitive: 160 withArgs: {Array someInstance} self assert: SmallFloat64 identityHash = 3. self assert: (Array someInstance first: 4) = {nil. SmallInteger. Character. SmallFloat64}]

Then create a second update map to ensure that that version of Kernel is loaded.

I will do this for Squeak immediately.

Apologies.

P.S. Interestingly enough, it shows what a horrible security hole the debugger primitives tryPrimitive:withArgs: and tryNamedPrimitive:withArgs: are.

Cheers,

...

-- Pavel

_,,,^..^,,,_ best, Eliot

-- _,,,^..^,,,_ best, Eliot

-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr http://www.synectique.eu / http://www.pharo.org 03 59 35 87 52 Assistant: Julie Jonas 03 59 57 78 50 03 59 35 86 16

S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France

-- _,,,^..^,,,_ best, Eliot