November 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] object format of true, false, nil and others...

by StÃ©phane Ducasse

Thanks for now I should eat my own dog food and do it and stop talking or even thinking about it. But I'm sucked by the world around me (wife, kids, colleagues, mail, admin, I should go to an island without humans and internet). Stef On Nov 25, 2011, at 10:14 PM, Toon Verwaest wrote: > >>> That's what we do in Pinocchio. We just ask for instSize, amongst other things... Maybe I'm missing something? :) >> right now I will start with stub to get some momentum and think after. >> I was reading the bootstrap of Mate and it should be close to the one of pinocchio. >> I'm reading also micro squeak and gst >> But I should bump into the walls to progress. >> >> Stef > Let me know if you have specific questions I can help with... I'm not hyper-responsive (nor reading mailing lists) since working on paper and thesis, but if you mail me directly I might be able to help out with questions at least. > > Toon >

Nov. 25, 2011

Re: [Pharo-project] object format of true, false, nil and others...

by Toon Verwaest

>> That's what we do in Pinocchio. We just ask for instSize, amongst other things... Maybe I'm missing something? :) > right now I will start with stub to get some momentum and think after. > I was reading the bootstrap of Mate and it should be close to the one of pinocchio. > I'm reading also micro squeak and gst > But I should bump into the walls to progress. > > Stef Let me know if you have specific questions I can help with... I'm not hyper-responsive (nor reading mailing lists) since working on paper and thesis, but if you mail me directly I might be able to help out with questions at least. Toon

Nov. 25, 2011

Re: [Pharo-project] object format of true, false, nil and others...

by StÃ©phane Ducasse

On Nov 25, 2011, at 10:03 PM, Toon Verwaest wrote: > To reply to the question from our pov: > > In Pinocchio we actually need that nil, true and false fall within the first 4gb of virtual memory space, since our native code directly addresses those objects; and AMD64 code can only use 32bits for direct addresses. > > This is kinda sucky since OSX thinks it's a good idea to reserve the whole first 4gb virtual address space as "null memory", to ensure they really really really capture all accidental NULL-pointer dereferences. And obviously the standard 4K isn't enough for OSX developers. Oh well, we managed to fix that with some ld flag on OSX. > > We do this since it gives a very slight advantage in performance, and we don't seriously expect nil, true and false to ever change. And if they do... we can still visit all code in the system and change it :) right now I just need a placeholder after I imagine that the specialObjectArray and system tracer can let me specify what I want in the resulting image. And may be I could use mate explicit object format description to migrate (instead of systemTracer). > > cheers, > Toon > > On 11/23/2011 11:25 PM, Stefan Marr wrote: >> Hi: >> >> On 23 Nov 2011, at 22:59, StÃ©phane Ducasse wrote: >>> imagine that I want to bootstrap our nice kernel. >>> I will need to somehow create a new nil, true, false, instances of new classes Undefined, True, False. >>> Such instances will then be put on the SpecialObjectArray somehow. >> Something kind of related: >> >> When debugging the VM, I am happy that I always encounter the same object as the first object in the heap. It is some strange string (think it encodes an error message or so). >> >> However, my thought was, also for other purposes, would it be possible to have these special objects sorted to the beginning? Especially, the nilObj would be very handy to have as the first object. >> >> That would need special VM support to arrange it that way, but knowing the nilObj from the very beginning would allow my to initialize some of my hacks more consistently, avoiding C-level NULLs and runtime checks. >> >> Just an idea, vaguely related to the question. >> >> Would be interesting to hear whether there are any historical insights with regard to ordering objects in the image. >> Guess it is not done because nobody needed it. (and then there isn't really a primitive for it, and a moving GC might also be contra-productive, etc.) >> >> Best regards >> Stefan >> >> >> >>> Stef > >

Nov. 25, 2011

Re: [Pharo-project] ProtoObject basicNew :)

by Toon Verwaest

Ok, I guess I was confused by your initial question: "the methods that I should define in a subclass of ProtoObject so that I can use basicNew"... :) On 11/25/2011 10:05 PM, StÃ©phane Ducasse wrote: > I'm confused. > I'm not creating a class but an instance. > if it would be Behavior basicNew then yes I made sure that we could do Behavior new and not crashing the system back in Squeak. > > On Nov 25, 2011, at 9:45 PM, Toon Verwaest wrote: > >> With the magic number I mean the number that defines the object's structure. It's one of the first 3 instance variables of every class, encoding how many instance variables there are; what kind of object it is supposed to be (normal, array, bytes, words...). Every "class" basically needs 3 instance variables. 1 defines the structure, 1 the superclass, 1 the behavior (method dictionary). I don't know the other; but just look at some class to know what the order is. > Yes format, methodDict, superclass (probably in a different order). > And format is ugly because not explicit and coded in several place like instSpec (berk) > >> And yes, if you want to inspect the object, you'll need to support whatever protocol your inspector is using. >> >> The format is formed as follows: >> >> format >> | fieldSize sizeHiBits format | >> fieldSize := self fieldSize + 1. >> sizeHiBits := fieldSize // 64. >> format := sizeHiBits. >> format := (format bitShift: 5) + compactClassIndex. >> format := (format bitShift: 4) + self instanceSpecification. >> format := (format bitShift: 6) + (fieldSize \\ 64). >> format := (format bitShift: 1). >> ^ format >> >> If I read it correctly (just glancing), instance specification specifies if it's arrayed and if it has fields. 0 -> no fields, not arrayed, 1 -> fields, not arrayed, 2 -> arrayed no fields, 3 -> arrayed fields. >> >> >> HTH, >> Toon >> >> On 11/25/2011 09:31 PM, StÃ©phane Ducasse wrote: >>> Apparently I can do a doit I was naively doing an inspect ;) >>> >>> >>>> Don't you just need the basicNew primitive, and make sure that the "magic number" is set in the object that has the basicNew installed to something that makes sense? >>>> >>> I do not know what is the magic number. >>> I guess that my problem was that I have first to define what is the minimal behavior. >>> >>> >>>> If you want to send messages to the object, probably also best install some dictionary as the method dict instvar. >>>> >>> But I guess that ProtoObject has some behavior since I can browse the class with a browser. >>> >>> >>> >>>> Toon >>>> >>>> On 11/25/2011 09:26 PM, StÃ©phane Ducasse wrote: >>>> >>>>> Hi >>>>> I would like to know the methods that I should define in a subclass of ProtoObject so that I can use basicNew. >>>>> Does any of you have an idea? >>>>> >>>>> Stef >>>>> >

Nov. 25, 2011

Re: [Pharo-project] object format of true, false, nil and others...

by StÃ©phane Ducasse

>>> >>> >> but you should be able to calculate size taken from object format >> descriptor, don't you? > That's what we do in Pinocchio. We just ask for instSize, amongst other things... Maybe I'm missing something? :) right now I will start with stub to get some momentum and think after. I was reading the bootstrap of Mate and it should be close to the one of pinocchio. I'm reading also micro squeak and gst But I should bump into the walls to progress. Stef

Nov. 25, 2011

Re: [Pharo-project] ProtoObject basicNew :)

by StÃ©phane Ducasse

I'm confused. I'm not creating a class but an instance. if it would be Behavior basicNew then yes I made sure that we could do Behavior new and not crashing the system back in Squeak. On Nov 25, 2011, at 9:45 PM, Toon Verwaest wrote: > With the magic number I mean the number that defines the object's structure. It's one of the first 3 instance variables of every class, encoding how many instance variables there are; what kind of object it is supposed to be (normal, array, bytes, words...). Every "class" basically needs 3 instance variables. 1 defines the structure, 1 the superclass, 1 the behavior (method dictionary). I don't know the other; but just look at some class to know what the order is. Yes format, methodDict, superclass (probably in a different order). And format is ugly because not explicit and coded in several place like instSpec (berk) > > And yes, if you want to inspect the object, you'll need to support whatever protocol your inspector is using. > > The format is formed as follows: > > format > | fieldSize sizeHiBits format | > fieldSize := self fieldSize + 1. > sizeHiBits := fieldSize // 64. > format := sizeHiBits. > format := (format bitShift: 5) + compactClassIndex. > format := (format bitShift: 4) + self instanceSpecification. > format := (format bitShift: 6) + (fieldSize \\ 64). > format := (format bitShift: 1). > ^ format > > If I read it correctly (just glancing), instance specification specifies if it's arrayed and if it has fields. 0 -> no fields, not arrayed, 1 -> fields, not arrayed, 2 -> arrayed no fields, 3 -> arrayed fields. > > > HTH, > Toon > > On 11/25/2011 09:31 PM, StÃ©phane Ducasse wrote: >> Apparently I can do a doit I was naively doing an inspect ;) >> >> >>> Don't you just need the basicNew primitive, and make sure that the "magic number" is set in the object that has the basicNew installed to something that makes sense? >>> >> I do not know what is the magic number. >> I guess that my problem was that I have first to define what is the minimal behavior. >> >> >>> If you want to send messages to the object, probably also best install some dictionary as the method dict instvar. >>> >> But I guess that ProtoObject has some behavior since I can browse the class with a browser. >> >> >> >>> Toon >>> >>> On 11/25/2011 09:26 PM, StÃ©phane Ducasse wrote: >>> >>>> Hi >>>> I would like to know the methods that I should define in a subclass of ProtoObject so that I can use basicNew. >>>> Does any of you have an idea? >>>> >>>> Stef >>>> >>> >> >

Nov. 25, 2011

Re: [Pharo-project] object format of true, false, nil and others...

by Toon Verwaest

To reply to the question from our pov: In Pinocchio we actually need that nil, true and false fall within the first 4gb of virtual memory space, since our native code directly addresses those objects; and AMD64 code can only use 32bits for direct addresses. This is kinda sucky since OSX thinks it's a good idea to reserve the whole first 4gb virtual address space as "null memory", to ensure they really really really capture all accidental NULL-pointer dereferences. And obviously the standard 4K isn't enough for OSX developers. Oh well, we managed to fix that with some ld flag on OSX. We do this since it gives a very slight advantage in performance, and we don't seriously expect nil, true and false to ever change. And if they do... we can still visit all code in the system and change it :) cheers, Toon On 11/23/2011 11:25 PM, Stefan Marr wrote: > Hi: > > On 23 Nov 2011, at 22:59, StÃ©phane Ducasse wrote: >> imagine that I want to bootstrap our nice kernel. >> I will need to somehow create a new nil, true, false, instances of new classes Undefined, True, False. >> Such instances will then be put on the SpecialObjectArray somehow. > Something kind of related: > > When debugging the VM, I am happy that I always encounter the same object as the first object in the heap. It is some strange string (think it encodes an error message or so). > > However, my thought was, also for other purposes, would it be possible to have these special objects sorted to the beginning? Especially, the nilObj would be very handy to have as the first object. > > That would need special VM support to arrange it that way, but knowing the nilObj from the very beginning would allow my to initialize some of my hacks more consistently, avoiding C-level NULLs and runtime checks. > > Just an idea, vaguely related to the question. > > Would be interesting to hear whether there are any historical insights with regard to ordering objects in the image. > Guess it is not done because nobody needed it. (and then there isn't really a primitive for it, and a moving GC might also be contra-productive, etc.) > > Best regards > Stefan > > > >> Stef

Nov. 25, 2011

Re: [Pharo-project] object format of true, false, nil and others...

by Toon Verwaest

On 11/25/2011 12:14 AM, Igor Stasenko wrote: > On 24 November 2011 23:29, Javier PimÃ¡s<elpochodelagente(a)gmail.com> wrote: >> >> On Thu, Nov 24, 2011 at 4:58 PM, StÃ©phane Ducasse >> <stephane.ducasse(a)inria.fr> wrote: >>> >>>> We do the same for our bootstrapper. First allocate space for the >>>> objects used by the metamodel: Metaclass, Class, Behavior, Object, >>>> UndefinedObject, True, False, their instances, metaclasses, etc. Each object >>>> of its corresponding size. >>> how do you compute the size? >> >> Good question. For now we still don't have a design that convinces me in >> this area. In some cases we calculate it manually, for example in objects >> without fields. In other cases we take it from the current image, this is >> the case of metaclass and class. >> > but you should be able to calculate size taken from object format > descriptor, don't you? That's what we do in Pinocchio. We just ask for instSize, amongst other things... Maybe I'm missing something? :) > >> In the end, it depends on which are the compact classes, and what fields you >> assume that some objects will have, like class and metaclass. >> >> After this kernel is bootstraped, objects are created sending new so the >> size is calculated from the class. >> >>> >>>> After that, fill the header and fields of these objects. Finally, new >>>> objects can be created by sending messages to objects in this kernel. >>> I was planning to do the same so I would like to see your code. >>> Tomorrow I want to read it. >>> >>> Stef >>> >>>> Cheers, >>>> Javier. >>>> >>>> On Wed, Nov 23, 2011 at 10:59 PM, StÃ©phane Ducasse >>>> <stephane.ducasse(a)inria.fr> wrote: >>>> Hi >>>> >>>> imagine that I want to bootstrap our nice kernel. >>>> I will need to somehow create a new nil, true, false, instances of new >>>> classes Undefined, True, False. >>>> Such instances will then be put on the SpecialObjectArray somehow. >>>> >>>> Now I can create a dummy object playing such roles and then use >>>> SystemTracer to substitute my placeholder with the true nil falseâ¦. >>>> or another trick to swap them. Now anybody has other ideas? >>>> >>>> Stef >>>> >>>> >>>> >>>> -- >>>> Lic. Javier PimÃ¡s >>>> Ciudad de Buenos Aires >>> >> >> >> -- >> Lic. Javier PimÃ¡s >> Ciudad de Buenos Aires > >

Nov. 25, 2011

[Pharo-project] class rename and Fuel

by Yanni Chiu

Here's my scenario: - a Pier kernel was previously serialized with Fuel-1.7 - a class was re-named (happens to be a subclass of PRCommand) - get a walkback when trying to de-serialize with the same Fuel-1.7 - define the missing class, then de-serialize works I was hoping that, after serializing the Pier kernel again (with both classes present, to be able to load), then on the next re-start, the old class would no longer be needed. That did not work - same problem of FLClassNotFound. To debug this, I de-serialized in a development image. To my surprise, evaluating "OldClass allInstances" found 0 instances of the OldClass. So I can't figure out how to track down why Fuel is still looking for that old class. BTW, since Pier looks for all subclasses of PRCommand, I made the NewClass a subclass of Object. That did not fix the problem. Any ideas why Fuel is looking for OldClass, yet there are no instances of OldClass in the de-serialized object graph. The stack trace is included at the end. -- Yanni ===== stack trace ===== [FLClassNotFound new signal: 'Class ' , className , ' not found at materialization'] in FLGlobalClassCluster(FLGlobalCluster)>>materializeGlobalClassFrom: SystemDictionary(Dictionary)>>at:ifAbsent: FLGlobalClassCluster(FLGlobalCluster)>>materializeGlobalClassFrom: FLGlobalClassCluster>>materializeFrom: FLGlobalClassCluster(FLPrimitiveCluster)>>materializeInstanceWith: [objects add: (self materializeInstanceWith: aMaterialization)] in FLGlobalClassCluster(FLIteratingCluster)>>materializeInstancesStepWith: SmallInteger(Integer)>>timesRepeat: FLGlobalClassCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [self clusterInstancesStep] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [:aStream | (FLMaterialization on: aStream) run; yourself] in FLMaterializer>>setDefaultMaterialization [self verifySignatureFrom: aStream. self verifyVersionFrom: aStream. ^ materializationFactory value: aStream] in FLMaterializer>>materializeFrom: BlockClosure>>ensure: FLMaterializer>>materializeFrom: [:aFileStream | self newDefault materializeFrom: aFileStream binary] in FLMaterializer class>>materializationFromFileNamed: [anotherBlock value: file] in StandardFileStream class(FileStream class)>>detectFile:do: BlockClosure>>ensure: StandardFileStream class(FileStream class)>>detectFile:do: StandardFileStream class(FileStream class)>>oldFileNamed:do: FLMaterializer class>>materializationFromFileNamed: UndefinedObject>>DoIt

Nov. 25, 2011

Re: [Pharo-project] ProtoObject basicNew :)

by Toon Verwaest

With the magic number I mean the number that defines the object's structure. It's one of the first 3 instance variables of every class, encoding how many instance variables there are; what kind of object it is supposed to be (normal, array, bytes, words...). Every "class" basically needs 3 instance variables. 1 defines the structure, 1 the superclass, 1 the behavior (method dictionary). I don't know the other; but just look at some class to know what the order is. And yes, if you want to inspect the object, you'll need to support whatever protocol your inspector is using. The format is formed as follows: |format | fieldSize sizeHiBits format | fieldSize := self fieldSize + 1. sizeHiBits := fieldSize // 64. format := sizeHiBits. format := (format bitShift: 5) + compactClassIndex. format := (format bitShift: 4) + self instanceSpecification. format := (format bitShift: 6) + (fieldSize \\ 64). format := (format bitShift: 1). ^ format If I read it correctly (just glancing), instance specification specifies if it's arrayed and if it has fields. 0 -> no fields, not arrayed, 1 -> fields, not arrayed, 2 -> arrayed no fields, 3 -> arrayed fields. | HTH, Toon On 11/25/2011 09:31 PM, StÃ©phane Ducasse wrote: > Apparently I can do a doit I was naively doing an inspect ;) > >> Don't you just need the basicNew primitive, and make sure that the "magic number" is set in the object that has the basicNew installed to something that makes sense? > I do not know what is the magic number. > I guess that my problem was that I have first to define what is the minimal behavior. > >> If you want to send messages to the object, probably also best install some dictionary as the method dict instvar. > But I guess that ProtoObject has some behavior since I can browse the class with a browser. > > >> Toon >> >> On 11/25/2011 09:26 PM, StÃ©phane Ducasse wrote: >>> Hi >>> I would like to know the methods that I should define in a subclass of ProtoObject so that I can use basicNew. >>> Does any of you have an idea? >>> >>> Stef >> >

Nov. 25, 2011