Improving thisContext in the Debugger using First Class Variables

Have you ever tried to inspect a thisContext variable in the debugger?

Just interrupt Pharo by "CMD-.", the debugger appears with the main UI loop waiting on a Semaphore
""[delaySemaphore wait] in Delay>>wait".

So if I now write "thisContext" in that block (without saving, I just want to explore), I naively expect this to be the context we are in right now. The debugger even shows that below in the inspector (as "implicit thisContext").

But if I inspect it, I get something completely different (try it, here is a picture):

So what happened?!

When you execute code in the debugger (via e.g. doIt, inspectIt or printIt), we give the code to the compiler to compile a DoIt method and then execute this DoIt method.

With thisContext being a very special PseudoVariable that always just leads to emitting the pushThisContext byte-code, hard-coded (thisContext, self and super are for are for this reason on the "Syntax" Postcard for ST80).

You can see that by inspecting a method of a DoIt that accesses "thisContext", just inspect "thisContext method" and look at the byte-code:

33 <52> pushThisContext
34 <80> send: method
35 <5C> returnTop

And as we execute the DoIt method, the pushThisContext byte-code pushes the context of the method we are executing, which is the DoIt method, not the method you are looking at in the debugger.

Can we do better: First Class Variables to the rescue

In Pharo, all Variables are modelled via a subclass of Variable. This includes the Pseudo Variables, the Variable subclass for 'thisContext' is ThisContextVariable.
names are never hard-coded, instead name-analysis looks up the name in a scope (the block or method that the variable is accessed in) and lookup goes to the outerScope until it reaches
the global scope.

There is of course a reflective API, too. You can send #lookupVar: to any scope or the context, and the system will return the Variable of that name. For example

thisContext lookupVar: #self. 
SmalltalkImage lookupVar: #CompilerClass.
Smalltalk globals lookupVar: #Object.

These are meta-objects describing Variables. As such, the API contains methods to allow the variable to set or return it's value. Depending on the Variable, they need some object the read themselves from.
Globals of course can just answer to #read, Slots have #read: to read from an Object. But all can read from a context using #readInContext:.

(Smalltalk globals lookupVar: #Object) readInContext: thisContext.

Of course, the readInContext: method just uses the other reflective read method after getting the value from the context if needed, e.g. Slots:

readInContext: aContext
	^self read: aContext receiver

or ThisContextVariable:

readInContext: aContext
	^aContext

This idea to have meta-object for Variables that provide a reflective API is very powerful, we use it in all tools: The inspector reads the values
of the instance variables (and thus does not need to send a message like #instVarAt:, nice when you inspect proxies), the Debugger uses this to read temps,
for example in DoIts. To read temps in the debugger that the programmer writes in the code, we have to support even the reading of temps that are actually not accessible in a block.

For this, when we compile code to be executed as a DoIt against a Context (like in the debugger), we use a special scope to lookup variables, the OCContextualDoItSemanticScope. It has
a special version of lookupVar:

OCContextualDoItSemanticScope>>#lookupVar: name

	(targetContext lookupVar: name) ifNotNil: [ :v | ^self importVariable: v].

	^super lookupVar: name

importVariable: aVariable

	^importedVariables
		at: aVariable name
		ifAbsentPut: [ aVariable asDoItVariableFrom: targetContext ]

Thus, it will wrap all variables that you look up in DoItVariable using asDoItVariableFrom:, which for now is implemented for temps (the others just return self):

asDoItVariableFrom: aContext
	^ DoItVariable fromContext: aContext variable: self

The DoItVariable is a decorator: it decorates the original temp with a context, and changes the method that generate code to force the reflective read
even at compile time, e.g for reading:

emitValue: aMethodBuilder
	aMethodBuilder
		pushLiteral: self;
		send: #read

with #read just forwarding the the original Variable:

read
	^actualVariable readInContext: doItContext

Let's fix it

So... what if we would add #asDoItVariableFrom: to ThisContextVariable? It would then, when we compile the DoIt, call #lookupVar: for thisContext, which
would create the decorator and return it. From that point on, the DoItVariable named thisContext shadows the real thisContext, the compiler will ask it
to generate code. That generated code will be the #read which will call #readInContext: on ThisContextVariable, which correctly returns the context.

Let's try, copy, paste, and: YES! It just works. Now thisContext behaves just as we expected.

Nice, and that was just one method added!

Marcus

# Improving thisContext in the Debugger using First Class Variables Have you ever tried to inspect a thisContext variable in the debugger? Just interrupt Pharo by "CMD-.", the debugger appears with the main UI loop waiting on a Semaphore ""[delaySemaphore wait] in Delay>>wait". So if I now write "thisContext" in that block (without saving, I just want to explore), I naively expect this to be the context we are in right now. The debugger even shows that below in the inspector (as "implicit thisContext"). But if I inspect it, I get something completely different (try it, here is a picture): ## So what happened?! When you execute code in the debugger (via e.g. doIt, inspectIt or printIt), we give the code to the compiler to compile a DoIt method and then execute this DoIt method. With thisContext being a very special PseudoVariable that always just leads to emitting the pushThisContext byte-code, hard-coded (thisContext, self and super are for are for this reason on the "Syntax" Postcard for ST80). You can see that by inspecting a method of a DoIt that accesses "thisContext", just inspect "thisContext method" and look at the byte-code: ``` 33 <52> pushThisContext 34 <80> send: method 35 <5C> returnTop ``` And as we execute the DoIt method, the pushThisContext byte-code pushes the context of the method we are executing, which is the DoIt method, not the method you are looking at in the debugger. ## Can we do better: First Class Variables to the rescue In Pharo, all Variables are modelled via a subclass of Variable. This includes the Pseudo Variables, the Variable subclass for 'thisContext' is ThisContextVariable. names are never hard-coded, instead name-analysis looks up the name in a scope (the block or method that the variable is accessed in) and lookup goes to the outerScope until it reaches the global scope. There is of course a reflective API, too. You can send #lookupVar: to any scope or the context, and the system will return the Variable of that name. For example ``` thisContext lookupVar: #self. SmalltalkImage lookupVar: #CompilerClass. Smalltalk globals lookupVar: #Object. ``` These are meta-objects describing Variables. As such, the API contains methods to allow the variable to set or return it's value. Depending on the Variable, they need some object the read themselves from. Globals of course can just answer to #read, Slots have #read: to read from an Object. But all can read from a context using #readInContext:. (Smalltalk globals lookupVar: #Object) readInContext: thisContext. Of course, the readInContext: method just uses the other reflective read method after getting the value from the context if needed, e.g. Slots: ``` readInContext: aContext ^self read: aContext receiver ``` or ThisContextVariable: ``` readInContext: aContext ^aContext ``` This idea to have meta-object for Variables that provide a reflective API is very powerful, we use it in all tools: The inspector reads the values of the instance variables (and thus does not need to send a message like #instVarAt:, nice when you inspect proxies), the Debugger uses this to read temps, for example in DoIts. To read temps in the debugger that the programmer writes in the code, we have to support even the reading of temps that are actually not accessible in a block. For this, when we compile code to be executed as a DoIt against a Context (like in the debugger), we use a special scope to lookup variables, the OCContextualDoItSemanticScope. It has a special version of lookupVar: ``` OCContextualDoItSemanticScope>>#lookupVar: name (targetContext lookupVar: name) ifNotNil: [ :v | ^self importVariable: v]. ^super lookupVar: name importVariable: aVariable ^importedVariables at: aVariable name ifAbsentPut: [ aVariable asDoItVariableFrom: targetContext ] ``` Thus, it will wrap all variables that you look up in DoItVariable using asDoItVariableFrom:, which for now is implemented for temps (the others just return self): ``` asDoItVariableFrom: aContext ^ DoItVariable fromContext: aContext variable: self ``` The DoItVariable is a decorator: it decorates the original temp with a context, and changes the method that generate code to force the reflective read even at compile time, e.g for reading: ``` emitValue: aMethodBuilder aMethodBuilder pushLiteral: self; send: #read ``` with #read just forwarding the the original Variable: ``` read ^actualVariable readInContext: doItContext ``` ## Let's fix it So... what if we would add #asDoItVariableFrom: to ThisContextVariable? It would then, when we compile the DoIt, call #lookupVar: for thisContext, which would create the decorator and return it. From that point on, the DoItVariable named thisContext shadows the real thisContext, the compiler will ask it to generate code. That generated code will be the #read which will call #readInContext: on ThisContextVariable, which correctly returns the context. Let's try, copy, paste, and: YES! It just works. Now thisContext behaves just as we expected. Nice, and that was just one method added! Marcus

sean@clipperadams.com

Fri, May 26, 2023 8:15 PM

Very interesting as usual. Please keep posting these because it is not always obvious all the positive behind-the-scenes effects that new features, like first class variables, bring.