May 2023 - Pharo-dev - lists.pharo.org

Re: Reading "self" in the Debugger

by Marcus Denker

I did a PR: https://github.com/pharo-project/pharo/pull/13855 differences: - guard for nil in #readInContext - two tests: SelfVariableTest >> testReadInContext [ | var | var := self class lookupVar: #self. self assert: (var readInContext: thisContext) identicalTo: self. "read from a block context" self assert: [(var readInContext: thisContext)] value identicalTo: self ] SelfVariableTest >> testReadInContextClean [ <compilerOptions: #( +optionCleanBlockClosure)> | var block | "if context is one stack we can read" block := [ (thisContext lookupVar: #self) readInContext: thisContext ]. self assert: block value equals: self. "but no chance if not, then it is nil" var := self class lookupVar: #self. block := [ thisContext ]. self assert: (var readInContext: block value ) equals: nil ] > On 29 May 2023, at 10:48, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > In the last post, we looked at reading the PseudoVariable thisContext in the Debugger. A summary of that might be: inspecting a variable compiled a DoIt, that DoIt it executed parametrised with the context that you look at in the debugger. As it is, in the end, just another method executing, the byte-code pushThisContext pushes the context of the executing DoIT method, not the context that you look at in the debugger. > > For "self", this is of course the same. Take Pharo11, add and insect "thisContext method inspect. self" > > 41 <52> pushThisContext > 42 <80> send: method > 43 <81> send: inspect > 44 <D8> pop > 45 <58> returnSelf > > > Thus even "self" is not really concerned with the context you are looking at, it is the "self" of the DoIt. The nice thing is that this is always (in the debugger) the correct object: In the case of blocks, the only time you look at a block in a debugger to read self is if you are in the home method of the block (the method that contains the definition of the block). And as the "self" of a block closure is the "self" of the home context, you get the value you expect. > > > # Self and Clean Blocks > > It gets interesting if you put clean blocks into the picture. Clean Blocks are blocks that just use information that is available at compile time. The compiler thus can pre-allocate them. For this discussion, we need to know that clean blocks do not have an outer Context, and self of the block is always nil. > > Clean block support is there, you can enable it and even re-compile the whole image, but there are still some details to be improved before we can enable it. One of that is to fix all the debugger infrastructure to deal with blocks that have no receiver and no home context, this post thus is one tiny step of that work. > > For this post, we just need to compile one method with clean blocks enabled, we create a class and method to play with (e.g. class TT with method tt): > > > ``` > tt > <compilerOptions: #(+ optionCleanBlockClosure)> > [ 1halt ] value > ``` > > (or instead of the pragma, enable the setting globally with the Settings browser) > > With just sending a message to 1, the block can be compiled as a clean block. The halt will mean that the debugger opens on the context of the clean block being executed. > > Let's do that, execute "TT new tt" and we get a debugger: > > <1CleanBlockDebugger.jpeg> > > > It is interesting to look at the context, if you inspect the context, you see that the receiver is nil. Looking at the block closure, we see that it is indeed a CleanBlockClosure, outerContext is nil (and thus it can not follow the outerContext till it finds the home context). > > So if we now just write "self" in the block and expect it (we do not save, we just want to explore), it evaluates to "nil": > > > <2SelfInClean.jpeg> > This is of course correct: The debugger looks at the home context of the block, we ask context for "self" and self in a context evaluating a clean block is nil, as the block does not know self. > > But I am sure you are not happy with that. It feels wrong. The reason is that if you would accept the method (recompile), restart, then self would be "correct" (the instance of TT). This is because the compiler now sees the self, realizes that this block requires a full block and compiles a full block. > > This is similar to accessing temps in blocks that do not access them. Change the method to have a temp, but one that is not accessed in the block: > > ``` > tt > > | temp | > temp := 1. > [ 1halt ] value. > ^temp. > ``` > > If you write "temp" into the block and inspect or print it, you get the value of temp: > > > <3ReadTemp.jpeg> > The context of the executing block does not know the variable "temp" at all, it's value is as unknown to it as the self is to the clean block. (This is true both if compiled as a clean block or a full block). > > So why can we nevertheless read it? The reason: > > - Looking up the variable by name of course works (after all, the compiler can compile a block that does access it, the compiler uses the same > data for variable lookup as the reflective subsystem does) > - The reflective read is carefully build to allow reading this variable even from the block context that does not know it (by leveraging the stack of contexts) > - We force the compiler to compile all temp variable accesses as reflective accesses in a DoIt > > So if we can "fake" it for temps (both clean and non-clean blocks), can we do it for "self in a clean block", too? > > If you think back at the discussion of reading thisContext, we fixed that by forcing reflective read for thisContext, too. And of course the PR that implements it, was done > already for all PseudoVariables. > > This means, reading self should, in Pharo12, already end up executing SelfVariable>>#readInContext:, so let's just put a halt there, trigger a read in > the debugger and, indeed: > > <4HaltInSelfRead.jpeg> > > > One thing we now see is that reflective read of self is implemented to do exactly the same as "pushSelf", it returns the receiver of the context we look at in the debugger: > > ``` > readInContext: aContext > self halt. > ^aContext receiver > ``` > > > But that is not really correct: in case of a block, it should be the receiver of the *home* context. For all practical purposes (when in the debugger) they are the same, and for Clean Blocks we even know that the home Context is unknown... So how does this help if we would implement it like that: > > ``` > readInContext: aContext > ^aContext home receiver > ``` > > The trick is that we can find a home context for clean blocks in some specific cases. Even though we can not get the home of a clean block via it's outer context (as is is nil), we can find the home method if it happens to be on the stack. And in the debugger, we are in exactly that case! > > So let's try to fix #home for clean blocks. > > ## The concept of active Home > > The Debugger already needs to know if the home is currently on the stack. For that, it used a method #activeHome, in Pharo10 this looked like that: > > > activeHome > | methodReturnContext | > self isBlockContext ifFalse: [^self]. > self sender ifNil: [^nil]. > methodReturnContext := self methodReturnContext. > ^self sender findContextSuchThat: [:ctxt | ctxt = methodReturnContext] > > > #activeHome returns the #home if it is currently on the stack, the debugger uses that to check if a âsave and proceedâ > is possible when editing code in a Block. Until Pharo11, it was implemented to search up the sender chain until it finds the #home. > > But it can be rewritten to do the same, but checking for the #homeMethod and using #findMethodContextSuchThat: > > activeHome > | homeMethod | > self isBlockContext ifFalse: [^self]. > homeMethod := self homeMethod. > ^self findMethodContextSuchThat: [:ctxt | ctxt method == homeMethod] > > > With #homeMethod being implemented to delegate to the bock: > > Context>>homeMethod > "Answer the method in which the receiver was defined, i.e. the context from which an ^-return ] should return from. Note: implemented to not need #home" > > ^ closureOrNil ifNil: [ self method ] ifNotNil: [ :closure | closure homeMethod ] > > > Where, if no #home via the outerContext is available, it asks the CompiledBlock: > > BlockClosure>>homeMethod > "return the home method. If no #home is available due to no outerContext, use the compiledBlock" > ^ (self home > ifNotNil: [ :homeContext | homeContext ] > ifNil: [ self compiledBlock ]) method > > > Which uses the static #outerCode chain (CompiledBlocks encode a back-pointer to the enclosing block or method), > with #method following #outerCode until it reaches a CompiledMethod: > > CompiledBlock>>method > "answer the compiled method that I am installed in, or nil if none.â > ^self outerCode method > > > This was already done in Pharo11, > > 11965-Implement-ContextactiveHome-without-using-home #12063 > https://github.com/pharo-project/pharo/pull/12063 > > > But we can now continue and use it to improve #home of Context, you see that it already has a (bad) workaround when the outerContext is nil: > > > ``` > home > "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." > > closureOrNil ifNil: [ ^ self ]. > "this happens for clean blocks. We should later check if it is not better to return nil" > closureOrNil outerContext ifNil: [ ^ self ]. > ^ closureOrNil outerContext home > ``` > > > returning self when the outerContext is nil (when we are in a clean block) is only correct for the home context of a clean block. > We can do better, and use #activeHome and search for it on the stack (with the added bonus to correctly return nil if we do not find it). > > > ``` > home > "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." > > closureOrNil ifNil: [ ^ self ]. > "no outerContext for clean blocks, we try to find it on the stack" > ^ closureOrNil outerContext > ifNil: [ self activeHome ] > ifNotNil: [:outer | outer home ] > ``` > > And it works! > > <5FixedReadSelf.jpeg> > > I am not sure if falling back to #activeHome in #home is really the best (maybe explicitly handling the nil and do the fallback to activeHome > at the side of the caller is less magic), but we can change that easily later if it turns out the be better. > > Another thing to look at is the inspector of the debugger showing self as nil (first line, the implicit self), but his is for another time. > > Marcus > > >

May 30, 2023

Reading "self" in the Debugger

by Marcus Denker

In the last post, we looked at reading the PseudoVariable thisContext in the Debugger. A summary of that might be: inspecting a variable compiled a DoIt, that DoIt it executed parametrised with the context that you look at in the debugger. As it is, in the end, just another method executing, the byte-code pushThisContext pushes the context of the executing DoIT method, not the context that you look at in the debugger. For "self", this is of course the same. Take Pharo11, add and insect "thisContext method inspect. self" 41 <52> pushThisContext 42 <80> send: method 43 <81> send: inspect 44 <D8> pop 45 <58> returnSelf Thus even "self" is not really concerned with the context you are looking at, it is the "self" of the DoIt. The nice thing is that this is always (in the debugger) the correct object: In the case of blocks, the only time you look at a block in a debugger to read self is if you are in the home method of the block (the method that contains the definition of the block). And as the "self" of a block closure is the "self" of the home context, you get the value you expect. # Self and Clean Blocks It gets interesting if you put clean blocks into the picture. Clean Blocks are blocks that just use information that is available at compile time. The compiler thus can pre-allocate them. For this discussion, we need to know that clean blocks do not have an outer Context, and self of the block is always nil. Clean block support is there, you can enable it and even re-compile the whole image, but there are still some details to be improved before we can enable it. One of that is to fix all the debugger infrastructure to deal with blocks that have no receiver and no home context, this post thus is one tiny step of that work. For this post, we just need to compile one method with clean blocks enabled, we create a class and method to play with (e.g. class TT with method tt): ``` tt <compilerOptions: #(+ optionCleanBlockClosure)> [ 1halt ] value ``` (or instead of the pragma, enable the setting globally with the Settings browser) With just sending a message to 1, the block can be compiled as a clean block. The halt will mean that the debugger opens on the context of the clean block being executed. Let's do that, execute "TT new tt" and we get a debugger: It is interesting to look at the context, if you inspect the context, you see that the receiver is nil. Looking at the block closure, we see that it is indeed a CleanBlockClosure, outerContext is nil (and thus it can not follow the outerContext till it finds the home context). So if we now just write "self" in the block and expect it (we do not save, we just want to explore), it evaluates to "nil": This is of course correct: The debugger looks at the home context of the block, we ask context for "self" and self in a context evaluating a clean block is nil, as the block does not know self. But I am sure you are not happy with that. It feels wrong. The reason is that if you would accept the method (recompile), restart, then self would be "correct" (the instance of TT). This is because the compiler now sees the self, realizes that this block requires a full block and compiles a full block. This is similar to accessing temps in blocks that do not access them. Change the method to have a temp, but one that is not accessed in the block: ``` tt | temp | temp := 1. [ 1halt ] value. ^temp. ``` If you write "temp" into the block and inspect or print it, you get the value of temp: The context of the executing block does not know the variable "temp" at all, it's value is as unknown to it as the self is to the clean block. (This is true both if compiled as a clean block or a full block). So why can we nevertheless read it? The reason: - Looking up the variable by name of course works (after all, the compiler can compile a block that does access it, the compiler uses the same data for variable lookup as the reflective subsystem does) - The reflective read is carefully build to allow reading this variable even from the block context that does not know it (by leveraging the stack of contexts) - We force the compiler to compile all temp variable accesses as reflective accesses in a DoIt So if we can "fake" it for temps (both clean and non-clean blocks), can we do it for "self in a clean block", too? If you think back at the discussion of reading thisContext, we fixed that by forcing reflective read for thisContext, too. And of course the PR that implements it, was done already for all PseudoVariables. This means, reading self should, in Pharo12, already end up executing SelfVariable>>#readInContext:, so let's just put a halt there, trigger a read in the debugger and, indeed: One thing we now see is that reflective read of self is implemented to do exactly the same as "pushSelf", it returns the receiver of the context we look at in the debugger: ``` readInContext: aContext self halt. ^aContext receiver ``` But that is not really correct: in case of a block, it should be the receiver of the *home* context. For all practical purposes (when in the debugger) they are the same, and for Clean Blocks we even know that the home Context is unknown... So how does this help if we would implement it like that: ``` readInContext: aContext ^aContext home receiver ``` The trick is that we can find a home context for clean blocks in some specific cases. Even though we can not get the home of a clean block via it's outer context (as is is nil), we can find the home method if it happens to be on the stack. And in the debugger, we are in exactly that case! So let's try to fix #home for clean blocks. ## The concept of active Home The Debugger already needs to know if the home is currently on the stack. For that, it used a method #activeHome, in Pharo10 this looked like that: activeHome | methodReturnContext | self isBlockContext ifFalse: [^self]. self sender ifNil: [^nil]. methodReturnContext := self methodReturnContext. ^self sender findContextSuchThat: [:ctxt | ctxt = methodReturnContext] #activeHome returns the #home if it is currently on the stack, the debugger uses that to check if a âsave and proceedâ is possible when editing code in a Block. Until Pharo11, it was implemented to search up the sender chain until it finds the #home. But it can be rewritten to do the same, but checking for the #homeMethod and using #findMethodContextSuchThat: activeHome | homeMethod | self isBlockContext ifFalse: [^self]. homeMethod := self homeMethod. ^self findMethodContextSuchThat: [:ctxt | ctxt method == homeMethod] With #homeMethod being implemented to delegate to the bock: Context>>homeMethod "Answer the method in which the receiver was defined, i.e. the context from which an ^-return ] should return from. Note: implemented to not need #home" ^ closureOrNil ifNil: [ self method ] ifNotNil: [ :closure | closure homeMethod ] Where, if no #home via the outerContext is available, it asks the CompiledBlock: BlockClosure>>homeMethod "return the home method. If no #home is available due to no outerContext, use the compiledBlock" ^ (self home ifNotNil: [ :homeContext | homeContext ] ifNil: [ self compiledBlock ]) method Which uses the static #outerCode chain (CompiledBlocks encode a back-pointer to the enclosing block or method), with #method following #outerCode until it reaches a CompiledMethod: CompiledBlock>>method "answer the compiled method that I am installed in, or nil if none.â ^self outerCode method This was already done in Pharo11, 11965-Implement-ContextactiveHome-without-using-home #12063 https://github.com/pharo-project/pharo/pull/12063 But we can now continue and use it to improve #home of Context, you see that it already has a (bad) workaround when the outerContext is nil: ``` home "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." closureOrNil ifNil: [ ^ self ]. "this happens for clean blocks. We should later check if it is not better to return nil" closureOrNil outerContext ifNil: [ ^ self ]. ^ closureOrNil outerContext home ``` returning self when the outerContext is nil (when we are in a clean block) is only correct for the home context of a clean block. We can do better, and use #activeHome and search for it on the stack (with the added bonus to correctly return nil if we do not find it). ``` home "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." closureOrNil ifNil: [ ^ self ]. "no outerContext for clean blocks, we try to find it on the stack" ^ closureOrNil outerContext ifNil: [ self activeHome ] ifNotNil: [:outer | outer home ] ``` And it works! I am not sure if falling back to #activeHome in #home is really the best (maybe explicitly handling the nil and do the fallback to activeHome at the side of the caller is less magic), but we can change that easily later if it turns out the be better. Another thing to look at is the inspector of the debugger showing self as nil (first line, the implicit self), but his is for another time. Marcus

May 29, 2023

Talk next wednesday

by stephane ducasse

Hello I will give a presentation with discussion via the UK smalltalk online gathering next wednesday. https://www.meetup.com/ukstug/events/293473947/ MeetupStephane Ducasse - Pharo: a vision implemented step by step, Wed, M...For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: " For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: "The vision of Pharo is based on three pillars: â¢ First we want to make sure that Pharo is used to develop complex and robust systems (by complex we means multiple millions lines of code or objects). â¢ Second we want Pharo to be a modular system that can be versatile (Pharo on iot, on large servers, in the web browserâ¦.) â¢ Third Pharo should be an evolvable system that can adapt to new needs (modular tools, first class slot, new debuggers, packagesâ¦). Stef

May 27, 2023

Re: About thisContext in the Debugger

by sean＠clipperadams.com

Very interesting as usual. Please keep posting these because it is not always obvious all the positive behind-the-scenes effects that new features, like first class variables, bring.

May 26, 2023

This week (21/2023) on the Pharo Issue Tracker

by Marcus Denker

# This week (21/2023) on the Pharo Issue Tracker. This we we merged all changes done during the last weeks in Pharo11 to Pharo12. ClassOrganization refactoring continues, we will soon be able to inline the class into ClassDescription. # Pharo 11 - Backport fix to calypso method search #13794 https://github.com/pharo-project/pharo/pull/13794 # Fixes - merge Pharo11 changes into Pharo12 #13784 https://github.com/pharo-project/pharo/pull/13784 - Fix reaction of Epicea to protocol removal #13801 https://github.com/pharo-project/pharo/pull/13801 - Migrate variables of VariableLayout only if old instance was also variable #13789 https://github.com/pharo-project/pharo/pull/13789 - Fix #13478: Filter on calypsoName instead of name #13778 https://github.com/pharo-project/pharo/pull/13778 - Fix: RBClassRegexRefactoring #13770 https://github.com/pharo-project/pharo/pull/13770 - Loading Pharo12 branch of Sindarin in BaselineOfNewTools for Pharo 12 #525 https://github.com/pharo-spec/NewTools/pull/525 - Add source folder of Sindarin #530 https://github.com/pharo-spec/NewTools/pull/530 # Protocols / ClassOrganizer - Remove user of #organization in Debugger #531 https://github.com/pharo-spec/NewTools/pull/531 - Method announcements should use real protocols #13730 https://github.com/pharo-project/pharo/pull/13730 - Inline ClassOrganization>>extensionProtocols #13783 https://github.com/pharo-project/pharo/pull/13783 - Reduce users of #organization #13785 https://github.com/pharo-project/pharo/pull/13785 - organization, not organisation #13782 https://github.com/pharo-project/pharo/pull/13782 - Remove dead method ClassDescription>>#zapOrganization #13774 https://github.com/pharo-project/pharo/pull/13774 - Inline ClassOrganization>>#hasProtocol: #13776 https://github.com/pharo-project/pharo/pull/13776 # Tests - Add tests for Calypso's item filter #13786 https://github.com/pharo-project/pharo/pull/13786 - FIx DebugSession test #13811 https://github.com/pharo-project/pharo/pull/13811 - Fix completion tests after p11 synch #13809 https://github.com/pharo-project/pharo/pull/13809 - Improve CoverageCollectorTest assertions #13787 https://github.com/pharo-project/pharo/pull/13787 - fix-testConstantBlockClosure-CleanBlocks #13781 https://github.com/pharo-project/pharo/pull/13781 # Cleanup - Remove unimplemented code StSpotterQuery.class.st #528 https://github.com/pharo-spec/NewTools/pull/528 - Remove Deprecated Package Spec2-ObservableSlot #1396 https://github.com/pharo-spec/Spec/pull/1396 - Merge SHTextStyler and SHRBTextStyler #13756 https://github.com/pharo-project/pharo/pull/13756 - Cleanup dead code in RubShoutStylerDecorator #13771 https://github.com/pharo-project/pharo/pull/13771 - Update StGenericGenerator.class.st #526 https://github.com/pharo-spec/NewTools/pull/526

May 26, 2023

About thisContext in the Debugger

by Marcus Denker

# Improving thisContext in the Debugger using First Class Variables Have you ever tried to inspect a thisContext variable in the debugger? Just interrupt Pharo by "CMD-.", the debugger appears with the main UI loop waiting on a Semaphore ""[delaySemaphore wait] in Delay>>wait". So if I now write "thisContext" in that block (without saving, I just want to explore), I naively expect this to be the context we are in right now. The debugger even shows that below in the inspector (as "implicit thisContext"). But if I inspect it, I get something completely different (try it, here is a picture): ## So what happened?! When you execute code in the debugger (via e.g. doIt, inspectIt or printIt), we give the code to the compiler to compile a DoIt method and then execute this DoIt method. With thisContext being a very special PseudoVariable that always just leads to emitting the pushThisContext byte-code, hard-coded (thisContext, self and super are for are for this reason on the "Syntax" Postcard for ST80). You can see that by inspecting a method of a DoIt that accesses "thisContext", just inspect "thisContext method" and look at the byte-code: ``` 33 <52> pushThisContext 34 <80> send: method 35 <5C> returnTop ``` And as we execute the DoIt method, the pushThisContext byte-code pushes the context of the method we are executing, which is the DoIt method, not the method you are looking at in the debugger. ## Can we do better: First Class Variables to the rescue In Pharo, all Variables are modelled via a subclass of Variable. This includes the Pseudo Variables, the Variable subclass for 'thisContext' is ThisContextVariable. names are never hard-coded, instead name-analysis looks up the name in a scope (the block or method that the variable is accessed in) and lookup goes to the outerScope until it reaches the global scope. There is of course a reflective API, too. You can send #lookupVar: to any scope or the context, and the system will return the Variable of that name. For example ``` thisContext lookupVar: #self. SmalltalkImage lookupVar: #CompilerClass. Smalltalk globals lookupVar: #Object. ``` These are meta-objects describing Variables. As such, the API contains methods to allow the variable to set or return it's value. Depending on the Variable, they need some object the read themselves from. Globals of course can just answer to #read, Slots have #read: to read from an Object. But all can read from a context using #readInContext:. (Smalltalk globals lookupVar: #Object) readInContext: thisContext. Of course, the readInContext: method just uses the other reflective read method after getting the value from the context if needed, e.g. Slots: ``` readInContext: aContext ^self read: aContext receiver ``` or ThisContextVariable: ``` readInContext: aContext ^aContext ``` This idea to have meta-object for Variables that provide a reflective API is very powerful, we use it in all tools: The inspector reads the values of the instance variables (and thus does not need to send a message like #instVarAt:, nice when you inspect proxies), the Debugger uses this to read temps, for example in DoIts. To read temps in the debugger that the programmer writes in the code, we have to support even the reading of temps that are actually not accessible in a block. For this, when we compile code to be executed as a DoIt against a Context (like in the debugger), we use a special scope to lookup variables, the OCContextualDoItSemanticScope. It has a special version of lookupVar: ``` OCContextualDoItSemanticScope>>#lookupVar: name (targetContext lookupVar: name) ifNotNil: [ :v | ^self importVariable: v]. ^super lookupVar: name importVariable: aVariable ^importedVariables at: aVariable name ifAbsentPut: [ aVariable asDoItVariableFrom: targetContext ] ``` Thus, it will wrap all variables that you look up in DoItVariable using asDoItVariableFrom:, which for now is implemented for temps (the others just return self): ``` asDoItVariableFrom: aContext ^ DoItVariable fromContext: aContext variable: self ``` The DoItVariable is a decorator: it decorates the original temp with a context, and changes the method that generate code to force the reflective read even at compile time, e.g for reading: ``` emitValue: aMethodBuilder aMethodBuilder pushLiteral: self; send: #read ``` with #read just forwarding the the original Variable: ``` read ^actualVariable readInContext: doItContext ``` ## Let's fix it So... what if we would add #asDoItVariableFrom: to ThisContextVariable? It would then, when we compile the DoIt, call #lookupVar: for thisContext, which would create the decorator and return it. From that point on, the DoItVariable named thisContext shadows the real thisContext, the compiler will ask it to generate code. That generated code will be the #read which will call #readInContext: on ThisContextVariable, which correctly returns the context. Let's try, copy, paste, and: YES! It just works. Now thisContext behaves just as we expected. Nice, and that was just one method added! Marcus

May 25, 2023

Fwd: ð Just scheduled: Stephane Ducasse - Pharo: a vision implemented step by step

by stephane ducasse

Hi guys I will present some points around Pharo and P11 in particular For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: "The vision of Pharo is based on three pillars: - First we want to make sure that Pharo is used to develop complex and robust systems (by complex we means multiple millions lines of code or objects). - Second we want Pharo to be a modular system that can be versatile (Pharo on iot, on large servers, in the web browserâ¦.) - Third Pharo should be an evolvable system that can adapt to new needs (modular tools, first class slot, new debuggers, packagesâ¦). Sometimes it can be unclear that we follow this vision but over the years we are delivering this vision and we will continue. In this talk I will briefly recall the vision behind Pharo and show the achievements so far. I will show that our development is heavily backed by tests. In the second part of the talk I will focus on the current effort to improve the user interface. I will show that the reimplementation of the Spec UI framework is a cornerstone of the future replacement of Morphic and use of GTK. I will also answer questions about Pharo 11 and Pharo 12." > Begin forwarded message: > > From: UK Smalltalk User Group <info(a)email.meetup.com> > Subject: ð Just scheduled: Stephane Ducasse - Pharo: a vision implemented step by step > Date: 24 May 2023 at 01:37:08 CEST > To: stephane.ducasse(a)inria.fr > > > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > UK Smalltalk User Group <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> scheduled a new event > > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Stephane Ducasse - Pharo: a vision implemented step by step <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Wednesday, may 31, 2023 7:00 PM - 9:00 PM > Online event > RSVP now <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Details: For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: "The vision of Pharo is based on three pillars: - First we want to make sure that Pharo is used to develop complex and robust systems (by complex we means multiple millions lin... > Hosted by: > gcorriga > > More events recommended for you: > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Reclaiming Your Power: A Woman's Journey to Healing from Narcissistic Abuse <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Midlife Reinvention For Women Meetup Group > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Meet up for wives of tech entrepreneursÂ <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Wives network of London based tech entrepreneurs > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Free Property Investment Training - Meet and Greet <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > FREE Property Investing Training > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpZmev6DrQOwiRvx…> <http://clicks.meetup.com/ls/click?upn=8Y2vsX5dSU9BIj3r0FDX3kuj5bdG4U9taHSR2…> > You received this message because you are a registered member of Meetup. > Unsubscribe <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> from this type of email. Manage your settingsÂ <http://clicks.meetup.com/ls/click?upn=8Y2vsX5dSU9BIj3r0FDX3hKPTxoZ-2F4CoTjt…>for all types of email updates. > Visit a your account page <http://clicks.meetup.com/ls/click?upn=8Y2vsX5dSU9BIj3r0FDX3hKPTxoZ-2F4CoTjt…> to change your contact details, privacy settings, and other settings. > Meetup LLC <http://clicks.meetup.com/ls/click?upn=8Y2vsX5dSU9BIj3r0FDX3lSPClZdoQRTM87Zc…>, POB 4668 #37895 NY NY USA 10163 > Read our Privacy Policy <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…>.

May 24, 2023

Re: The curious case of constant blocks

by stephane ducasse

I love the idea of a blog post. I like to be able to read about when I want. S > On 22 May 2023, at 14:12, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > I have put a slightly improved version here, it at the end adds a discussion that the mapping still > works (you can inspect ConstantBlockClosure allSubInstances and it can even show the block highlighted > in the home method). > > https://blog.marcusdenker.de/constant-blocks-in-pharo11 > > I will put this in the Queue for the Pharo Dev blog next. > > Marcus > >> On 20 May 2023, at 11:02, Marcus Denker <marcus.denker(a)inria.fr> wrote: >> >> You might have come across code like this: >> >> ``` >> minHeight >> "answer the receiver's minHeight" >> ^ self >> valueOfProperty: #minHeight >> ifAbsent: [2] >> ``` >> >> In the case the #minHeight property is not set, it returns 2. >> >> Code like this is quite common, another example are empty ifAbsent blocks: >> >> >> ``` >> someDictonary remove: anObject ifAbsent: [] >> ``` >> >> If we analyse the system, we can easily find all of them. The best is to use the AST for this: >> >> ``` >> allBlocks := Smalltalk globals methods flatCollect: [:method | method ast blockNodes ]. >> allBlocks size. "86805" >> >> nonInlinedBlocks := allBlocks select: [:blockNode | blockNode isInlined not]. >> nonInlinedBlocks size. "36661" >> >> âthe blocks are actually just constant" >> constantBlocks := nonInlinedBlocks select: [:blockNode | blockNode isConstant]. >> constantBlocks size. "2572" >> >> ``` >> >> So there are 2572 constant (literal) blocks. You can inspect constantBlocks to explore them: >> >> <Constant.jpeg> >> >> Constant or empty blocks ([] is just [nil]) do not feel like something to think too much about. >> >> After all, they just retunr the literal when you send #value to them. What can be the problem? >> >> But: they are blocks, and in a system without clean blocks, they are full blocks, which means they are created at runtime for *every* execution of the [] block. And they are blocks, so there is a CompiledBlock created for each and sending #value will execute that bytecode, with the JIT having to create binary code. >> >> For Morph>>#minHeight the bytecode would be: >> >> ``` >> "'49 <4C> self >> 50 <20> pushConstant: #minHeight >> 51 <F9 01 00> fullClosure:a CompiledBlock: [2] NumCopied: 0 >> 54 <A2> send: valueOfProperty:ifAbsent: >> 55 <5C> returnTop'" >> ``` >> >> This is expensive! [2] is the same as 2 (the only thing we can do with the block is to send #value, and we can do that with the literal directly). >> >> ``` >> [ 2 value ] bench. >> [ [2] value ] bench >> >> 218625362.000/25750416.833 "8.490167884188363" >> ``` >> >> So there >factor 8 for "create and evaluate" in difference between the two! >> >> This lead to people actually rewriting code to use the literal directly, e.g. we could just change it to >> >> >> ``` >> minHeight >> "answer the receiver's minHeight" >> ^ self >> valueOfProperty: #minHeight >> ifAbsent: 2 >> ``` >> >> I am guilty of using this sometimes when optimizing for performance, but it does not feel nice. Yet another rule for performance to think about, and the number of constant blocks that are there shows that this is not how people want to do it. And, most important: it just works for 0 arg constant blocks, as literals undestand #value, but not #value:, #value:value: and so on. >> >> >> So what can we do? The first thing (and I am sure you are thinking about that alreary) is the idea of clean blocks. Clean blocks are blocks that only need (to be created) information that the compiler has statically at compile time. you can look at RBProgramNode>>#isClean and the overrides in RBBlockNode>>#isClean RBVariableNode>>#isClean to see the exact cases, but for this case, all what you need to know is that a constant block, as it accesses nothing, is of course the trivial case of a clean block. >> >> If we compile them as clean blocks, we will immediatly move creation to compile time, and runtime property will be the same as using a literal. With the added benefit that constant blocks with arguments are supported, too. >> >> But: using "2" instead of [2] is not only faster for *creation*, it is faster when evaluting, too. The reason is that "2 value" sends #value, which executes Object>>#value, which is >> >> ``` >> value >> >> ^self >> ``` >> >> Which is a Quick return self method, aka a primitive: >> >> >> ``` >> self symbolic "'Quick return self'" >> ``` >> >> This is *very fast*. While even as a clean block, we have, for every clean block, it's own method (compiledBlock) that the VM has to execute and thus create >> code for: >> >> self symbolic >> >> "'25 <20> pushConstant: 2 >> 26 <5E> blockReturn'" >> >> >> It seems the fact that one is a quick return and the other a push/return is for the JIT not that of a difference, it matters for the interpreter more. But the JIT has to create code for *every* constant block, and #value means executing BlockClosure>>#value, which triggers execution of that compiedBlock. >> >> We thus have to execute two methods, not one. And the JIT has to cache all the generated code. >> >> So can we do better? It is actually easy to implement a class ConstantBlockClosure, subclass of CleanBlockClosure, that implements all the #value methods to just return >> the constant value: >> >> ``` >> value >> ^literal >> ``` >> >> Thus we get the same as with sending #value to the literal directly: we send #value, we execute one method that is a quick return. >> >> And the good news: there is #optionConstantBlockClosure in the compiler, and it is enabled by default in Pharo11! >> >> The reason why we can turn on Constant Bocks without problem is that they are never on the stack, so we do not need to take care to fix all the tools to know how to deal with them. >> (Constant Bocks actually do have a CompiledBlock so that the e.g. for âsenders ofâ we check the literals just as if it would be a normal clean block, it is just never executed) >> >> If we go back to our method #minHeight, this means the bytecode looks like that: >> >> >> ``` >> self symbolic "'49 <4C> self >> 50 <20> pushConstant: #minHeight >> 51 <21> pushConstant: [2] >> 52 <A2> send: valueOfProperty:ifAbsent: >> 53 <5C> returnTop'" >> ``` >> >> Thus, in Pharo11, the execution path of all the >2500 constant blocks end up executing one of the #value methods of ConstantBlockClosure. To get all the exceptions corect when sending e.g. #value ot a 1-arg block, there are subclasses for 1/2/3 args, we do not support 4 arg cleanBlocks for now (there are not many). >> >> If you want to check that this really works, go to ConstantBlockClosure>>#value and add a Counter via the Debug menu, it's really called a lot! >> >> Marcus >> >> >

May 22, 2023

Re: The curious case of constant blocks

by Marcus Denker

I have put a slightly improved version here, it at the end adds a discussion that the mapping still works (you can inspect ConstantBlockClosure allSubInstances and it can even show the block highlighted in the home method). https://blog.marcusdenker.de/constant-blocks-in-pharo11 I will put this in the Queue for the Pharo Dev blog next. Marcus > On 20 May 2023, at 11:02, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > You might have come across code like this: > > ``` > minHeight > "answer the receiver's minHeight" > ^ self > valueOfProperty: #minHeight > ifAbsent: [2] > ``` > > In the case the #minHeight property is not set, it returns 2. > > Code like this is quite common, another example are empty ifAbsent blocks: > > > ``` > someDictonary remove: anObject ifAbsent: [] > ``` > > If we analyse the system, we can easily find all of them. The best is to use the AST for this: > > ``` > allBlocks := Smalltalk globals methods flatCollect: [:method | method ast blockNodes ]. > allBlocks size. "86805" > > nonInlinedBlocks := allBlocks select: [:blockNode | blockNode isInlined not]. > nonInlinedBlocks size. "36661" > > âthe blocks are actually just constant" > constantBlocks := nonInlinedBlocks select: [:blockNode | blockNode isConstant]. > constantBlocks size. "2572" > > ``` > > So there are 2572 constant (literal) blocks. You can inspect constantBlocks to explore them: > > <Constant.jpeg> > > Constant or empty blocks ([] is just [nil]) do not feel like something to think too much about. > > After all, they just retunr the literal when you send #value to them. What can be the problem? > > But: they are blocks, and in a system without clean blocks, they are full blocks, which means they are created at runtime for *every* execution of the [] block. And they are blocks, so there is a CompiledBlock created for each and sending #value will execute that bytecode, with the JIT having to create binary code. > > For Morph>>#minHeight the bytecode would be: > > ``` > "'49 <4C> self > 50 <20> pushConstant: #minHeight > 51 <F9 01 00> fullClosure:a CompiledBlock: [2] NumCopied: 0 > 54 <A2> send: valueOfProperty:ifAbsent: > 55 <5C> returnTop'" > ``` > > This is expensive! [2] is the same as 2 (the only thing we can do with the block is to send #value, and we can do that with the literal directly). > > ``` > [ 2 value ] bench. > [ [2] value ] bench > > 218625362.000/25750416.833 "8.490167884188363" > ``` > > So there >factor 8 for "create and evaluate" in difference between the two! > > This lead to people actually rewriting code to use the literal directly, e.g. we could just change it to > > > ``` > minHeight > "answer the receiver's minHeight" > ^ self > valueOfProperty: #minHeight > ifAbsent: 2 > ``` > > I am guilty of using this sometimes when optimizing for performance, but it does not feel nice. Yet another rule for performance to think about, and the number of constant blocks that are there shows that this is not how people want to do it. And, most important: it just works for 0 arg constant blocks, as literals undestand #value, but not #value:, #value:value: and so on. > > > So what can we do? The first thing (and I am sure you are thinking about that alreary) is the idea of clean blocks. Clean blocks are blocks that only need (to be created) information that the compiler has statically at compile time. you can look at RBProgramNode>>#isClean and the overrides in RBBlockNode>>#isClean RBVariableNode>>#isClean to see the exact cases, but for this case, all what you need to know is that a constant block, as it accesses nothing, is of course the trivial case of a clean block. > > If we compile them as clean blocks, we will immediatly move creation to compile time, and runtime property will be the same as using a literal. With the added benefit that constant blocks with arguments are supported, too. > > But: using "2" instead of [2] is not only faster for *creation*, it is faster when evaluting, too. The reason is that "2 value" sends #value, which executes Object>>#value, which is > > ``` > value > > ^self > ``` > > Which is a Quick return self method, aka a primitive: > > > ``` > self symbolic "'Quick return self'" > ``` > > This is *very fast*. While even as a clean block, we have, for every clean block, it's own method (compiledBlock) that the VM has to execute and thus create > code for: > > self symbolic > > "'25 <20> pushConstant: 2 > 26 <5E> blockReturn'" > > > It seems the fact that one is a quick return and the other a push/return is for the JIT not that of a difference, it matters for the interpreter more. But the JIT has to create code for *every* constant block, and #value means executing BlockClosure>>#value, which triggers execution of that compiedBlock. > > We thus have to execute two methods, not one. And the JIT has to cache all the generated code. > > So can we do better? It is actually easy to implement a class ConstantBlockClosure, subclass of CleanBlockClosure, that implements all the #value methods to just return > the constant value: > > ``` > value > ^literal > ``` > > Thus we get the same as with sending #value to the literal directly: we send #value, we execute one method that is a quick return. > > And the good news: there is #optionConstantBlockClosure in the compiler, and it is enabled by default in Pharo11! > > The reason why we can turn on Constant Bocks without problem is that they are never on the stack, so we do not need to take care to fix all the tools to know how to deal with them. > (Constant Bocks actually do have a CompiledBlock so that the e.g. for âsenders ofâ we check the literals just as if it would be a normal clean block, it is just never executed) > > If we go back to our method #minHeight, this means the bytecode looks like that: > > > ``` > self symbolic "'49 <4C> self > 50 <20> pushConstant: #minHeight > 51 <21> pushConstant: [2] > 52 <A2> send: valueOfProperty:ifAbsent: > 53 <5C> returnTop'" > ``` > > Thus, in Pharo11, the execution path of all the >2500 constant blocks end up executing one of the #value methods of ConstantBlockClosure. To get all the exceptions corect when sending e.g. #value ot a 1-arg block, there are subclasses for 1/2/3 args, we do not support 4 arg cleanBlocks for now (there are not many). > > If you want to check that this really works, go to ConstantBlockClosure>>#value and add a Counter via the Debug menu, it's really called a lot! > > Marcus > >

May 22, 2023

Re: Woden download from Github error

by Bernardo Ezequiel Contreras

try to report it. https://github.com/woden-engine/woden/issues On Sat, May 20, 2023 at 7:00â¯PM Bernardo Ezequiel Contreras < vonbecmann(a)gmail.com> wrote: > i think there's an error in the baseline of Woden > the project github://ronsaldo/sysmel has no master branch. it has a main > branch. > take a look at the stack trace. > > On Sat, May 20, 2023 at 12:46â¯AM Aik-Siong Koh <askoh(a)askoh.com> wrote: > >> Hi: >> >> I opened Pharo 9.0 - 64bit image. >> Executed >> >> Metacello new >> baseline: 'WodenEngine'; >> repository: 'github://woden-engine/woden'; >> load >> >> I got the following error: >> NotFound: revspec 'master' not found >> >> Please advise. >> Thanks, >> Aik-Siong Koh >> > > > -- > Bernardo E.C. > > Sent from a cheap desktop computer in South America. > -- Bernardo E.C. Sent from a cheap desktop computer in South America.

May 20, 2023