May 2023 - Pharo-dev - lists.pharo.org

Sockets not working under heavy VM load?
by Pavel Krivanek Oct. 3, 2023

Oct. 3, 2023

Hi, we have some performance issues with Glorp over P3 and, after some investigation, I have a theory that could explain it. And I would like to know your opinion if it is correct. To reproduce this issue, you can try something easy. Just create a process with normal or higher priority that will try to read something from a socket. And, in parallel, do in a separate thread an operation that takes all the processor time for several seconds. ```` [ 5 seconds wait. Time now traceCr. ZnClient new url: 'https://www.google.com'; get. Time now traceCr. ] forkAt: 60. [1000 timesRepeat: [ 10000 factorial ] ] timeToRun ```` (you may try a different safe server address) The processes run in parallel until the time when the first process tries to obtain data from the socket. Then it hangs and waits until finishing of the second process. My theory is that the VM socket plugin is able to receive incoming data from the connection and save it into the buffer. In the meantime, the reading process is waiting on the socket semaphore. But it looks like the plugin is not able to release the semaphore while the VM is actually doing something. It needs to wait until the VM has nothing useful to do, and then it is finally able to release the semaphores so Pharo processes are resumed and can start to read. Is my theory right? Cheers, -- Pavel

2 2

GSoC idea: a code generation model for smalltalk
by Nicolas Anquetil Aug. 21, 2023

Aug. 21, 2023

hello, this paper : "An Empirical Comparison of Pre-Trained Models of Source Code" https://arxiv.org/pdf/2302.04026.pdf studied "19 recently-developed pre-trained models of source code" I am a complete noob in this domain, but wouldn't it be possible to use one of these to train it on smalltalk code and get an advanced code completion tool from that ? nicolas

2 2

Reading "self" in the Debugger
by Marcus Denker June 1, 2023

June 1, 2023

In the last post, we looked at reading the PseudoVariable thisContext in the Debugger. A summary of that might be: inspecting a variable compiled a DoIt, that DoIt it executed parametrised with the context that you look at in the debugger. As it is, in the end, just another method executing, the byte-code pushThisContext pushes the context of the executing DoIT method, not the context that you look at in the debugger. For "self", this is of course the same. Take Pharo11, add and insect "thisContext method inspect. self" 41 <52> pushThisContext 42 <80> send: method 43 <81> send: inspect 44 <D8> pop 45 <58> returnSelf Thus even "self" is not really concerned with the context you are looking at, it is the "self" of the DoIt. The nice thing is that this is always (in the debugger) the correct object: In the case of blocks, the only time you look at a block in a debugger to read self is if you are in the home method of the block (the method that contains the definition of the block). And as the "self" of a block closure is the "self" of the home context, you get the value you expect. # Self and Clean Blocks It gets interesting if you put clean blocks into the picture. Clean Blocks are blocks that just use information that is available at compile time. The compiler thus can pre-allocate them. For this discussion, we need to know that clean blocks do not have an outer Context, and self of the block is always nil. Clean block support is there, you can enable it and even re-compile the whole image, but there are still some details to be improved before we can enable it. One of that is to fix all the debugger infrastructure to deal with blocks that have no receiver and no home context, this post thus is one tiny step of that work. For this post, we just need to compile one method with clean blocks enabled, we create a class and method to play with (e.g. class TT with method tt): ``` tt <compilerOptions: #(+ optionCleanBlockClosure)> [ 1halt ] value ``` (or instead of the pragma, enable the setting globally with the Settings browser) With just sending a message to 1, the block can be compiled as a clean block. The halt will mean that the debugger opens on the context of the clean block being executed. Let's do that, execute "TT new tt" and we get a debugger: It is interesting to look at the context, if you inspect the context, you see that the receiver is nil. Looking at the block closure, we see that it is indeed a CleanBlockClosure, outerContext is nil (and thus it can not follow the outerContext till it finds the home context). So if we now just write "self" in the block and expect it (we do not save, we just want to explore), it evaluates to "nil": This is of course correct: The debugger looks at the home context of the block, we ask context for "self" and self in a context evaluating a clean block is nil, as the block does not know self. But I am sure you are not happy with that. It feels wrong. The reason is that if you would accept the method (recompile), restart, then self would be "correct" (the instance of TT). This is because the compiler now sees the self, realizes that this block requires a full block and compiles a full block. This is similar to accessing temps in blocks that do not access them. Change the method to have a temp, but one that is not accessed in the block: ``` tt | temp | temp := 1. [ 1halt ] value. ^temp. ``` If you write "temp" into the block and inspect or print it, you get the value of temp: The context of the executing block does not know the variable "temp" at all, it's value is as unknown to it as the self is to the clean block. (This is true both if compiled as a clean block or a full block). So why can we nevertheless read it? The reason: - Looking up the variable by name of course works (after all, the compiler can compile a block that does access it, the compiler uses the same data for variable lookup as the reflective subsystem does) - The reflective read is carefully build to allow reading this variable even from the block context that does not know it (by leveraging the stack of contexts) - We force the compiler to compile all temp variable accesses as reflective accesses in a DoIt So if we can "fake" it for temps (both clean and non-clean blocks), can we do it for "self in a clean block", too? If you think back at the discussion of reading thisContext, we fixed that by forcing reflective read for thisContext, too. And of course the PR that implements it, was done already for all PseudoVariables. This means, reading self should, in Pharo12, already end up executing SelfVariable>>#readInContext:, so let's just put a halt there, trigger a read in the debugger and, indeed: One thing we now see is that reflective read of self is implemented to do exactly the same as "pushSelf", it returns the receiver of the context we look at in the debugger: ``` readInContext: aContext self halt. ^aContext receiver ``` But that is not really correct: in case of a block, it should be the receiver of the *home* context. For all practical purposes (when in the debugger) they are the same, and for Clean Blocks we even know that the home Context is unknown... So how does this help if we would implement it like that: ``` readInContext: aContext ^aContext home receiver ``` The trick is that we can find a home context for clean blocks in some specific cases. Even though we can not get the home of a clean block via it's outer context (as is is nil), we can find the home method if it happens to be on the stack. And in the debugger, we are in exactly that case! So let's try to fix #home for clean blocks. ## The concept of active Home The Debugger already needs to know if the home is currently on the stack. For that, it used a method #activeHome, in Pharo10 this looked like that: activeHome | methodReturnContext | self isBlockContext ifFalse: [^self]. self sender ifNil: [^nil]. methodReturnContext := self methodReturnContext. ^self sender findContextSuchThat: [:ctxt | ctxt = methodReturnContext] #activeHome returns the #home if it is currently on the stack, the debugger uses that to check if a âsave and proceedâ is possible when editing code in a Block. Until Pharo11, it was implemented to search up the sender chain until it finds the #home. But it can be rewritten to do the same, but checking for the #homeMethod and using #findMethodContextSuchThat: activeHome | homeMethod | self isBlockContext ifFalse: [^self]. homeMethod := self homeMethod. ^self findMethodContextSuchThat: [:ctxt | ctxt method == homeMethod] With #homeMethod being implemented to delegate to the bock: Context>>homeMethod "Answer the method in which the receiver was defined, i.e. the context from which an ^-return ] should return from. Note: implemented to not need #home" ^ closureOrNil ifNil: [ self method ] ifNotNil: [ :closure | closure homeMethod ] Where, if no #home via the outerContext is available, it asks the CompiledBlock: BlockClosure>>homeMethod "return the home method. If no #home is available due to no outerContext, use the compiledBlock" ^ (self home ifNotNil: [ :homeContext | homeContext ] ifNil: [ self compiledBlock ]) method Which uses the static #outerCode chain (CompiledBlocks encode a back-pointer to the enclosing block or method), with #method following #outerCode until it reaches a CompiledMethod: CompiledBlock>>method "answer the compiled method that I am installed in, or nil if none.â ^self outerCode method This was already done in Pharo11, 11965-Implement-ContextactiveHome-without-using-home #12063 https://github.com/pharo-project/pharo/pull/12063 But we can now continue and use it to improve #home of Context, you see that it already has a (bad) workaround when the outerContext is nil: ``` home "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." closureOrNil ifNil: [ ^ self ]. "this happens for clean blocks. We should later check if it is not better to return nil" closureOrNil outerContext ifNil: [ ^ self ]. ^ closureOrNil outerContext home ``` returning self when the outerContext is nil (when we are in a clean block) is only correct for the home context of a clean block. We can do better, and use #activeHome and search for it on the stack (with the added bonus to correctly return nil if we do not find it). ``` home "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." closureOrNil ifNil: [ ^ self ]. "no outerContext for clean blocks, we try to find it on the stack" ^ closureOrNil outerContext ifNil: [ self activeHome ] ifNotNil: [:outer | outer home ] ``` And it works! I am not sure if falling back to #activeHome in #home is really the best (maybe explicitly handling the nil and do the fallback to activeHome at the side of the caller is less magic), but we can change that easily later if it turns out the be better. Another thing to look at is the inspector of the debugger showing self as nil (first line, the implicit self), but his is for another time. Marcus

1 2

Talk next wednesday
by stephane ducasse May 27, 2023

May 27, 2023

Hello I will give a presentation with discussion via the UK smalltalk online gathering next wednesday. https://www.meetup.com/ukstug/events/293473947/ MeetupStephane Ducasse - Pharo: a vision implemented step by step, Wed, M...For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: " For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: "The vision of Pharo is based on three pillars: â¢ First we want to make sure that Pharo is used to develop complex and robust systems (by complex we means multiple millions lines of code or objects). â¢ Second we want Pharo to be a modular system that can be versatile (Pharo on iot, on large servers, in the web browserâ¦.) â¢ Third Pharo should be an evolvable system that can adapt to new needs (modular tools, first class slot, new debuggers, packagesâ¦). Stef

1 0

About thisContext in the Debugger
by Marcus Denker May 26, 2023

May 26, 2023

# Improving thisContext in the Debugger using First Class Variables Have you ever tried to inspect a thisContext variable in the debugger? Just interrupt Pharo by "CMD-.", the debugger appears with the main UI loop waiting on a Semaphore ""[delaySemaphore wait] in Delay>>wait". So if I now write "thisContext" in that block (without saving, I just want to explore), I naively expect this to be the context we are in right now. The debugger even shows that below in the inspector (as "implicit thisContext"). But if I inspect it, I get something completely different (try it, here is a picture): ## So what happened?! When you execute code in the debugger (via e.g. doIt, inspectIt or printIt), we give the code to the compiler to compile a DoIt method and then execute this DoIt method. With thisContext being a very special PseudoVariable that always just leads to emitting the pushThisContext byte-code, hard-coded (thisContext, self and super are for are for this reason on the "Syntax" Postcard for ST80). You can see that by inspecting a method of a DoIt that accesses "thisContext", just inspect "thisContext method" and look at the byte-code: ``` 33 <52> pushThisContext 34 <80> send: method 35 <5C> returnTop ``` And as we execute the DoIt method, the pushThisContext byte-code pushes the context of the method we are executing, which is the DoIt method, not the method you are looking at in the debugger. ## Can we do better: First Class Variables to the rescue In Pharo, all Variables are modelled via a subclass of Variable. This includes the Pseudo Variables, the Variable subclass for 'thisContext' is ThisContextVariable. names are never hard-coded, instead name-analysis looks up the name in a scope (the block or method that the variable is accessed in) and lookup goes to the outerScope until it reaches the global scope. There is of course a reflective API, too. You can send #lookupVar: to any scope or the context, and the system will return the Variable of that name. For example ``` thisContext lookupVar: #self. SmalltalkImage lookupVar: #CompilerClass. Smalltalk globals lookupVar: #Object. ``` These are meta-objects describing Variables. As such, the API contains methods to allow the variable to set or return it's value. Depending on the Variable, they need some object the read themselves from. Globals of course can just answer to #read, Slots have #read: to read from an Object. But all can read from a context using #readInContext:. (Smalltalk globals lookupVar: #Object) readInContext: thisContext. Of course, the readInContext: method just uses the other reflective read method after getting the value from the context if needed, e.g. Slots: ``` readInContext: aContext ^self read: aContext receiver ``` or ThisContextVariable: ``` readInContext: aContext ^aContext ``` This idea to have meta-object for Variables that provide a reflective API is very powerful, we use it in all tools: The inspector reads the values of the instance variables (and thus does not need to send a message like #instVarAt:, nice when you inspect proxies), the Debugger uses this to read temps, for example in DoIts. To read temps in the debugger that the programmer writes in the code, we have to support even the reading of temps that are actually not accessible in a block. For this, when we compile code to be executed as a DoIt against a Context (like in the debugger), we use a special scope to lookup variables, the OCContextualDoItSemanticScope. It has a special version of lookupVar: ``` OCContextualDoItSemanticScope>>#lookupVar: name (targetContext lookupVar: name) ifNotNil: [ :v | ^self importVariable: v]. ^super lookupVar: name importVariable: aVariable ^importedVariables at: aVariable name ifAbsentPut: [ aVariable asDoItVariableFrom: targetContext ] ``` Thus, it will wrap all variables that you look up in DoItVariable using asDoItVariableFrom:, which for now is implemented for temps (the others just return self): ``` asDoItVariableFrom: aContext ^ DoItVariable fromContext: aContext variable: self ``` The DoItVariable is a decorator: it decorates the original temp with a context, and changes the method that generate code to force the reflective read even at compile time, e.g for reading: ``` emitValue: aMethodBuilder aMethodBuilder pushLiteral: self; send: #read ``` with #read just forwarding the the original Variable: ``` read ^actualVariable readInContext: doItContext ``` ## Let's fix it So... what if we would add #asDoItVariableFrom: to ThisContextVariable? It would then, when we compile the DoIt, call #lookupVar: for thisContext, which would create the decorator and return it. From that point on, the DoItVariable named thisContext shadows the real thisContext, the compiler will ask it to generate code. That generated code will be the #read which will call #readInContext: on ThisContextVariable, which correctly returns the context. Let's try, copy, paste, and: YES! It just works. Now thisContext behaves just as we expected. Nice, and that was just one method added! Marcus

2 1

This week (21/2023) on the Pharo Issue Tracker
by Marcus Denker May 26, 2023

May 26, 2023

# This week (21/2023) on the Pharo Issue Tracker. This we we merged all changes done during the last weeks in Pharo11 to Pharo12. ClassOrganization refactoring continues, we will soon be able to inline the class into ClassDescription. # Pharo 11 - Backport fix to calypso method search #13794 https://github.com/pharo-project/pharo/pull/13794 # Fixes - merge Pharo11 changes into Pharo12 #13784 https://github.com/pharo-project/pharo/pull/13784 - Fix reaction of Epicea to protocol removal #13801 https://github.com/pharo-project/pharo/pull/13801 - Migrate variables of VariableLayout only if old instance was also variable #13789 https://github.com/pharo-project/pharo/pull/13789 - Fix #13478: Filter on calypsoName instead of name #13778 https://github.com/pharo-project/pharo/pull/13778 - Fix: RBClassRegexRefactoring #13770 https://github.com/pharo-project/pharo/pull/13770 - Loading Pharo12 branch of Sindarin in BaselineOfNewTools for Pharo 12 #525 https://github.com/pharo-spec/NewTools/pull/525 - Add source folder of Sindarin #530 https://github.com/pharo-spec/NewTools/pull/530 # Protocols / ClassOrganizer - Remove user of #organization in Debugger #531 https://github.com/pharo-spec/NewTools/pull/531 - Method announcements should use real protocols #13730 https://github.com/pharo-project/pharo/pull/13730 - Inline ClassOrganization>>extensionProtocols #13783 https://github.com/pharo-project/pharo/pull/13783 - Reduce users of #organization #13785 https://github.com/pharo-project/pharo/pull/13785 - organization, not organisation #13782 https://github.com/pharo-project/pharo/pull/13782 - Remove dead method ClassDescription>>#zapOrganization #13774 https://github.com/pharo-project/pharo/pull/13774 - Inline ClassOrganization>>#hasProtocol: #13776 https://github.com/pharo-project/pharo/pull/13776 # Tests - Add tests for Calypso's item filter #13786 https://github.com/pharo-project/pharo/pull/13786 - FIx DebugSession test #13811 https://github.com/pharo-project/pharo/pull/13811 - Fix completion tests after p11 synch #13809 https://github.com/pharo-project/pharo/pull/13809 - Improve CoverageCollectorTest assertions #13787 https://github.com/pharo-project/pharo/pull/13787 - fix-testConstantBlockClosure-CleanBlocks #13781 https://github.com/pharo-project/pharo/pull/13781 # Cleanup - Remove unimplemented code StSpotterQuery.class.st #528 https://github.com/pharo-spec/NewTools/pull/528 - Remove Deprecated Package Spec2-ObservableSlot #1396 https://github.com/pharo-spec/Spec/pull/1396 - Merge SHTextStyler and SHRBTextStyler #13756 https://github.com/pharo-project/pharo/pull/13756 - Cleanup dead code in RubShoutStylerDecorator #13771 https://github.com/pharo-project/pharo/pull/13771 - Update StGenericGenerator.class.st #526 https://github.com/pharo-spec/NewTools/pull/526

1 0

Fwd: ð Just scheduled: Stephane Ducasse - Pharo: a vision implemented step by step
by stephane ducasse May 24, 2023

May 24, 2023

Hi guys I will present some points around Pharo and P11 in particular For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: "The vision of Pharo is based on three pillars: - First we want to make sure that Pharo is used to develop complex and robust systems (by complex we means multiple millions lines of code or objects). - Second we want Pharo to be a modular system that can be versatile (Pharo on iot, on large servers, in the web browserâ¦.) - Third Pharo should be an evolvable system that can adapt to new needs (modular tools, first class slot, new debuggers, packagesâ¦). Sometimes it can be unclear that we follow this vision but over the years we are delivering this vision and we will continue. In this talk I will briefly recall the vision behind Pharo and show the achievements so far. I will show that our development is heavily backed by tests. In the second part of the talk I will focus on the current effort to improve the user interface. I will show that the reimplementation of the Spec UI framework is a cornerstone of the future replacement of Morphic and use of GTK. I will also answer questions about Pharo 11 and Pharo 12." > Begin forwarded message: > > From: UK Smalltalk User Group <info(a)email.meetup.com> > Subject: ð Just scheduled: Stephane Ducasse - Pharo: a vision implemented step by step > Date: 24 May 2023 at 01:37:08 CEST > To: stephane.ducasse(a)inria.fr > > > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > UK Smalltalk User Group <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> scheduled a new event > > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Stephane Ducasse - Pharo: a vision implemented step by step <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Wednesday, may 31, 2023 7:00 PM - 9:00 PM > Online event > RSVP now <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Details: For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: "The vision of Pharo is based on three pillars: - First we want to make sure that Pharo is used to develop complex and robust systems (by complex we means multiple millions lin... > Hosted by: > gcorriga > > More events recommended for you: > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Reclaiming Your Power: A Woman's Journey to Healing from Narcissistic Abuse <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Midlife Reinvention For Women Meetup Group > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Meet up for wives of tech entrepreneursÂ <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Wives network of London based tech entrepreneurs > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > Free Property Investment Training - Meet and Greet <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> > FREE Property Investing Training > <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpZmev6DrQOwiRvx…> <http://clicks.meetup.com/ls/click?upn=8Y2vsX5dSU9BIj3r0FDX3kuj5bdG4U9taHSR2…> > You received this message because you are a registered member of Meetup. > Unsubscribe <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…> from this type of email. Manage your settingsÂ <http://clicks.meetup.com/ls/click?upn=8Y2vsX5dSU9BIj3r0FDX3hKPTxoZ-2F4CoTjt…>for all types of email updates. > Visit a your account page <http://clicks.meetup.com/ls/click?upn=8Y2vsX5dSU9BIj3r0FDX3hKPTxoZ-2F4CoTjt…> to change your contact details, privacy settings, and other settings. > Meetup LLC <http://clicks.meetup.com/ls/click?upn=8Y2vsX5dSU9BIj3r0FDX3lSPClZdoQRTM87Zc…>, POB 4668 #37895 NY NY USA 10163 > Read our Privacy Policy <http://clicks.meetup.com/ls/click?upn=XbaZ37larFA-2FuV5MohrYpdrra25MtI4Czod…>.

1 0

The curious case of constant blocks
by Marcus Denker May 22, 2023

May 22, 2023

You might have come across code like this: ``` minHeight "answer the receiver's minHeight" ^ self valueOfProperty: #minHeight ifAbsent: [2] ``` In the case the #minHeight property is not set, it returns 2. Code like this is quite common, another example are empty ifAbsent blocks: ``` someDictonary remove: anObject ifAbsent: [] ``` If we analyse the system, we can easily find all of them. The best is to use the AST for this: ``` allBlocks := Smalltalk globals methods flatCollect: [:method | method ast blockNodes ]. allBlocks size. "86805" nonInlinedBlocks := allBlocks select: [:blockNode | blockNode isInlined not]. nonInlinedBlocks size. "36661" âthe blocks are actually just constant" constantBlocks := nonInlinedBlocks select: [:blockNode | blockNode isConstant]. constantBlocks size. "2572" ``` So there are 2572 constant (literal) blocks. You can inspect constantBlocks to explore them: Constant or empty blocks ([] is just [nil]) do not feel like something to think too much about. After all, they just retunr the literal when you send #value to them. What can be the problem? But: they are blocks, and in a system without clean blocks, they are full blocks, which means they are created at runtime for *every* execution of the [] block. And they are blocks, so there is a CompiledBlock created for each and sending #value will execute that bytecode, with the JIT having to create binary code. For Morph>>#minHeight the bytecode would be: ``` "'49 <4C> self 50 <20> pushConstant: #minHeight 51 <F9 01 00> fullClosure:a CompiledBlock: [2] NumCopied: 0 54 <A2> send: valueOfProperty:ifAbsent: 55 <5C> returnTop'" ``` This is expensive! [2] is the same as 2 (the only thing we can do with the block is to send #value, and we can do that with the literal directly). ``` [ 2 value ] bench. [ [2] value ] bench 218625362.000/25750416.833 "8.490167884188363" ``` So there >factor 8 for "create and evaluate" in difference between the two! This lead to people actually rewriting code to use the literal directly, e.g. we could just change it to ``` minHeight "answer the receiver's minHeight" ^ self valueOfProperty: #minHeight ifAbsent: 2 ``` I am guilty of using this sometimes when optimizing for performance, but it does not feel nice. Yet another rule for performance to think about, and the number of constant blocks that are there shows that this is not how people want to do it. And, most important: it just works for 0 arg constant blocks, as literals undestand #value, but not #value:, #value:value: and so on. So what can we do? The first thing (and I am sure you are thinking about that alreary) is the idea of clean blocks. Clean blocks are blocks that only need (to be created) information that the compiler has statically at compile time. you can look at RBProgramNode>>#isClean and the overrides in RBBlockNode>>#isClean RBVariableNode>>#isClean to see the exact cases, but for this case, all what you need to know is that a constant block, as it accesses nothing, is of course the trivial case of a clean block. If we compile them as clean blocks, we will immediatly move creation to compile time, and runtime property will be the same as using a literal. With the added benefit that constant blocks with arguments are supported, too. But: using "2" instead of [2] is not only faster for *creation*, it is faster when evaluting, too. The reason is that "2 value" sends #value, which executes Object>>#value, which is ``` value ^self ``` Which is a Quick return self method, aka a primitive: ``` self symbolic "'Quick return self'" ``` This is *very fast*. While even as a clean block, we have, for every clean block, it's own method (compiledBlock) that the VM has to execute and thus create code for: self symbolic "'25 <20> pushConstant: 2 26 <5E> blockReturn'" It seems the fact that one is a quick return and the other a push/return is for the JIT not that of a difference, it matters for the interpreter more. But the JIT has to create code for *every* constant block, and #value means executing BlockClosure>>#value, which triggers execution of that compiedBlock. We thus have to execute two methods, not one. And the JIT has to cache all the generated code. So can we do better? It is actually easy to implement a class ConstantBlockClosure, subclass of CleanBlockClosure, that implements all the #value methods to just return the constant value: ``` value ^literal ``` Thus we get the same as with sending #value to the literal directly: we send #value, we execute one method that is a quick return. And the good news: there is #optionConstantBlockClosure in the compiler, and it is enabled by default in Pharo11! The reason why we can turn on Constant Bocks without problem is that they are never on the stack, so we do not need to take care to fix all the tools to know how to deal with them. (Constant Bocks actually do have a CompiledBlock so that the e.g. for âsenders ofâ we check the literals just as if it would be a normal clean block, it is just never executed) If we go back to our method #minHeight, this means the bytecode looks like that: ``` self symbolic "'49 <4C> self 50 <20> pushConstant: #minHeight 51 <21> pushConstant: [2] 52 <A2> send: valueOfProperty:ifAbsent: 53 <5C> returnTop'" ``` Thus, in Pharo11, the execution path of all the >2500 constant blocks end up executing one of the #value methods of ConstantBlockClosure. To get all the exceptions corect when sending e.g. #value ot a 1-arg block, there are subclasses for 1/2/3 args, we do not support 4 arg cleanBlocks for now (there are not many). If you want to check that this really works, go to ConstantBlockClosure>>#value and add a Counter via the Debug menu, it's really called a lot! Marcus

2 3

Woden download from Github error
by Aik-Siong Koh May 20, 2023

May 20, 2023

Hi: I opened Pharo 9.0 - 64bit image. Executed Metacello new baseline: 'WodenEngine'; repository: 'github://woden-engine/woden'; load I got the following error: NotFound: revspec 'master' not found Please advise. Thanks, Aik-Siong Koh

3 4

Fun with CleanBlocks or: Speeding up Dictionary>>#at:
by Marcus Denker May 19, 2023

May 19, 2023

Dictionary lookup is used a lot. Let's look at Dictionary>>#at: at: key "Answer the value associated with the key." ^ self at: key ifAbsent: [self errorKeyNotFound: key] The method looks simple and at a first glance, there are no obvious problems. But if we look at it again, we see that for every execution of #at:, the block [self errorKeyNotFound: key] has to be created. The method stores a CompiledBlock in the literals, and a bytecode "create block" creates the block: ``` (Dictionary>>#at:) symbolic "'41 <4C> self 42 <40> pushTemp: 0 43 <40> pushTemp: 0 44 <F9 00 01> fullClosure:a CompiledBlock: [self errorKeyNotFound: key] NumCopied: 1 47 <A1> send: at:ifAbsent: 48 <5C> returnTop'" ``` This happens at *every* execution of #at:, even though the block that we spend time to create will never be executed at runtime (outside of real errors). If you look at the rest of the code path of #at: in a Dictionary, it is carefully written to avoid block creation by using optimized contructs (ifTrue: and friends). Could clean blocks help? Clean blocks are blocks that only use data that the compiler knows at compile time, thus we can create a CleanBlockClosure (which has the CompiledBlock) and store *that* as a Literal. This then would mean that we could just use as fast pushLiteral: to push the block, no creation at runtime needed. Clean blocks are not yet enabled by default, but we can use a compiler option pragma to enable them just for this method: ``` <compilerOptions: #(+ optionCleanBlockClosure)> ``` The problem is that the block references both "self" and the key to be able to show a nice error message. Referencing either one make the block not clean. Could we change the block to be clean? I guess we could not refer self and inline the method #errorKeyNotFound: (knowing the instance is not that important for logging the error in production, for example). But we do want to know the key that is not found, it really simplifies debugging especially if you have to look at log files. If we closely look at the block: we know a bit more about it and how it is used. We know that if it gets evaluated, that evaluation will *always* happen with the homeContext of the block on the stack, as it is will always be #at:ifAbsent: that evaluates that block. And in that case, there is a trick that we can do: we can use reflection to read the needed values via the stack. You might not be aware, but the debugger needs some quite interesting features from the reflective layer of the system to provide the user experience that you take for granted (and that feels trivial). Imagine a block like that: ``` tt | temp1 temp2 | temp1 := 1. temp2 := 2. self class methods do: [ :each | self halt. each with: temp1 ]. ^ temp2 ``` The block does not reference temp2, so it actually is created as a block that does not know temp2 at all. temp2 is not accessibe from the block or it's context. Yet, when debugging, you want to just be able to write temp2 in the block and eval it (without saving the method), as *if* you would access the temp in this block and recompile, the compiler would compile a different block that would know temp2. So the reflective API of Context (and the infrastructure of reading Variables reflectively), is build in a way that #readVariableNamed: will use the stack to find the value of temps that are not available in the block context, but could be available if they would be referenced statically. And if you think about it, we have just a case like that here, in reverse: if we would use #readVariableNamed: on the context to read the argument, the compiler would not see a read of "key", thus compile a block that does not have that temp available. And if we then read "self" via thisContext, too, the compiler sees a block that it can compile as a clean block: at: key "Answer the value associated with the key." <compilerOptions: #(+ optionCleanBlockClosure)> ^ self at: key ifAbsent: [ "this block is never executed, yet we pay runtime cost to create it if it is a full block. We use instead the reflective API to read the argument and receiver via the stack, making the block clean. We enable cleanBlocks for this method by setting optionCleanBlockClosure as it is not yet enabled by default" KeyNotFound signalFor: (thisContext readVariableNamed: #key) in: thisContext sender receiver "This is equivalent to: self errorKeyNotFound: key" ] Let's look at the printed symbolic bytecode: ``` Dictionary>>#at:) symbolic "'41 <4C> self 42 <40> pushTemp: 0 43 <20> pushConstant: [ "this block is never executed, yet we pay runtime cost to create it if it is a full block. We use instead the reflective API to read the argument and receiver via the stack, making the block clean. We enable cleanBlocks for this method by setting optionCleanBlockClosure as it is not yet enabled by default" KeyNotFound signalFor: (thisContext readVariableNamed: #key) in: thisContext sender receiver "This is equivalent to: self errorKeyNotFound: key" ] 44 <A1> send: at:ifAbsent: 45 <5C> returnTop'" ``` you see that it now uses pushConstant: to push the pre-compiled CleanBlockClosure, which is *much* faster. A very naive benchmark using a fairly large Dictionary (Smalltalk globals): ``` [Smalltalk globals at: #Object] bench 8591996.801/7040335.266 "1.220395972119888" ``` Shows a speedup of ~20%, which is qute a lot! Of course, this is a hack. For one, it just solves this for this one case. And we do not want to add code like that everywhere. And real world impact of speed will be quite limited due to that (it just speeds up Dictionary>#at:) The alternative would be to inline #at:ifAbsent: This inlining, due to the existing subclasses, leads to quite some needed refactorings, and has the same problem of being a solution for this problem in this one place. The real solution in the long term is a JIT that removes blocks like that by inlining automatically for the code it executes, while not changing the image level code at all. This would then solve this for all similar cases everywhere all at once. But the nice aspect of the hack is: it is local (touches just this one method), gives us some speedup now and will be trivially removable when we deploy a better solution in the future. But the code is quite ugly and indeed relies on reflection... which maybe for Dictionary>>#at: is not that nice (e.g. if you want to create a minimal image without reflection) But it is save to add this to your own image, it works in Pharo11 and Pharo12 and I guess even in Pharo10 (not tested there). Marcus

2 1