- Pharo-dev - lists.pharo.org

Do not miss ESUG 2023

by stephane ducasse

Just a little reminderâ¦ Do not miss ESUG 2023. http://registration.esug.org/ come and meet the community https://www.youtube.com/watch?v=9p-AkoHjRVk The city is really nice, the food great, super nice conference. Really nice museums. https://esug.github.io/2023-Conference/city2023.html by train â¦ 2:30 hours from Paris 4 hours from Brussels 5 hours from Barcelona 2 hours from Geneves An international airport It is uncertain that we will organize a PharoDays in 2023/2024 so enjoy the fact that you can come to that conference now. S. PS: do not hesitate to contact the ESUG board in case of special situations

June 14, 2023

This week (23/2023) on the Pharo Issue Tracker

by Marcus Denker

ClassOrganization is finally inlined into ClassDescription. Image size is now (with welcome window) below 53MB. We continued to improve the read of pseudo variables in the debugger, read about it here: - Improving reading self in the Debugger for Clean Blocks https://blog.marcusdenker.de/improving-reading-self-in-the-debugger-for-cle… - Reflective read of SuperVariable (and what is âsuperâ anyway?) https://blog.marcusdenker.de/reflective-read-of-supervariable-and-what-is-s… # Features - Add benchmark comparison #13923 https://github.com/pharo-project/pharo/pull/13923 - Show completion details setting #13917 https://github.com/pharo-project/pharo/pull/13917 - Fix Dr Tests Code Coverage: support #ignoreForCoverage Pragma #13915 https://github.com/pharo-project/pharo/pull/13915 - Enable-Fluid-Skip-Tests #13918 https://github.com/pharo-project/pharo/pull/13918 # Fixes - [Pharo12] Error Fueling out SpStyleEnvironmentColorProxyTest instances #1400 https://github.com/pharo-spec/Spec/pull/1400 - fix: exception when opening Spec demo "Object inspector" notebook #1402 https://github.com/pharo-spec/Spec/pull/1402 - Fix-CleanBlocks-StInspectorSelfNode #542 https://github.com/pharo-spec/NewTools/pull/542 - Fix doubleClick on tab and on text selection. #13906 https://github.com/pharo-project/pharo/pull/13906 - ReGlobalVariablesUsageRule should ignore the aliases of deprecated classes #13948 https://github.com/pharo-project/pharo/pull/13948 - Fix missing Finder method #13888 https://github.com/pharo-project/pharo/pull/13888 - ReSelfSentNotImplementedRule and #ignoreNotImplementedSelectors: #13946 https://github.com/pharo-project/pharo/pull/13946 # Variables / Compiler - 3839-New-slot-is-not-initialized-for-an-existing-instance #13950 https://github.com/pharo-project/pharo/pull/13950 - 13938-Changing-a-class-side-Slot-to-a-WriteOnceSlot-renders-a-NonBooleanReceiver-proceed-for-truth #13949 https://github.com/pharo-project/pharo/pull/13949 - Simplify IconStyler variable visiting #13898 https://github.com/pharo-project/pharo/pull/13898 - remove-isReservedVariable-check-from-ReCollectionMessagesToExternalObjectRule #13897 https://github.com/pharo-project/pharo/pull/13897 - Fix reading "super" reflectively: same as reading self #13878 https://github.com/pharo-project/pharo/pull/13878 - Some simple cleanups (lookupVar: category, #home use ifNotNil...) #13882 https://github.com/pharo-project/pharo/pull/13882 - Small compiler API useage fix in StDebuggerActionModel #539 https://github.com/pharo-spec/NewTools/pull/539 - StDebuggerTestFailureContextPredicate #printDescription can use #tempNamed: #537 https://github.com/pharo-spec/NewTools/pull/537 # Protocol / ClassOrganization - Update usage of protocols #1407 https://github.com/pharo-spec/Spec/pull/1407 - Inline ClassOrganization #13861 https://github.com/pharo-project/pharo/pull/13861 - Update more users of #protocol #13936 https://github.com/pharo-project/pharo/pull/13936 - Introduce CompiledMethod>>protocolName and start to use it #13881 https://github.com/pharo-project/pharo/pull/13881 - Deprecate ClassDescription>>#addMethodTag: #13880 https://github.com/pharo-project/pharo/pull/13880 - Use better protocol API #545 https://github.com/pharo-spec/NewTools/pull/545 # RPackage - Some cleanings in RPackage related to protocols #13885 https://github.com/pharo-project/pharo/pull/13885 - Simplify check of category in RPackage #13899 https://github.com/pharo-project/pharo/pull/13899 # Reflectivity - Moved Metalevel management to its own package #13876 https://github.com/pharo-project/pharo/pull/13876 - Reflectivity: thisProcess and remove #rfactiveProcess and #rfeffectiveProcess #13883 https://github.com/pharo-project/pharo/pull/13883 # Refactoring - Refactor: Move methods to class that only uses them #13956 https://github.com/pharo-project/pharo/pull/13956 - Fixes #13908: isTransformation and remove duplicated code for now. #13924 https://github.com/pharo-project/pharo/pull/13924 # Tests / CI - Ensure all tests are loaded #13944 https://github.com/pharo-project/pharo/pull/13944 - Bring back coverage test #13941 https://github.com/pharo-project/pharo/pull/13941 - Remove some leftover packages after tests #13930 https://github.com/pharo-project/pharo/pull/13930 - Fix last inconsistancy in Fluid tests package generation #13926 https://github.com/pharo-project/pharo/pull/13926 - Clean fluid class builder tests #13896 https://github.com/pharo-project/pharo/pull/13896 # Git / Iceberg - openWithSpec -> open #1704 https://github.com/pharo-vcs/iceberg/pull/1704 - Cleanup-DeadCode-MCDefinition #1703 https://github.com/pharo-vcs/iceberg/pull/1703 - remove methods that are the same in the superclass #1702 https://github.com/pharo-vcs/iceberg/pull/1702 - theirVersionOn: has no senders but does send non-existing selectors.â¦ #1701 https://github.com/pharo-vcs/iceberg/pull/1701 # Cleanup - Cleanup-debuggerColor #540 https://github.com/pharo-spec/NewTools/pull/540 - Clean OSWorldRender>>#doActivate #13875 https://github.com/pharo-project/pharo/pull/13875 - clean conditionals #13822 https://github.com/pharo-project/pharo/pull/13822 - Use new class deprecation when possible #13872 https://github.com/pharo-project/pharo/pull/13872 - The decompiler had an extension method on the encoder but the method â¦ #13891 https://github.com/pharo-project/pharo/pull/13891 - Split Finder package between model and UI #13900 https://github.com/pharo-project/pharo/pull/13900 - Introduce strategy design pattern in Finder model #13879 https://github.com/pharo-project/pharo/pull/13879 - 11476-TestAsserter--shouldnt-raise-whoseDescriptionIncludes-description-is-sending-an-unimplemented-messsage #13912 https://github.com/pharo-project/pharo/pull/13912 - tag some tests with #ignoreNotImplementedSelectors: pragma #13914 https://github.com/pharo-project/pharo/pull/13914 - remove two methods duplicated in the hierarchy #13916 https://github.com/pharo-project/pharo/pull/13916 - Reduce references to Smalltalk #13919 https://github.com/pharo-project/pharo/pull/13919 - Fix some typos in the docs #13793 https://github.com/pharo-project/pharo/pull/13793 - Cut dependency of Collection-Strings over SessionManager #13858 https://github.com/pharo-project/pharo/pull/13858 - Deprecate SHRBTextStyler>>#format: #13927 https://github.com/pharo-project/pharo/pull/13927 - Cleanup: VariableNotDeclaredTest and some #ignoreNotImplementedSelectors: #13940 https://github.com/pharo-project/pharo/pull/13940 - Cleanup: methods duplicated in the hierarchy / tag selectors #13933 https://github.com/pharo-project/pharo/pull/13933 - Remove some leftover packages #13931 https://github.com/pharo-project/pharo/pull/13931 - Clean #dependentClasses #13928 https://github.com/pharo-project/pharo/pull/13928 - Remove font logic from Shout since it is all dead code #12950 https://github.com/pharo-project/pharo/pull/12950 - tag some tests methods #ignoreNotImplementedSelectors:: #544 https://github.com/pharo-spec/NewTools/pull/544

June 9, 2023

Reflectively reading "super"

by Marcus Denker

After we improved how thisContext and self is read reflectively, we should look at âsuperâ. Super is always interesting, as , at first, you think that it is some magic object âself but with the class of the superclassâ, or something? Like a âperspective" on that object? But it is much more trivial than that, and we can see that when we look at SuperVariable. This is one nice aspect of reflective implementations: you can just look at the code to understand... So we know that in Pharo âsuperâ is not some hard-coded magic, but just a Variable, with an implementation in class SuperVariable. The Compiler delegates code generation to that object, and after [PR #13812] https://github.com/pharo-project/pharo/pull/13812, reading super in the debugger is using #readInContext: like temporary variables do. So letâs see what super does regarding code generation. ``` SuperVariable>>#emitValue: "super references the receiver, send that follows is a super send (the message lookup starts in the superclass, see OCASTTranslator>>#emitMessageNode:)" methodBuilder pushReceiver ``` Who calls this? is is the OCASTTranslator, the visitor that takes a name-analyzed AST, visits that an calls methods on IRBuilder to create byte-code: ``` visitVariableNode: aVariableNode "Invalid variable should fail" aVariableNode variable isInvalidVariable ifTrue: [ ^ self emitRuntimeError: aVariableNode ]. "Because we are producing a CompiledMethod, register undeclared variables to `Undeclared`" aVariableNode variable isUndeclaredVariable ifTrue: [ aVariableNode variable register ]. aVariableNode variable emitValue: methodBuilder ``` So in the end, the OCASTTranslator just delegates code generation to the Variable. The self variable got set in the name analyzer, just like any other variable by looking it up in the current scope: ``` resolveVariableNode: aVariableNode | var | var := (scope lookupVar: aVariableNode name) ifNil: [ self undeclaredVariable: aVariableNode ]. aVariableNode variable: var. ^var `` lookupVar is sent to the current scope, but if the variable is not defined there, it goes to the parent scope until it reaches âSmalltalk gloablsâ, which (in Pharo) knows all the pseudo variables: ``` SystemDictionary>>#lookupVar: name "Return a var with this name. Return nil if none found" ^self reservedVariables at: name ifAbsent: [self bindingOf: name] ``` (we still call these âReservedVariablesâ, but with this scheme they are not reserved at all, they are just variables of the global scope, just like âObjectâ. Shadowing rules apply (thus we can shadow these variables with a DoItVariable, to force reflective read as explained in https://blog.marcusdenker.de/improving-thiscontext-in-the-debugger-using-fi… ) So how comes super sends are different if super is just read the same as self? The magic happens just for sends, as you can see in OCASTTranslator>>#emitMessageNode: ``` emitMessageNode: aMessageNode aMessageNode isCascaded ifFalse: [ self visitNode: aMessageNode receiver]. aMessageNode arguments do: [:each | self visitNode: each]. aMessageNode isSuperSend ifTrue: [methodBuilder send: aMessageNode selector toSuperOf: aMessageNode superOf ] ifFalse: [methodBuilder send: aMessageNode selector] ``` with: ``` RBMessageNode>>#isSuperSend ^ self receiver isSuperVariable ``` So back to reflective read. As super is just self when reading, we need to implement readInContext: the same as in SelfVariable: ``` readInContext: aContext "super in a block is the receiver of the home context For clean blocks it might not be known (nil)" ^aContext home ifNotNil: [:home | home receiver ] ``` PR is here: https://github.com/pharo-project/pharo/pull/13878

June 2, 2023

This week (22/2023) on the Pharo Issue Tracker.

by Marcus Denker

ClassOrganization refactoring continues, ClassDescription soon can now be inlined (see https://github.com/pharo-project/pharo/pull/13861) One interesting improvement this week is that we fixed reading the pseudovarible "thisContext" in the debugger. Read about it here: https://blog.marcusdenker.de/improving-thiscontext-in-the-debugger-using-fi… # Pharo 11 - [Pharo11] Error Fueling out SpStyleEnvironmentColorProxyTest instances #1399 https://github.com/pharo-spec/Spec/pull/1399 # Improvement: UI Scaling - Fix System Browser tables scaling #13850 https://github.com/pharo-project/pharo/pull/13850 - Fix System Browser scaling #13848 https://github.com/pharo-project/pharo/pull/13848 - Remove unused UITheme forms and add support for scale 2.0 icons #13767 https://github.com/pharo-project/pharo/pull/13767 - Fix for 13767: Remove unused UITheme forms and add support for scale 2.0 icons #13837 https://github.com/pharo-project/pharo/pull/13837 - Add class variable to control scaling in SpFontStyle #1397 https://github.com/pharo-spec/Spec/pull/1397 # Features - Add stack coloring to the debugger #521 https://github.com/pharo-spec/NewTools/pull/521 - FFI: Add +optAutoReleaseReturnedValue option #13229 https://github.com/pharo-project/pharo/pull/13229 - Fix/13741: Print it behavior changed in Calypso #13832 https://github.com/pharo-project/pharo/pull/13832 - Change the bench method to return a BenchmarkResult instance #13823 https://github.com/pharo-project/pharo/pull/13823 - Add #getTemporaryDirectory to UnixPlatform and WinPlatform #13777 https://github.com/pharo-project/pharo/pull/13777 - Add highlight text color to theme for debugger stack coloring #13693 https://github.com/pharo-project/pharo/pull/13693 # Fixes - 13847-Typing-in-Class-definition-pane-in-Calypso-raises-error-at-each-keystroke #13851 https://github.com/pharo-project/pharo/pull/13851 - Fix/13718: Installing a version of a class method adds an instance method #13827 https://github.com/pharo-project/pharo/pull/13827 - Sprint #13819 from uNouss: delete unneeded tools registry redefinition #13835 https://github.com/pharo-project/pharo/pull/13841 - delete unneeded tools registry redefinition #13819 https://github.com/pharo-project/pharo/pull/13819 - improved TempsReadBeforeWrittenRule #13828 https://github.com/pharo-project/pharo/pull/13828 - Run coverage button is enabled with unselected packages ( Dr Tests ) #13836 https://github.com/pharo-project/pharo/pull/13836 # Variables - Fix bug in slot names #13862 https://github.com/pharo-project/pharo/pull/13862 - Compile 'thisContext' (and all other PseudoVariables) as DoitVariable in DoIts #13812 https://github.com/pharo-project/pharo/pull/13812 - 13591-Clean-Blocks-self-is-nil-on-Debugger #13855 https://github.com/pharo-project/pharo/pull/13855 - First step at class name deprecation #13618 https://github.com/pharo-project/pharo/pull/13618 # Compiler and related - Deprecate Context>>#lookupTempVar: #13838 https://github.com/pharo-project/pharo/pull/13838 - Just a tiny change to use the #compiler on Context in more cases #13874 https://github.com/pharo-project/pharo/pull/13874 - Compiler-canAddBindingOf-Check-Synthezised #13864 https://github.com/pharo-project/pharo/pull/13864 - 13769-Image-hangs-when-copying-a-text-from-Transcript #13840 https://github.com/pharo-project/pharo/pull/13840 - 13797-Monticello-can-not-create-the-class-CompileCode-subclasses #13829 https://github.com/pharo-project/pharo/pull/13829 - Move CompiledMethod>> basicNew: and basicNew to CompiledCode #13826 https://github.com/pharo-project/pharo/pull/13826 # CI / Tests - Enhance logs #13815 https://github.com/pharo-project/pharo/pull/13815 - fix variable names in BlockClosureTest #13859 https://github.com/pharo-project/pharo/pull/13859 - Skip testRatioOfCommentedClasses for now #13852 https://github.com/pharo-project/pharo/pull/13852 - skip: testExample #13853 https://github.com/pharo-project/pharo/pull/13853 # Protocol / ClassOrganization - New try at inlining #extensionProtocols #13842 https://github.com/pharo-project/pharo/pull/13842 - Adding some comments to Protocol #13456 https://github.com/pharo-project/pharo/pull/13456 - Finish to remove #organization from protocol tests #13868 https://github.com/pharo-project/pharo/pull/13868 - Remove some organization references #13863 https://github.com/pharo-project/pharo/pull/13863 - Move extensions of ClassOrganization #13856 https://github.com/pharo-project/pharo/pull/13856 - ClassDescription>>protocols should return protocols #13674 https://github.com/pharo-project/pharo/pull/13674 - Clean #extensionCategoriesForClass: #13854 https://github.com/pharo-project/pharo/pull/13854 - Multiple cleaning of Protocol #13305 https://github.com/pharo-project/pharo/pull/13305 - Mulitple cleanups for ClassOrganization inlining #13845 https://github.com/pharo-project/pharo/pull/13845 - Inline ClassOrganization>>classify:under: #13843 https://github.com/pharo-project/pharo/pull/13843 - Remove last #organization from Spec #1405 https://github.com/pharo-spec/Spec/pull/1405 # Epicea (Code Changes Logger) - Remove Epicea catch all #13866 https://github.com/pharo-project/pharo/pull/13866 - Disable Epicea debug mode in tests #13798 https://github.com/pharo-project/pharo/pull/13798 - Fix bug in Epicea management #13857 https://github.com/pharo-project/pharo/pull/13857 - Fix bug in Epicea #13846 https://github.com/pharo-project/pharo/pull/13846 - Inline ClassOrganization>>#protocolNamed:[ifAbsent:] #13792 https://github.com/pharo-project/pharo/pull/13792 # Cleanup / Refactorings - Fix some deprected sends to #method #535 https://github.com/pharo-spec/NewTools/pull/535 - remove #VariableNotDeclared, the class has been deprecated #536 https://github.com/pharo-spec/NewTools/pull/536 - cleanup SpCodePresenter>>#findClassFromSelection #1398 https://github.com/pharo-spec/Spec/pull/1398 - 1350-SpAbstractMorphicAdapter-should-be-made-abstract #1404 https://github.com/pharo-spec/Spec/pull/1404 - Change Object>>#split: definition as recomended by pharo-project/Pharo#13656 #13824 https://github.com/pharo-project/pharo/pull/13824 - Clean Form magnifying #13820 https://github.com/pharo-project/pharo/pull/13820 - Less subclass: instancevariablenames: .... #13834 https://github.com/pharo-project/pharo/pull/13834 - Enh/refac: Factorize code in refactoring transformation #13873 https://github.com/pharo-project/pharo/pull/13873 - Remove FLPharo11Platform #13844 https://github.com/pharo-project/pharo/pull/13844 - Color: little cleaning and optimization #13833 https://github.com/pharo-project/pharo/pull/13833 - Avoid NewUndeclaredWarning when loading RingChunkImporter #13821 https://github.com/pharo-project/pharo/pull/13821 - Remove unecessary cleanup in Gofer #13860 https://github.com/pharo-project/pharo/pull/13860

June 2, 2023

Re: Reading "self" in the Debugger

by Marcus Denker

>> Another thing to look at is the inspector of the debugger showing self as nil (first line, the implicit self), but his is for another time. Here is a PR: Fix-CleanBlocks-StInspectorSelfNode #542 https://github.com/pharo-spec/NewTools/pull/542 <https://github.com/pharo-spec/NewTools/pull/542> > On 30 May 2023, at 17:29, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > I did a PR: > > > https://github.com/pharo-project/pharo/pull/13855 > > > differences: > > - guard for nil in #readInContext > - two tests: > > SelfVariableTest >> testReadInContext [ > > | var | > var := self class lookupVar: #self. > self assert: (var readInContext: thisContext) identicalTo: self. > "read from a block context" > self assert: [(var readInContext: thisContext)] value identicalTo: self > ] > > > SelfVariableTest >> testReadInContextClean [ > <compilerOptions: #( +optionCleanBlockClosure)> > | var block | > "if context is one stack we can read" > block := [ (thisContext lookupVar: #self) readInContext: thisContext ]. > self assert: block value equals: self. > "but no chance if not, then it is nil" > var := self class lookupVar: #self. > block := [ thisContext ]. > self assert: (var readInContext: block value ) equals: nil > ] > >> On 29 May 2023, at 10:48, Marcus Denker <marcus.denker(a)inria.fr> wrote: >> >> In the last post, we looked at reading the PseudoVariable thisContext in the Debugger. A summary of that might be: inspecting a variable compiled a DoIt, that DoIt it executed parametrised with the context that you look at in the debugger. As it is, in the end, just another method executing, the byte-code pushThisContext pushes the context of the executing DoIT method, not the context that you look at in the debugger. >> >> For "self", this is of course the same. Take Pharo11, add and insect "thisContext method inspect. self" >> >> 41 <52> pushThisContext >> 42 <80> send: method >> 43 <81> send: inspect >> 44 <D8> pop >> 45 <58> returnSelf >> >> >> Thus even "self" is not really concerned with the context you are looking at, it is the "self" of the DoIt. The nice thing is that this is always (in the debugger) the correct object: In the case of blocks, the only time you look at a block in a debugger to read self is if you are in the home method of the block (the method that contains the definition of the block). And as the "self" of a block closure is the "self" of the home context, you get the value you expect. >> >> >> # Self and Clean Blocks >> >> It gets interesting if you put clean blocks into the picture. Clean Blocks are blocks that just use information that is available at compile time. The compiler thus can pre-allocate them. For this discussion, we need to know that clean blocks do not have an outer Context, and self of the block is always nil. >> >> Clean block support is there, you can enable it and even re-compile the whole image, but there are still some details to be improved before we can enable it. One of that is to fix all the debugger infrastructure to deal with blocks that have no receiver and no home context, this post thus is one tiny step of that work. >> >> For this post, we just need to compile one method with clean blocks enabled, we create a class and method to play with (e.g. class TT with method tt): >> >> >> ``` >> tt >> <compilerOptions: #(+ optionCleanBlockClosure)> >> [ 1halt ] value >> ``` >> >> (or instead of the pragma, enable the setting globally with the Settings browser) >> >> With just sending a message to 1, the block can be compiled as a clean block. The halt will mean that the debugger opens on the context of the clean block being executed. >> >> Let's do that, execute "TT new tt" and we get a debugger: >> >> <1CleanBlockDebugger.jpeg> >> >> >> It is interesting to look at the context, if you inspect the context, you see that the receiver is nil. Looking at the block closure, we see that it is indeed a CleanBlockClosure, outerContext is nil (and thus it can not follow the outerContext till it finds the home context). >> >> So if we now just write "self" in the block and expect it (we do not save, we just want to explore), it evaluates to "nil": >> >> >> <2SelfInClean.jpeg> >> This is of course correct: The debugger looks at the home context of the block, we ask context for "self" and self in a context evaluating a clean block is nil, as the block does not know self. >> >> But I am sure you are not happy with that. It feels wrong. The reason is that if you would accept the method (recompile), restart, then self would be "correct" (the instance of TT). This is because the compiler now sees the self, realizes that this block requires a full block and compiles a full block. >> >> This is similar to accessing temps in blocks that do not access them. Change the method to have a temp, but one that is not accessed in the block: >> >> ``` >> tt >> >> | temp | >> temp := 1. >> [ 1halt ] value. >> ^temp. >> ``` >> >> If you write "temp" into the block and inspect or print it, you get the value of temp: >> >> >> <3ReadTemp.jpeg> >> The context of the executing block does not know the variable "temp" at all, it's value is as unknown to it as the self is to the clean block. (This is true both if compiled as a clean block or a full block). >> >> So why can we nevertheless read it? The reason: >> >> - Looking up the variable by name of course works (after all, the compiler can compile a block that does access it, the compiler uses the same >> data for variable lookup as the reflective subsystem does) >> - The reflective read is carefully build to allow reading this variable even from the block context that does not know it (by leveraging the stack of contexts) >> - We force the compiler to compile all temp variable accesses as reflective accesses in a DoIt >> >> So if we can "fake" it for temps (both clean and non-clean blocks), can we do it for "self in a clean block", too? >> >> If you think back at the discussion of reading thisContext, we fixed that by forcing reflective read for thisContext, too. And of course the PR that implements it, was done >> already for all PseudoVariables. >> >> This means, reading self should, in Pharo12, already end up executing SelfVariable>>#readInContext:, so let's just put a halt there, trigger a read in >> the debugger and, indeed: >> >> <4HaltInSelfRead.jpeg> >> >> >> One thing we now see is that reflective read of self is implemented to do exactly the same as "pushSelf", it returns the receiver of the context we look at in the debugger: >> >> ``` >> readInContext: aContext >> self halt. >> ^aContext receiver >> ``` >> >> >> But that is not really correct: in case of a block, it should be the receiver of the *home* context. For all practical purposes (when in the debugger) they are the same, and for Clean Blocks we even know that the home Context is unknown... So how does this help if we would implement it like that: >> >> ``` >> readInContext: aContext >> ^aContext home receiver >> ``` >> >> The trick is that we can find a home context for clean blocks in some specific cases. Even though we can not get the home of a clean block via it's outer context (as is is nil), we can find the home method if it happens to be on the stack. And in the debugger, we are in exactly that case! >> >> So let's try to fix #home for clean blocks. >> >> ## The concept of active Home >> >> The Debugger already needs to know if the home is currently on the stack. For that, it used a method #activeHome, in Pharo10 this looked like that: >> >> >> activeHome >> | methodReturnContext | >> self isBlockContext ifFalse: [^self]. >> self sender ifNil: [^nil]. >> methodReturnContext := self methodReturnContext. >> ^self sender findContextSuchThat: [:ctxt | ctxt = methodReturnContext] >> >> >> #activeHome returns the #home if it is currently on the stack, the debugger uses that to check if a âsave and proceedâ >> is possible when editing code in a Block. Until Pharo11, it was implemented to search up the sender chain until it finds the #home. >> >> But it can be rewritten to do the same, but checking for the #homeMethod and using #findMethodContextSuchThat: >> >> activeHome >> | homeMethod | >> self isBlockContext ifFalse: [^self]. >> homeMethod := self homeMethod. >> ^self findMethodContextSuchThat: [:ctxt | ctxt method == homeMethod] >> >> >> With #homeMethod being implemented to delegate to the bock: >> >> Context>>homeMethod >> "Answer the method in which the receiver was defined, i.e. the context from which an ^-return ] should return from. Note: implemented to not need #home" >> >> ^ closureOrNil ifNil: [ self method ] ifNotNil: [ :closure | closure homeMethod ] >> >> >> Where, if no #home via the outerContext is available, it asks the CompiledBlock: >> >> BlockClosure>>homeMethod >> "return the home method. If no #home is available due to no outerContext, use the compiledBlock" >> ^ (self home >> ifNotNil: [ :homeContext | homeContext ] >> ifNil: [ self compiledBlock ]) method >> >> >> Which uses the static #outerCode chain (CompiledBlocks encode a back-pointer to the enclosing block or method), >> with #method following #outerCode until it reaches a CompiledMethod: >> >> CompiledBlock>>method >> "answer the compiled method that I am installed in, or nil if none.â >> ^self outerCode method >> >> >> This was already done in Pharo11, >> >> 11965-Implement-ContextactiveHome-without-using-home #12063 >> https://github.com/pharo-project/pharo/pull/12063 >> >> >> But we can now continue and use it to improve #home of Context, you see that it already has a (bad) workaround when the outerContext is nil: >> >> >> ``` >> home >> "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." >> >> closureOrNil ifNil: [ ^ self ]. >> "this happens for clean blocks. We should later check if it is not better to return nil" >> closureOrNil outerContext ifNil: [ ^ self ]. >> ^ closureOrNil outerContext home >> ``` >> >> >> returning self when the outerContext is nil (when we are in a clean block) is only correct for the home context of a clean block. >> We can do better, and use #activeHome and search for it on the stack (with the added bonus to correctly return nil if we do not find it). >> >> >> ``` >> home >> "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." >> >> closureOrNil ifNil: [ ^ self ]. >> "no outerContext for clean blocks, we try to find it on the stack" >> ^ closureOrNil outerContext >> ifNil: [ self activeHome ] >> ifNotNil: [:outer | outer home ] >> ``` >> >> And it works! >> >> <5FixedReadSelf.jpeg> >> >> I am not sure if falling back to #activeHome in #home is really the best (maybe explicitly handling the nil and do the fallback to activeHome >> at the side of the caller is less magic), but we can change that easily later if it turns out the be better. >> >> Another thing to look at is the inspector of the debugger showing self as nil (first line, the implicit self), but his is for another time. >> >> Marcus >> >> >> >

June 1, 2023

Re: Reading "self" in the Debugger

by Marcus Denker

I did a PR: https://github.com/pharo-project/pharo/pull/13855 differences: - guard for nil in #readInContext - two tests: SelfVariableTest >> testReadInContext [ | var | var := self class lookupVar: #self. self assert: (var readInContext: thisContext) identicalTo: self. "read from a block context" self assert: [(var readInContext: thisContext)] value identicalTo: self ] SelfVariableTest >> testReadInContextClean [ <compilerOptions: #( +optionCleanBlockClosure)> | var block | "if context is one stack we can read" block := [ (thisContext lookupVar: #self) readInContext: thisContext ]. self assert: block value equals: self. "but no chance if not, then it is nil" var := self class lookupVar: #self. block := [ thisContext ]. self assert: (var readInContext: block value ) equals: nil ] > On 29 May 2023, at 10:48, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > In the last post, we looked at reading the PseudoVariable thisContext in the Debugger. A summary of that might be: inspecting a variable compiled a DoIt, that DoIt it executed parametrised with the context that you look at in the debugger. As it is, in the end, just another method executing, the byte-code pushThisContext pushes the context of the executing DoIT method, not the context that you look at in the debugger. > > For "self", this is of course the same. Take Pharo11, add and insect "thisContext method inspect. self" > > 41 <52> pushThisContext > 42 <80> send: method > 43 <81> send: inspect > 44 <D8> pop > 45 <58> returnSelf > > > Thus even "self" is not really concerned with the context you are looking at, it is the "self" of the DoIt. The nice thing is that this is always (in the debugger) the correct object: In the case of blocks, the only time you look at a block in a debugger to read self is if you are in the home method of the block (the method that contains the definition of the block). And as the "self" of a block closure is the "self" of the home context, you get the value you expect. > > > # Self and Clean Blocks > > It gets interesting if you put clean blocks into the picture. Clean Blocks are blocks that just use information that is available at compile time. The compiler thus can pre-allocate them. For this discussion, we need to know that clean blocks do not have an outer Context, and self of the block is always nil. > > Clean block support is there, you can enable it and even re-compile the whole image, but there are still some details to be improved before we can enable it. One of that is to fix all the debugger infrastructure to deal with blocks that have no receiver and no home context, this post thus is one tiny step of that work. > > For this post, we just need to compile one method with clean blocks enabled, we create a class and method to play with (e.g. class TT with method tt): > > > ``` > tt > <compilerOptions: #(+ optionCleanBlockClosure)> > [ 1halt ] value > ``` > > (or instead of the pragma, enable the setting globally with the Settings browser) > > With just sending a message to 1, the block can be compiled as a clean block. The halt will mean that the debugger opens on the context of the clean block being executed. > > Let's do that, execute "TT new tt" and we get a debugger: > > <1CleanBlockDebugger.jpeg> > > > It is interesting to look at the context, if you inspect the context, you see that the receiver is nil. Looking at the block closure, we see that it is indeed a CleanBlockClosure, outerContext is nil (and thus it can not follow the outerContext till it finds the home context). > > So if we now just write "self" in the block and expect it (we do not save, we just want to explore), it evaluates to "nil": > > > <2SelfInClean.jpeg> > This is of course correct: The debugger looks at the home context of the block, we ask context for "self" and self in a context evaluating a clean block is nil, as the block does not know self. > > But I am sure you are not happy with that. It feels wrong. The reason is that if you would accept the method (recompile), restart, then self would be "correct" (the instance of TT). This is because the compiler now sees the self, realizes that this block requires a full block and compiles a full block. > > This is similar to accessing temps in blocks that do not access them. Change the method to have a temp, but one that is not accessed in the block: > > ``` > tt > > | temp | > temp := 1. > [ 1halt ] value. > ^temp. > ``` > > If you write "temp" into the block and inspect or print it, you get the value of temp: > > > <3ReadTemp.jpeg> > The context of the executing block does not know the variable "temp" at all, it's value is as unknown to it as the self is to the clean block. (This is true both if compiled as a clean block or a full block). > > So why can we nevertheless read it? The reason: > > - Looking up the variable by name of course works (after all, the compiler can compile a block that does access it, the compiler uses the same > data for variable lookup as the reflective subsystem does) > - The reflective read is carefully build to allow reading this variable even from the block context that does not know it (by leveraging the stack of contexts) > - We force the compiler to compile all temp variable accesses as reflective accesses in a DoIt > > So if we can "fake" it for temps (both clean and non-clean blocks), can we do it for "self in a clean block", too? > > If you think back at the discussion of reading thisContext, we fixed that by forcing reflective read for thisContext, too. And of course the PR that implements it, was done > already for all PseudoVariables. > > This means, reading self should, in Pharo12, already end up executing SelfVariable>>#readInContext:, so let's just put a halt there, trigger a read in > the debugger and, indeed: > > <4HaltInSelfRead.jpeg> > > > One thing we now see is that reflective read of self is implemented to do exactly the same as "pushSelf", it returns the receiver of the context we look at in the debugger: > > ``` > readInContext: aContext > self halt. > ^aContext receiver > ``` > > > But that is not really correct: in case of a block, it should be the receiver of the *home* context. For all practical purposes (when in the debugger) they are the same, and for Clean Blocks we even know that the home Context is unknown... So how does this help if we would implement it like that: > > ``` > readInContext: aContext > ^aContext home receiver > ``` > > The trick is that we can find a home context for clean blocks in some specific cases. Even though we can not get the home of a clean block via it's outer context (as is is nil), we can find the home method if it happens to be on the stack. And in the debugger, we are in exactly that case! > > So let's try to fix #home for clean blocks. > > ## The concept of active Home > > The Debugger already needs to know if the home is currently on the stack. For that, it used a method #activeHome, in Pharo10 this looked like that: > > > activeHome > | methodReturnContext | > self isBlockContext ifFalse: [^self]. > self sender ifNil: [^nil]. > methodReturnContext := self methodReturnContext. > ^self sender findContextSuchThat: [:ctxt | ctxt = methodReturnContext] > > > #activeHome returns the #home if it is currently on the stack, the debugger uses that to check if a âsave and proceedâ > is possible when editing code in a Block. Until Pharo11, it was implemented to search up the sender chain until it finds the #home. > > But it can be rewritten to do the same, but checking for the #homeMethod and using #findMethodContextSuchThat: > > activeHome > | homeMethod | > self isBlockContext ifFalse: [^self]. > homeMethod := self homeMethod. > ^self findMethodContextSuchThat: [:ctxt | ctxt method == homeMethod] > > > With #homeMethod being implemented to delegate to the bock: > > Context>>homeMethod > "Answer the method in which the receiver was defined, i.e. the context from which an ^-return ] should return from. Note: implemented to not need #home" > > ^ closureOrNil ifNil: [ self method ] ifNotNil: [ :closure | closure homeMethod ] > > > Where, if no #home via the outerContext is available, it asks the CompiledBlock: > > BlockClosure>>homeMethod > "return the home method. If no #home is available due to no outerContext, use the compiledBlock" > ^ (self home > ifNotNil: [ :homeContext | homeContext ] > ifNil: [ self compiledBlock ]) method > > > Which uses the static #outerCode chain (CompiledBlocks encode a back-pointer to the enclosing block or method), > with #method following #outerCode until it reaches a CompiledMethod: > > CompiledBlock>>method > "answer the compiled method that I am installed in, or nil if none.â > ^self outerCode method > > > This was already done in Pharo11, > > 11965-Implement-ContextactiveHome-without-using-home #12063 > https://github.com/pharo-project/pharo/pull/12063 > > > But we can now continue and use it to improve #home of Context, you see that it already has a (bad) workaround when the outerContext is nil: > > > ``` > home > "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." > > closureOrNil ifNil: [ ^ self ]. > "this happens for clean blocks. We should later check if it is not better to return nil" > closureOrNil outerContext ifNil: [ ^ self ]. > ^ closureOrNil outerContext home > ``` > > > returning self when the outerContext is nil (when we are in a clean block) is only correct for the home context of a clean block. > We can do better, and use #activeHome and search for it on the stack (with the added bonus to correctly return nil if we do not find it). > > > ``` > home > "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." > > closureOrNil ifNil: [ ^ self ]. > "no outerContext for clean blocks, we try to find it on the stack" > ^ closureOrNil outerContext > ifNil: [ self activeHome ] > ifNotNil: [:outer | outer home ] > ``` > > And it works! > > <5FixedReadSelf.jpeg> > > I am not sure if falling back to #activeHome in #home is really the best (maybe explicitly handling the nil and do the fallback to activeHome > at the side of the caller is less magic), but we can change that easily later if it turns out the be better. > > Another thing to look at is the inspector of the debugger showing self as nil (first line, the implicit self), but his is for another time. > > Marcus > > >

May 30, 2023

Reading "self" in the Debugger

by Marcus Denker

In the last post, we looked at reading the PseudoVariable thisContext in the Debugger. A summary of that might be: inspecting a variable compiled a DoIt, that DoIt it executed parametrised with the context that you look at in the debugger. As it is, in the end, just another method executing, the byte-code pushThisContext pushes the context of the executing DoIT method, not the context that you look at in the debugger. For "self", this is of course the same. Take Pharo11, add and insect "thisContext method inspect. self" 41 <52> pushThisContext 42 <80> send: method 43 <81> send: inspect 44 <D8> pop 45 <58> returnSelf Thus even "self" is not really concerned with the context you are looking at, it is the "self" of the DoIt. The nice thing is that this is always (in the debugger) the correct object: In the case of blocks, the only time you look at a block in a debugger to read self is if you are in the home method of the block (the method that contains the definition of the block). And as the "self" of a block closure is the "self" of the home context, you get the value you expect. # Self and Clean Blocks It gets interesting if you put clean blocks into the picture. Clean Blocks are blocks that just use information that is available at compile time. The compiler thus can pre-allocate them. For this discussion, we need to know that clean blocks do not have an outer Context, and self of the block is always nil. Clean block support is there, you can enable it and even re-compile the whole image, but there are still some details to be improved before we can enable it. One of that is to fix all the debugger infrastructure to deal with blocks that have no receiver and no home context, this post thus is one tiny step of that work. For this post, we just need to compile one method with clean blocks enabled, we create a class and method to play with (e.g. class TT with method tt): ``` tt <compilerOptions: #(+ optionCleanBlockClosure)> [ 1halt ] value ``` (or instead of the pragma, enable the setting globally with the Settings browser) With just sending a message to 1, the block can be compiled as a clean block. The halt will mean that the debugger opens on the context of the clean block being executed. Let's do that, execute "TT new tt" and we get a debugger: It is interesting to look at the context, if you inspect the context, you see that the receiver is nil. Looking at the block closure, we see that it is indeed a CleanBlockClosure, outerContext is nil (and thus it can not follow the outerContext till it finds the home context). So if we now just write "self" in the block and expect it (we do not save, we just want to explore), it evaluates to "nil": This is of course correct: The debugger looks at the home context of the block, we ask context for "self" and self in a context evaluating a clean block is nil, as the block does not know self. But I am sure you are not happy with that. It feels wrong. The reason is that if you would accept the method (recompile), restart, then self would be "correct" (the instance of TT). This is because the compiler now sees the self, realizes that this block requires a full block and compiles a full block. This is similar to accessing temps in blocks that do not access them. Change the method to have a temp, but one that is not accessed in the block: ``` tt | temp | temp := 1. [ 1halt ] value. ^temp. ``` If you write "temp" into the block and inspect or print it, you get the value of temp: The context of the executing block does not know the variable "temp" at all, it's value is as unknown to it as the self is to the clean block. (This is true both if compiled as a clean block or a full block). So why can we nevertheless read it? The reason: - Looking up the variable by name of course works (after all, the compiler can compile a block that does access it, the compiler uses the same data for variable lookup as the reflective subsystem does) - The reflective read is carefully build to allow reading this variable even from the block context that does not know it (by leveraging the stack of contexts) - We force the compiler to compile all temp variable accesses as reflective accesses in a DoIt So if we can "fake" it for temps (both clean and non-clean blocks), can we do it for "self in a clean block", too? If you think back at the discussion of reading thisContext, we fixed that by forcing reflective read for thisContext, too. And of course the PR that implements it, was done already for all PseudoVariables. This means, reading self should, in Pharo12, already end up executing SelfVariable>>#readInContext:, so let's just put a halt there, trigger a read in the debugger and, indeed: One thing we now see is that reflective read of self is implemented to do exactly the same as "pushSelf", it returns the receiver of the context we look at in the debugger: ``` readInContext: aContext self halt. ^aContext receiver ``` But that is not really correct: in case of a block, it should be the receiver of the *home* context. For all practical purposes (when in the debugger) they are the same, and for Clean Blocks we even know that the home Context is unknown... So how does this help if we would implement it like that: ``` readInContext: aContext ^aContext home receiver ``` The trick is that we can find a home context for clean blocks in some specific cases. Even though we can not get the home of a clean block via it's outer context (as is is nil), we can find the home method if it happens to be on the stack. And in the debugger, we are in exactly that case! So let's try to fix #home for clean blocks. ## The concept of active Home The Debugger already needs to know if the home is currently on the stack. For that, it used a method #activeHome, in Pharo10 this looked like that: activeHome | methodReturnContext | self isBlockContext ifFalse: [^self]. self sender ifNil: [^nil]. methodReturnContext := self methodReturnContext. ^self sender findContextSuchThat: [:ctxt | ctxt = methodReturnContext] #activeHome returns the #home if it is currently on the stack, the debugger uses that to check if a âsave and proceedâ is possible when editing code in a Block. Until Pharo11, it was implemented to search up the sender chain until it finds the #home. But it can be rewritten to do the same, but checking for the #homeMethod and using #findMethodContextSuchThat: activeHome | homeMethod | self isBlockContext ifFalse: [^self]. homeMethod := self homeMethod. ^self findMethodContextSuchThat: [:ctxt | ctxt method == homeMethod] With #homeMethod being implemented to delegate to the bock: Context>>homeMethod "Answer the method in which the receiver was defined, i.e. the context from which an ^-return ] should return from. Note: implemented to not need #home" ^ closureOrNil ifNil: [ self method ] ifNotNil: [ :closure | closure homeMethod ] Where, if no #home via the outerContext is available, it asks the CompiledBlock: BlockClosure>>homeMethod "return the home method. If no #home is available due to no outerContext, use the compiledBlock" ^ (self home ifNotNil: [ :homeContext | homeContext ] ifNil: [ self compiledBlock ]) method Which uses the static #outerCode chain (CompiledBlocks encode a back-pointer to the enclosing block or method), with #method following #outerCode until it reaches a CompiledMethod: CompiledBlock>>method "answer the compiled method that I am installed in, or nil if none.â ^self outerCode method This was already done in Pharo11, 11965-Implement-ContextactiveHome-without-using-home #12063 https://github.com/pharo-project/pharo/pull/12063 But we can now continue and use it to improve #home of Context, you see that it already has a (bad) workaround when the outerContext is nil: ``` home "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." closureOrNil ifNil: [ ^ self ]. "this happens for clean blocks. We should later check if it is not better to return nil" closureOrNil outerContext ifNil: [ ^ self ]. ^ closureOrNil outerContext home ``` returning self when the outerContext is nil (when we are in a clean block) is only correct for the home context of a clean block. We can do better, and use #activeHome and search for it on the stack (with the added bonus to correctly return nil if we do not find it). ``` home "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from." closureOrNil ifNil: [ ^ self ]. "no outerContext for clean blocks, we try to find it on the stack" ^ closureOrNil outerContext ifNil: [ self activeHome ] ifNotNil: [:outer | outer home ] ``` And it works! I am not sure if falling back to #activeHome in #home is really the best (maybe explicitly handling the nil and do the fallback to activeHome at the side of the caller is less magic), but we can change that easily later if it turns out the be better. Another thing to look at is the inspector of the debugger showing self as nil (first line, the implicit self), but his is for another time. Marcus

May 29, 2023

Talk next wednesday

by stephane ducasse

Hello I will give a presentation with discussion via the UK smalltalk online gathering next wednesday. https://www.meetup.com/ukstug/events/293473947/ MeetupStephane Ducasse - Pharo: a vision implemented step by step, Wed, M...For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: " For our May presentation, Stephane Ducasse will present the vision behind Pharo and how that is been implemented incrementally across multiple releases. In Stef's words: "The vision of Pharo is based on three pillars: â¢ First we want to make sure that Pharo is used to develop complex and robust systems (by complex we means multiple millions lines of code or objects). â¢ Second we want Pharo to be a modular system that can be versatile (Pharo on iot, on large servers, in the web browserâ¦.) â¢ Third Pharo should be an evolvable system that can adapt to new needs (modular tools, first class slot, new debuggers, packagesâ¦). Stef

May 27, 2023

Re: About thisContext in the Debugger

by sean＠clipperadams.com

Very interesting as usual. Please keep posting these because it is not always obvious all the positive behind-the-scenes effects that new features, like first class variables, bring.

May 26, 2023

This week (21/2023) on the Pharo Issue Tracker

by Marcus Denker

# This week (21/2023) on the Pharo Issue Tracker. This we we merged all changes done during the last weeks in Pharo11 to Pharo12. ClassOrganization refactoring continues, we will soon be able to inline the class into ClassDescription. # Pharo 11 - Backport fix to calypso method search #13794 https://github.com/pharo-project/pharo/pull/13794 # Fixes - merge Pharo11 changes into Pharo12 #13784 https://github.com/pharo-project/pharo/pull/13784 - Fix reaction of Epicea to protocol removal #13801 https://github.com/pharo-project/pharo/pull/13801 - Migrate variables of VariableLayout only if old instance was also variable #13789 https://github.com/pharo-project/pharo/pull/13789 - Fix #13478: Filter on calypsoName instead of name #13778 https://github.com/pharo-project/pharo/pull/13778 - Fix: RBClassRegexRefactoring #13770 https://github.com/pharo-project/pharo/pull/13770 - Loading Pharo12 branch of Sindarin in BaselineOfNewTools for Pharo 12 #525 https://github.com/pharo-spec/NewTools/pull/525 - Add source folder of Sindarin #530 https://github.com/pharo-spec/NewTools/pull/530 # Protocols / ClassOrganizer - Remove user of #organization in Debugger #531 https://github.com/pharo-spec/NewTools/pull/531 - Method announcements should use real protocols #13730 https://github.com/pharo-project/pharo/pull/13730 - Inline ClassOrganization>>extensionProtocols #13783 https://github.com/pharo-project/pharo/pull/13783 - Reduce users of #organization #13785 https://github.com/pharo-project/pharo/pull/13785 - organization, not organisation #13782 https://github.com/pharo-project/pharo/pull/13782 - Remove dead method ClassDescription>>#zapOrganization #13774 https://github.com/pharo-project/pharo/pull/13774 - Inline ClassOrganization>>#hasProtocol: #13776 https://github.com/pharo-project/pharo/pull/13776 # Tests - Add tests for Calypso's item filter #13786 https://github.com/pharo-project/pharo/pull/13786 - FIx DebugSession test #13811 https://github.com/pharo-project/pharo/pull/13811 - Fix completion tests after p11 synch #13809 https://github.com/pharo-project/pharo/pull/13809 - Improve CoverageCollectorTest assertions #13787 https://github.com/pharo-project/pharo/pull/13787 - fix-testConstantBlockClosure-CleanBlocks #13781 https://github.com/pharo-project/pharo/pull/13781 # Cleanup - Remove unimplemented code StSpotterQuery.class.st #528 https://github.com/pharo-spec/NewTools/pull/528 - Remove Deprecated Package Spec2-ObservableSlot #1396 https://github.com/pharo-spec/Spec/pull/1396 - Merge SHTextStyler and SHRBTextStyler #13756 https://github.com/pharo-project/pharo/pull/13756 - Cleanup dead code in RubShoutStylerDecorator #13771 https://github.com/pharo-project/pharo/pull/13771 - Update StGenericGenerator.class.st #526 https://github.com/pharo-spec/NewTools/pull/526

May 26, 2023