September 2022 - Pharo-vm - lists.pharo.org

JIT Code Zone
by stephane ducasse Sept. 28, 2022

Sept. 28, 2022

Hello, I am wondering how the Pharo VM and Cogit manage the JIT code zone and have several questions related to that! I will ask my questions and what could be the answers, feel free to correct me! Context: I would like to secure this part of the VM, instrument accesses to it and experiment with changes in its layout/location! Initialization: When and how is the code zone memory allocated? And initialized? The allocation is made through the succession of: allocateMemoryForImage:withHeader: then allocateMemory:desiredPosition:. The initialization is made through initializeCodeZoneFrom:upTo: where trampolines, PIC prototypes and offsets are generated/computed. What is the layout of the different components of the code zone? Trampolines and methods/inline caches coexist, trampolines being the first thing generated. Runtime: JIT compilation - What is the granularity of a JIT compilation? When does it write to machine code? The compiler processes bytecodes at the granularity of a method, using outputInstructionsForGeneratedRuntimeAt: or outputInstructionsAt:. It instruments the actual write with enableCodeZoneWriteDuring:flushingCacheWith:. This step only needs write accesses to the method zone region (after the trampolines). Garbage collection - When is the GC interfering with machine code? For rewriting and compaction? Marking with mapMachineCode: and compaction compactCogCompiledCode are the two ways the GC temper with machine code. While the first one only needs to read data, the second needs both read and write accesses. Interaction between interpreter and machine code - How is forward execution (interpreter - machine code) control flow performed? How is backward execution (machine code - interpreter) performed? I guess the trampolines ceEnterCogCodePopReceiverReg and ceReturnToInterpreter: are the main bridges between both worlds. Bonus: What is the difference between Cogit and the CoInterpreter? Thanks in advance for the answers! <https://www.ensta-bretagne.fr/> <https://www.labsticc.fr/en/index/> Quentin Ducasse Doctorant / PhD Student Lab-STICC â UMR CNRS 6285 www.labsticc.fr ENSTA Bretagne Grande Ã©cole d'ingÃ©nieurs et centre de recherche French State Graduate, Post-Graduate and Research Institute 2 rue FranÃ§ois Verny - 29806 Brest Cedex 9 - France www.ensta-bretagne.fr <https://www.ensta-bretagne.fr/>

2 1

About primitiveMethodXray
by stephane ducasse Sept. 25, 2022

Sept. 25, 2022

Hi In CoInterpreter we can find initializePrimitiveTable super initializePrimitiveTable. PrimNumberHashMultiply := 159. self assert: (PrimitiveTable at: PrimNumberHashMultiply + 1) = #primitiveHashMultiply. #(216 253) do: [:pidx| self assert: (PrimitiveTable at: pidx + 1) = #primitiveFail]. self assert: (PrimitiveTable at: 215 + 1) = #primitiveFlushCacheByMethod. PrimitiveTable at: 253 + 1 put: #primitiveCollectCogCodeConstituents; at: 215 + 1 put: #primitiveVoidVMStateForMethod; at: 216 + 1 put: #primitiveMethodXray There is a primitive primitiveMethodXray :) primitiveMethodXray "Lift the veil from a method and answer an integer describing the interior state of its machine code. Used for e.g. VM tests so they can verify they're testing what they think they're testing. 0 implies a vanilla method. Bit 0 = method might be compiled to machine code Bit 1 = method is currently compiled to machine code Bit 2 = is compiled frameless. Bit 3 = method refers to young object. Bit 4 = method too big to be jitted (more than 64k of code, or needs more than 1.5Mb of stack space to compile) Bit 5 = method contains unknown/unjittable bytecode Bit 6 = method should not be jitted because it contains a primitive not to be called from machine code (unused)" | alreadyCogged flags cogMethod | <var: #cogMethod type: #'CogMethod *'> (self methodWithHeaderShouldBeCogged: (objectMemory methodHeaderOf: self stackTop)) ifTrue: [alreadyCogged := self maybeMethodHasCogMethod: self stackTop. flags := 1. alreadyCogged ifFalse: [cogMethod := cogit cog: self stackTop selector: objectMemory nilObject. (cogMethod = nil and: [cogCompiledCodeCompactionCalledFor]) ifTrue: [self commenceCogCompiledCodeCompaction. cogMethod := cogit cog: self stackTop selector: objectMemory nilObject]. cogMethod asInteger caseOf: { [MethodTooBig] -> [flags := 1 + 16]. [EncounteredUnknownBytecode] -> [flags := 1 + 32]. [ShouldNotJIT] -> [flags := 1 + 64] } otherwise: [self deny: (cogMethod asInteger between: MaxNegativeErrorCode and: NotFullyInitialized)]]. (flags = 1 and: [self maybeMethodHasCogMethod: self stackTop]) ifTrue: [cogMethod := self cogMethodOf: self stackTop. flags := cogMethod stackCheckOffset = 0 ifTrue: [7] ifFalse: [3]. cogMethod cmRefersToYoung ifTrue: [flags := flags + 8]. alreadyCogged ifFalse: [cogit freeMethod: cogMethod]]] ifFalse: [flags := 0]. self pop: 1 thenPush: (objectMemory integerObjectOf: flags) and I wanted to execute it. So on compiledMethod I defined a method CompiledMethod >> xray <primitive: 216> and I did (Point>>#degrees) xray but when I execute I get a compiled method and not an integer. So I probably do something silly. But I would like to understand what. Tx

3 3

Updated plugin compilation info?
by HernÃ¡n Morales Durand Sept. 23, 2022

Sept. 23, 2022

Dear all, I wrote a stupid debugger plugin and generated the C code using: ProtoDebugInterpreterPlugin translateInDirectory: 'ProtoDebug' asFileReference doInlining: false. I remember time ago I modified a plugins.ext file and then copied necessary C headers to a directory (along with my generated .c from Slang) etc etc. Now where should I put my new .c file? Which command/script do you use to compile the VM with a new plugin? Cheers, HernÃ¡n

1 0

Nooby questio - getting hello world out of a headless vm
by David Mason Sept. 16, 2022

Sept. 16, 2022

Hi, Iâve built a stackVM straight out of the box from the instructions atÂ https://github.com/pharo-project/pharo-vmÂ to run some experiments. So Iâm trying for a simple Hello world on stdout/stderr. I assumed on headless Transcript would go to some useful place, so I tried, I have a hello.st ----- Transcript show: 'hello world!';cr. "(Smalltalk at: #Transcript) show: 'hello world!';cr." ----- and run-test: ----- #! /bin/sh # # Run it with a headless Pharo: # DIR=~/git/pharo-project/stackVM/build VM=$DIR/vm/Debug/Pharo.app/Contents/MacOS/Pharo #IMAGE=$DIR/dist/pharo-vm/resources/img.image IMAGE=img.image ls -l $VM $IMAGE grep . "$@" /dev/null set -x $VM -vm-display-null -headless --logLevel=4 $IMAGE "$@" --save --quit ----- and I get: : Sep14 ; date;./run-testÂ hello.stÂ ;date Wed 14 Sep 2022 14:09:17 EDT -rwxr-xr-x 1 dmason staff 50226 9 Sep 14:08 /Users/dmason/git/pharo-project/stackVM/build/vm/Debug/Pharo.app/Contents/MacOS/Pharo -rw-r--r-- 1 dmason staff 206336 13 Sep 23:21 img.image hello.st:Transcript show: 'hello world!';cr. hello.st:"(Smalltalk at: #Transcript) show: 'hello world!';cr." + /Users/dmason/git/pharo-project/stackVM/build/vm/Debug/Pharo.app/Contents/MacOS/Pharo -vm-display-null -headless --logLevel=4 img.imageÂ hello.stÂ --save --quit [DEBUG] 2022-09-14 14:09:17.000 logParameters (src/parameters/parameters.c:341):Image file name: img.image [DEBUG] 2022-09-14 14:09:17.000 logParameters (src/parameters/parameters.c:342):Is default Image: no [DEBUG] 2022-09-14 14:09:17.000 logParameters (src/parameters/parameters.c:343):Is interactive session: no [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:331):vmParameters [count = 5]: [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:334): /Users/dmason/git/pharo-project/stackVM/build/vm/Debug/Pharo.app/Contents/MacOS/Pharo [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:334): -vm-display-null [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:334): -headless [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:334): --logLevel=4 [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:334): --headless [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:331):imageParameters [count = 3]: [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:334):Â hello.st [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:334): --save [DEBUG] 2022-09-14 14:09:17.000 logParameterVector (src/parameters/parameters.c:334): --quit [INFO ] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:113):Opening Image: img.image [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:135):Working Directory /Users/dmason/git/AST-Smalltalk/testing [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:137):sizeof(int): 4 [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:138):sizeof(long): 8 [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:139):sizeof(long long): 8 [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:140):sizeof(void*): 8 [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:141):sizeof(sqInt): 8 [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:142):sizeof(sqLong): 8 [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:143):sizeof(float): 4 [DEBUG] 2022-09-14 14:09:17.000 vm_main_with_parameters (src/client.c:144):sizeof(double): 8 [DEBUG] 2022-09-14 14:09:17.000 runOnMainThread (src/client.c:254):Running VM on main thread [DEBUG] 2022-09-14 14:09:17.000 sqAllocateMemory (src/memoryUnix.c:139):Trying to load the image in 0x804000 [DEBUG] 2022-09-14 14:09:17.000 sqAllocateMemory (src/memoryUnix.c:181):Loading the image in 0x106fa4000 [WARN ] 2022-09-14 14:09:17.000 ioLoadModule (src/externalPrimitives.c:102):Failed to load module: SecurityPlugin and 40 minutes later it exits having been kill -9âed Once, it exited quickly, but I have no idea what was different that time. Also âsave does seem to do anything. If it reliably exited, I could experiment, but I have no idea whatâs happening. Help! (and Thanks in advance) ../Dave

3 5

A/B Testing the latest VM on the Pharo repository PRs
by Guillermo Polito Sept. 14, 2022

Sept. 14, 2022

Hi all, TL;DR; I propose we start testing the latest VMs on PRs to the Pharo repository in a "semi-randomâ way. The idea is to earn some confidence while having a small impact on all of you. ==== We need to slowly start pushing the latest VM version, right now hosted in the pharoX branch of the VM repository. This branch so far has all our latest work on the VM for at least one year. Some highlights Interpreter variable auto-localization (Feat/automatic localizationÂ pharo-vm#371 <https://github.com/pharo-project/pharo-vm/pull/371>) Slang cleanups and improvements (Clean slang warningsÂ pharo-vm#413 <https://github.com/pharo-project/pharo-vm/pull/413>, Fix while statement printingÂ pharo-vm#265 <https://github.com/pharo-project/pharo-vm/pull/265>, CastÂ pharo-vm#230 <https://github.com/pharo-project/pharo-vm/pull/230>, Clean slang warningsÂ pharo-vm#413 <https://github.com/pharo-project/pharo-vm/pull/413>, Switch translation to CAST, and testsÂ pharo-vm#311 <https://github.com/pharo-project/pharo-vm/pull/311>, Add several CAST translations and testsÂ pharo-vm#307 <https://github.com/pharo-project/pharo-vm/pull/307>, Fix type inference testsÂ pharo-vm#291 <https://github.com/pharo-project/pharo-vm/pull/291>) SIMD support in the JIT (Speed up object initialization in AArch64 using SIMD instructionsÂ pharo-vm#421 <https://github.com/pharo-project/pharo-vm/pull/421>, SIMD bytecodesÂ pharo-vm#444 <https://github.com/pharo-project/pharo-vm/pull/444>) Initial work on the perm space (Composed image formatÂ pharo-vm#377 <https://github.com/pharo-project/pharo-vm/pull/377>, Composed image format: C translationÂ pharo-vm#388 <https://github.com/pharo-project/pharo-vm/pull/388>, Refactoring of Snapshot and Loading of ImagesÂ pharo-vm#372 <https://github.com/pharo-project/pharo-vm/pull/372>, Perm space on imageÂ pharo-vm#416 <https://github.com/pharo-project/pharo-vm/pull/416>, New & old remembered setsÂ pharo-vm#418 <https://github.com/pharo-project/pharo-vm/pull/418>) Many many bugfixes â¦ To not disrupt anybodyâs work, I propose to make some kind of AB testing on the CI, only impacting PRs to the Pharo repository. odd builds (1,3,5,â¦) use the latest VM even builds (2,4,6,â¦) use the stable VM This will make sure that: the latest VM will be tested in a complex scenario (load code, run testsâ¦) people that have a problem can re-run their builds to get a stable run Iâve made the following PR that follows this idea https://github.com/pharo-project/pharo/pull/11678 G

1 0

https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/
by stephane ducasse Sept. 2, 2022

Sept. 2, 2022

https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/

1 0