Hi Ben, as Nicolas said, yes, MOP is for Meta Object Protocol (really nice book btw https://www.amazon.com/Art-Metaobject-Protocol-Gregor-Kiczales/dp/0262610744 ). Ben, the process is not carved in stone and we are fully aware that it can be enhanced :). Any enhancement, even the littlest one, is important. If it is important for your workflows to add more logging information, let's add them! Nicolas, Actually the special interpreter is not the simulator. We do not use the bytecode interpreter in the simulator, we use an AST interpreter that has "lazy class creation" semantics plus some other little tricks to initialize the circular dependencies of metaclasses and so on... Behind, we use the simulator only to store the objects and decouple ourselves from the object format (this removed for example a lot of work when migrating from V3 to Spur). On Wed, May 2, 2018 at 10:04 PM, Nicolas Cellier < nicolas.cellier.aka.nice@gmail.com> wrote:

Nice reverse engineering!

MOP is Meta Object Protocol (serach in reference [9]).

Maybe you miss a reference to the definition of BaselineOfPharoBootstrap https://github.com/pharo-project/pharo/blob/development/bootstrap/src/ Pharo30Bootstrap.package/PBBootstrap.class/instance/ allPackagesForHermes.st https://github.com/pharo-project/pharo/blob/development/bootstrap/src/ Pharo30Bootstrap.package/PBBootstrap.class/instance/ ensureBaselineOfPharoBootstrap.st

And the simulator too (the special interpreter).

2018-05-02 20:53 GMT+02:00 Ben Coman <btc@openinworld.com>:

...

I found it hard to find info on the Pharo Bootstrap, so I went looking to understand the it by identifying very first creation on disk of the smallest "seed".image file that will later become the final build artifact. I'm sharing here to have a record I can refer to later, and maybe others find interesting.

Looking at the log from the latest build... https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20p ull%20request%20and%20branch%20Pipeline/job/development/822/consoleFull

Searching on "BUILD_NUMBER=" indicates the first major CI step is...

+ BUILD_NUMBER=822 BOOTSTRAP_ARCH=32 bash ./bootstrap/scripts/2-download.sh [1] which downloads 24MB " bootstrapImage.zip" [2] which I presume is an almost full image to run the bootstrap process.

[1] https://github.com/pharo-project/pharo/blob/development/ bootstrap/scripts/2-download.sh [2] https://github.com/guillep/PharoBootstrap/releases/download/ v1.4.1/bootstrapImage.zip

The next major CI step is...

+ BUILD_NUMBER=822 BOOTSTRAP_ARCH=32 bash ./bootstrap/scripts/3-prepare.sh [3] which uses the downloaded "Pharo.image" to (I presume) produce the seed "bootstrap.image" file. (This is not indicated explicitly in the logs, but step 4 copies that file with a shell command, so my presumption. It would be nice for this to be explicitly logged"

prepare.sh executes two commands...

./pharo Pharo.image ..../bootstrap/scripts/prepare_image.st --save --quit [4] which seems to modify the downloaded "Pharo.image", not create bootstrap.image

./pharo Pharo.image ..../bootstrap/scripts/bootstrap.st --ARCH=32 --BUILD_NUMBER=822 --quit [5] which does... "PBBootstrap fromCommandLine bootstrap" [6] [7] which does... PBBootstrap>>#bootstrap PBStepStartedSignal log: 'Starting Process.'. self prepareBootstrap. [self createImage ] on:AssertionFailure do: [ :e | e resume ]. self initializeImage. PBStepFinishedSignal log: 'Process ended.' and digging a bit...

PBBootstrap>>#createImage | builder | builder := PBImageBuilderSpur50 forArchitecture: architecture. builder definitionFetcher: versionFetcher. builder buildNumber: buildNumber. builder imageFileReference: imageReference. builder systemDefinition: ringEnvironment. builder espellBackend instanceVariableMapping: (PBInstanceVariableMapping onEnvironment: ringEnvironment ). builder bootstrap

PBImageBuilderSpur50-class>>#forArchitecture: architecture | candidates | candidates := self subclasses select: [ :sc | sc name endsWith: architecture, 'bit' ]. candidates ifEmpty: [ self error: 'No candidates for architecture: ', architecture ]. candidates size > 1 ifTrue: [ self error: 'More than one candidate for architecture: ', architecture ]. ^ candidates anyOne new

PBImageBuilderSpur50 >>#initialize super initialize. statistics := PBBootstrapStatistics new. logger := PBBootstrapLogger new. imageFileReference := 'bootstrap.image' asFileReference.

"the default espell backend for the bootstrap" self initializeBackend. self instantiateClassLoader.

ahh, there is the reference to "bootstrap.image"

So just to round out a high level overview, the last two messages sent are defined as...

PBImageBuilderSpur5032bit>>#initializeBackend espellBackend := EPSimulatorBackend for32Bit forBootstrap

which I won't dig further but the class seems available at [8] and Espell is described briefly at [9] (but what is a "MOP"? the term is not defined there)

PBImageBuilderSpur50>>#instantiateClassLoader classLoader := PBSpurClassLoader new.

which is defined at [10], and there is an interesting example at [11].

[3] https://github.com/pharo-project/pharo/blob/development/ bootstrap/scripts/3-prepare.sh [4] https://github.com/pharo-project/pharo/blob/development/ bootstrap/scripts/prepare_image.st [5] https://github.com/pharo-project/pharo/blob/development/ bootstrap/scripts/bootstrap.st [6] https://github.com/pharo-project/pharo/blob/development/ bootstrap/src/Pharo30Bootstrap.package/PBBootstrap.class/ class/fromCommandLine.st [7] https://github.com/pharo-project/pharo/blob/development/ bootstrap/src/Pharo30Bootstrap.package/PBBootstrap.class/ instance/bootstrap.st [8] https://github.com/guillep/espell/blob/master/src/ EspellSimulator.package/EPSimulatorBackend.class/class/for32Bit.st [9] https://rmod.inria.fr/archives/papers/Poli15a-Onward- Bootstrapping.pdf [10] https://github.com/pharo-project/pharo/tree/development /bootstrap/src/Pharo30Bootstrap.package/PBSpurClassLoader.class [11] https://github.com/pharo-project/pharo/blob/development /bootstrap/src/Pharo30Bootstrap.package/PBImageBuilderSpur50 .class/class/example5617.st

So now I guess that bootstrap.image has been generated? Just curious, what is the size of this file right now, before the next step? Perhaps an `ls -l` could echo this to the build log.

The penultimate major CI step seems to then start adding packages to bootstrap.image...

+ BUILD_NUMBER=822 BOOTSTRAP_ARCH=32 bash ./bootstrap/scripts/4-build.sh

located at [12] and searching on "${VM}" is enlightening. Its help information says... Pharo Build Script ================== This script assumes the existence of a new uninitialized image. Then it proceeds to its initialization and \"growing\". * Step 1: - Initialize the image - output: core.image * Step 2: - Bootstrap Monticello local repositories - output: monticello_bootstrap.image and changes file * Step 3: - Load Monticello remote repositories - output: monticello.image and changes file * Step 4: - Load Metacello - output: metacello.image and changes file * Step 5: - Load the rest of the image using BaselineOfIDE - output: Pharo.image and changes file

Unfortunately those steps are labelled in the script, but the first action shown in the log is... cp bootstrap.image Pharo7.0-compiler-32bit-9912d7c.image Initializing Bootstraped Image pharo --headless Pharo7.0-compiler-32bit-9912d7c.image

[12] https://github.com/pharo-project/pharo/blob/development /bootstrap/scripts/4-build.sh

The final major CI step was a fair way past what I was interested in right now. ...

+ BUILD_NUMBER=822 bash ../bootstrap/scripts/prepare_for_upload.sh

cheers -ben

-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13