This is what we call having a vision :)

Nice!

It is interesting how all these projects are coming together… :-)

Marcus

...

On 24 Feb 2017, at 13:36, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

Hi,

I have published a short video of the Pharo bootstrapping from a Ring model that is browsed and modified by the Calypso browser. https://www.youtube.com/watch?v=QguZSPKo7-w

No audio, please enable subtitles for the description.

Cheers, -- Pavel

-- Using Opera's mail client: http://www.opera.com/mail/

2017-02-24 13:53 GMT+01:00 Cyril Ferlicot D. <cyril.ferlicot@gmail.com>:

On 24/02/2017 13:36, Pavel Krivanek wrote:

...

Hi,

I have published a short video of the Pharo bootstrapping from a Ring model that is browsed and modified by the Calypso browser.

https://www.youtube.com/watch?v=QguZSPKo7-w

No audio, please enable subtitles for the description.

Cheers, -- Pavel

Hi!

Can you explain a little what is Ring model for those who don't know as me? :D

Ring is a Smalltalk metamodel implementation. It's simply a set of classes that statically describe elements in the Smalltalk system, like classes, methods, instance variables etc. Ring was included into the image in Pharo 1.4. Currently we have several similar metamodels in the image or as external projects. Monticello definitions, RBBrowserEnvironment, Ring, Ficus. We are working on the new Ring implementation that tries to solve problems of the original one and enables things like: - browse packages without need of loading them - analyse them (QA checks, ...) - bootstrap from an environment model - work on remote images - join and analyze several models as one (like image model+external package model) - work with incomplete models etc. In ideal state the metamodel should be a layer on top of the environment which all tools use and thus you can use the same workflow for various code sources. Like be able to modify and explore an external package with the standard Pharo tools without need of installation of it. Cheers, -- Pavel

-- Cyril Ferlicot

http://www.synectique.eu

2 rue Jacques Prévert 01, 59650 Villeneuve d'ascq France

Great! any idea how far is new Ring from replacing old Ring? Martín On Fri, Feb 24, 2017 at 11:58 AM, Ben Coman <btc@openinworld.com> wrote:

On Fri, Feb 24, 2017 at 8:36 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

...

Hi,

I have published a short video of the Pharo bootstrapping from a Ring model that is browsed and modified by the Calypso browser.

https://www.youtube.com/watch?v=QguZSPKo7-w

No audio, please enable subtitles for the description.

Wow wow wow !! Super-cool !! I had to watch it a few times. 314k image is pretty impressive.

Just to clarify, Calypso is not connected to a remote image but browses the bootstrap source code loaded into a Ring Environment? (btw, should we call this a "guest environment"?)

When you show the previously opened window "Methods with a string aText 'Smalltalk run' ..." showing PBImageBuilder50 >> initializeImage, that was opened before you created the new Ring Environment and Calypso was opened, so can I presume it was opened on a similar Ring Environment created earlier?

There were no processes in the guest image prior to doing this... process := objectSpace createProcessWithPriority: 40 doing: ('Smalltalk primitiveGarbageCollect. Smalltalk run'). objectSpace installActiveProcess: process. so this was a single process Image?

You say "The image must contain support for the class builder to be able to be bootstrapped." When you create the SmalltalkImage>>run method in the guest environment, is that actually *running* to compiled as normal using its class builder?

How difficult would it be for the guest image to invoke class builder via seamless in the host image? And how much smaller could the image get if the class builder was removed.

It could be cool if this bootstrap creation process was integrated in PharoLauncher as a GUI tool. Then later the ability to download these mini-image to an embedded controller could be added.

cheers -ben

P.S. Could you maybe blog this also, to make it easier to replicate and copy-paste the Playground text.

2017-02-24 15:58 GMT+01:00 Ben Coman <btc@openinworld.com>:

On Fri, Feb 24, 2017 at 8:36 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

...

Hi,

I have published a short video of the Pharo bootstrapping from a Ring model that is browsed and modified by the Calypso browser.

https://www.youtube.com/watch?v=QguZSPKo7-w

No audio, please enable subtitles for the description.

Wow wow wow !! Super-cool !! I had to watch it a few times. 314k image is pretty impressive.

Just to clarify, Calypso is not connected to a remote image but browses the bootstrap source code loaded into a Ring Environment?

Yes, Calypso browses an off-line source code loaded into the Ring environment.

(btw, should we call this a "guest environment"?)

It is possible to open Calypso on a Ring model that takes data from the current image so in that case it would be confusing:-)

When you show the previously opened window "Methods with a string aText 'Smalltalk run' ..." showing PBImageBuilder50 >> initializeImage, that was opened before you created the new Ring Environment and Calypso was opened, so can I presume it was opened on a similar Ring Environment created earlier?

It is a standard Nautilus window on the code in the image that does the bootstrapping. For the standard bootstrap it does much more things. See: https://github.com/guillep/pharo-core/blob/dfbfb77844a31701da6e49691af2a05ad...

There were no processes in the guest image prior to doing this... process := objectSpace createProcessWithPriority: 40 doing: ('Smalltalk primitiveGarbageCollect. Smalltalk run'). objectSpace installActiveProcess: process. so this was a single process Image?

Yes, see the method above

You say "The image must contain support for the class builder to be able to be bootstrapped." When you create the SmalltalkImage>>run method in the guest environment, is that actually *running* to compiled as normal using its class builder?

No, when I create the method, only the source code is set in the Ring model. The model is "dead". The real installation of the method into the generated object memory is done during the bootstrapping.

How difficult would it be for the guest image to invoke class builder via seamless in the host image?

The "guest" object memory runs in a special virtual machine simulator during the bootstrapping. That simulator uses AST Interpreter to execute the code inside the guest environment before it is installed for real. It uses own special kind of proxies and tricks to do it. The current bootstrap cannot run without class builder in the guest environment, but...

And how much smaller could the image get if the class builder was removed.

...Guille in his thesis tried different approaches and he was able to produce extremely small images (~10 KiB) that did for example only a sum of two numbers. https://guillep.github.io/files/publications/Poli15Thesis.pdf

It could be cool if this bootstrap creation process was integrated in PharoLauncher as a GUI tool. Then later the ability to download these mini-image to an embedded controller could be added.

cheers -ben

P.S. Could you maybe blog this also, to make it easier to replicate and copy-paste the Playground text.

2017-02-24 17:17 GMT+01:00 Denis Kudriashov <dionisiydk@gmail.com>:

2017-02-24 16:36 GMT+01:00 Pavel Krivanek <pavel.krivanek@gmail.com>:

...

The "guest" object memory runs in a special virtual machine simulator during the bootstrapping. That simulator uses AST Interpreter to execute the code inside the guest environment before it is installed for real. It uses own special kind of proxies and tricks to do it. The current bootstrap cannot run without class builder in the guest environment, but...

...

And how much smaller could the image get if the class builder was removed.

...Guille in his thesis tried different approaches and he was able to produce extremely small images (~10 KiB) that did for example only a sum of two numbers.

Is it future approach for bootstrap? How faster or slower bootstrap process with it?

That's more question for Guille, but I do not think we will adopt this approach of bootstrapping in near future because it needs to be updated for the Spur format and it needs a special VM modifications. And the question is how much is it really needed because if you want to do something with the image, you will anyway need most of methods needed for the class builder like basic strings and collections support.

Hi, yes this is cool :) There is also a new smart student that will work on this infrastructure a bit to make it "normal developer friendly", you know, hiding some implementation details, making maybe a UI for the process and so on. Now, about the questions: - And how much smaller could the image get if the class builder was removed? Well, as Pavel says, you can get for example 10K. You can get even less if you tweak it manually. Around 10k is the minimal required by the VM to run a full featured image (at least in cog 32 bits, I did not measure it in Spur). Imagine that the VM requires at least: * one process scheduler * one process in that scheduler * one context on that process * one method on that context Then, the VM requires some information to interact with the runtime, e.g., * what is the selector used for doesNotUnderstand: (yes, we can change it!) * what are the first 256 character objects in the Character flyweight * what is the class of SmallInteger immediate instances * some special/optimized selectors My automatic process generated a somewhat 10k of objects required for this VM/image interaction. Of course this list changes a bit with Spur, and could be reduced (for example, if you don't use characters, you wouldn't need the character flyweight and so on...) - Is it future approach for bootstrap? (with respect to the 10k image) Well, if you read my thesis, what I'm doing there is to lazily pick what classes and methods will go inside an image. While this is nice to experiment, I do not think It's a good approach for bootstrapping: * you need to exercise/execute all possible cases your image needs to support * and that means you need to know all those cases in advance * and that you have to be careful with side effects Otherwise, code not executed is not taken in the resulting image. Also take the fact that changing a little line of code may result in bootstrapping an image a couple of times bigger because of a wrong dependency. All these points make this approach less reproducible and a bit more complicated. I prefer an explicit list of packages to bootstrap, that we can control and really test the output :) Also, as Igor says, we could be able to generate an image without a class builder, as we are able to generate an image without a compiler. But we do not want to. Because that is an image that we cannot almost interact with. We cannot load code inside. So the more interesting question is where is the balance between space and functionality. BTW Pavel, this approach may need to be updated but I think we could make it run on top of the simulator also ;). - How faster or slower bootstrap process with it? They are not really comparable. They do not use the same infrastructure, nor the same process, and they do not generate the same results. The 10k image is of course generated in a matter of seconds, but a 600k image containing the seaside counter may take half an hour. In our current approach, the speed of bootstrap is directly related to the number of classes and methods inside the result. There are two ways to speed up the process: - by speeding up the ast interpreter (or replacing it by a compiled version of it) - and by cleaning the kernel Guille On Fri, Feb 24, 2017 at 9:15 PM, Igor Stasenko <siguctua@gmail.com> wrote:

On 24 February 2017 at 21:01, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

...

2017-02-24 17:17 GMT+01:00 Denis Kudriashov <dionisiydk@gmail.com>:

...

2017-02-24 16:36 GMT+01:00 Pavel Krivanek <pavel.krivanek@gmail.com>:

...

The "guest" object memory runs in a special virtual machine simulator during the bootstrapping. That simulator uses AST Interpreter to execute the code inside the guest environment before it is installed for real. It uses own special kind of proxies and tricks to do it. The current bootstrap cannot run without class builder in the guest environment, but...

...

And how much smaller could the image get if the class builder was removed.

...Guille in his thesis tried different approaches and he was able to produce extremely small images (~10 KiB) that did for example only a sum of two numbers.

Is it future approach for bootstrap? How faster or slower bootstrap process with it?

That's more question for Guille, but I do not think we will adopt this approach of bootstrapping in near future because it needs to be updated for the Spur format and it needs a special VM modifications. And the question is how much is it really needed because if you want to do something with the image, you will anyway need most of methods needed for the class builder like basic strings and collections support.

I think, you can shrink image even more, by running something after booting it to remove ClassBuilder & co, if you don't need to create classes etc.

-- Best regards, Igor Stasenko.

On Fri, Feb 24, 2017 at 11:36 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-02-24 15:58 GMT+01:00 Ben Coman <btc@openinworld.com>:

...

On Fri, Feb 24, 2017 at 8:36 PM, Pavel Krivanek <pavel.krivanek@gmail.com

...

wrote:

...

Hi,

I have published a short video of the Pharo bootstrapping from a Ring model that is browsed and modified by the Calypso browser.

https://www.youtube.com/watch?v=QguZSPKo7-w

No audio, please enable subtitles for the description.

Wow wow wow !! Super-cool !! I had to watch it a few times. 314k image is pretty impressive.

Just to clarify, Calypso is not connected to a remote image but browses the bootstrap source code loaded into a Ring Environment?

Yes, Calypso browses an off-line source code loaded into the Ring environment.

...

(btw, should we call this a "guest environment"?)

It is possible to open Calypso on a Ring model that takes data from the current image so in that case it would be confusing:-)

I meant only for the purpose of discussing bootstrap, not Calypso in general. :) The rest of your responses made things clear. Thx. cheers -ben

...

When you show the previously opened window "Methods with a string aText 'Smalltalk run' ..." showing PBImageBuilder50 >> initializeImage, that was opened before you created the new Ring Environment and Calypso was opened, so can I presume it was opened on a similar Ring Environment created earlier?

It is a standard Nautilus window on the code in the image that does the bootstrapping. For the standard bootstrap it does much more things. See: https://github.com/guillep/pharo-core/blob/dfbfb77844a31701da6e49691af2a0 5ad55b4bc6/bootstrap/src/Pharo30Bootstrap.package/PBImageBuilder50.class/ instance/initializeImage.st

...

There were no processes in the guest image prior to doing this... process := objectSpace createProcessWithPriority: 40 doing: ('Smalltalk primitiveGarbageCollect. Smalltalk run'). objectSpace installActiveProcess: process. so this was a single process Image?

Yes, see the method above

...

You say "The image must contain support for the class builder to be able to be bootstrapped." When you create the SmalltalkImage>>run method in the guest environment, is that actually *running* to compiled as normal using its class builder?

No, when I create the method, only the source code is set in the Ring model. The model is "dead". The real installation of the method into the generated object memory is done during the bootstrapping.

...

How difficult would it be for the guest image to invoke class builder via seamless in the host image?

The "guest" object memory runs in a special virtual machine simulator during the bootstrapping. That simulator uses AST Interpreter to execute the code inside the guest environment before it is installed for real. It uses own special kind of proxies and tricks to do it. The current bootstrap cannot run without class builder in the guest environment, but...

...

And how much smaller could the image get if the class builder was removed.

...Guille in his thesis tried different approaches and he was able to produce extremely small images (~10 KiB) that did for example only a sum of two numbers.

https://guillep.github.io/files/publications/Poli15Thesis.pdf

...

It could be cool if this bootstrap creation process was integrated in PharoLauncher as a GUI tool. Then later the ability to download these mini-image to an embedded controller could be added.

...

cheers -ben

P.S. Could you maybe blog this also, to make it easier to replicate and copy-paste the Playground text.