Re: [Pharo-dev] [ANN] Pharo bootstrap
On Wed, Jan 20, 2016 at 4:19 PM, Peter H. Meadows via Pharo-dev < pharo-dev@lists.pharo.org> wrote:
---------- Forwarded message ---------- From: "Peter H. Meadows" <peter.h.meadows@googlemail.com> To: Pharo Development List <pharo-dev@lists.pharo.org> Cc: Date: Wed, 20 Jan 2016 15:17:59 +0000 Subject: Re: [Pharo-dev] [ANN] Pharo bootstrap Cool. Any chance someone can explain for beginners to understand what's going on here? Why is cleaning the image not as simple as running it and deleting any object that wasn't used?
The point is not to clean but to start with an empty object engine and fill it in with the minimum required core. Currently, the image is the result of an evolutionary process and rebuilding it from zero was not possible. This maes it possible to audit the whole build process, create different base image for various purposes (very small, headless with no UI objects at all, using the object engine in a completely different way that is used today). That makes the image the result of a fully reproducible process, from a file of zero bytes, to a file with a working image. That would make us master the whole chain. Today, there is still black magic in the image, due to the history. That's an exciting development. For example, in C, one compiles all files, and links, giving the exe. All that from a file of zero size. We will be able to do the same kind of thing at the image level. Also, this allows to experiment with new low level mechanisms. This is currently not possible since it makes us like performing surgery on our own brain. Also, the Oz-inspired system will help in exploring other image memories (small or large) and manipulate them as entities. At this point, this is also not common pratice. HTH Phil, wishing to master the mitosis competency...
(Thanks in advance)
On 20 January 2016 at 12:52, Guillermo Polito <guillermopolito@gmail.com> wrote:
Well, there is no formal specification⦠But I could summarize the features as follows (and as Christophe points mostly out)
OzVM supports having two images in logically separated spaces. This separation is achieved by using the free bit in the object header. Thus we support only two spaces by now.
- The main addition is actually a new primitive: #resumeFromSpecialObjectsArray that receives an special objects array as argument and: - resumes the execution from the active process of such array - on context switch, if the VM is running from a second special objects array, it will return to the first one instead.
- fixes to primitives/bytecodes that assume a single special objects array. e.g., - #someObject and #nextObject should only iterate objects from the correct âspaceâ - #isNil comparisons should take the correct nil - the GC should set the correct nil object on Weak containers
The extensions I made are about 20 overrides to StackPrimitive and NewObjectMemory methods.
The rest is implemented completely on image side. We use mirror primitives to manipulate objects from other space (as each image/space has itâs own selector table).
I have in my long TODO list see if the "clever hackâ I did can be supported in a nice manner on top of Spurâs segmented memory. However, I should stop commencing new projects and start finishing them :P.
On 20 ene 2016, at 9:03 a.m., Christophe Demarey <Christophe.Demarey@inria.fr> wrote:
Hi Eliot,
Le 19 janv. 2016 à 19:29, Eliot Miranda a écrit :
Hi All,
great news! Where can I read a specification of the Oz VM facilities?
I do not know all the details of the Oz VM but basically, it is a standard stack interpreter vm specialized to be able to run 2 images at the same time on top of it. Guillermo made it possible by using one available bit in the object header of objects to mark the ownership of an object (e.g. is my object from image1 or image2?). Then, he mainly had to specialize VM primitives dealing with objects retrieval from memory. By example, he had to modify the garbage collector to set the right nil instance (yes, we have 2 images so 2 nil instances. we need to take the one of the active image) when an object is garbage collected; the primitive someObject has to return an object from the active image, etc. There is also a way to switch execution from an image to the other just by switching the special objects array reference.
You can find some information in: https://guillep.github.io/files/publications/Poli15Thesis.pdf. The Oz code is in http://smalltalkhub.com/#!/~Guille/ObjectSpace and in https://github.com/guillep/OzVM The current implementation uses Cog 6.6.1and OzVM-GuillermoPolito.22. I do not know if Guille has a more formal specification of the Oz VM.
If you have advices on what is the best way to handle two distinct object (memory) spaces in Spur, they will be welcomed :)
Cheers, Christophe
_,,,^..^,,,_ (phone)
On Jan 19, 2016, at 6:29 AM, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Hi all,
In case you do not know, we work on bootstrapping Pharo, i.e. create a Pharo image from sources, not based on a previous image (well, we use a pharo image to produce it but no code / state from it).
This process will allow to define a minimal Pharo kernel (currently 52 packages but we could have it far smaller) and to modularize the whole image (currently packages have too much dependencies on packages already loaded into the image). The bootstrap process also allows to write down the recipe to initialize a new image from scratch (some code is missing in the image or is wrong). In addition, I think we will clean a lot of historical objects that are not used anymore.
With the amazing work done by Guillermo Polito during his Phd (around Espell, Oz): https://guillep.github.io/files/publications/Poli15Thesis.pdf, we succeeded to get a first prototype of a bootstraped Pharo 5 image (from 5.392). This prototype is able to run an eval command line handler and to log output / errors. Not all classes are yet initialized and you cannot yet save / restart this image but it is a big step forward. It is a 4 mb image (could be half the size without unicode data). You can download it at:
http://chercheurs.lille.inria.fr/~demarey/pmwiki/pub/pharo-bootstrap/pharo-b... .
Next steps are to have a bootstrapped image fully working, then to load packages on top of it (like network, sunit) to produce a minimal image. Then, we need to implement an Oz VM on top of Spur. After that, we need to work on a reliable way to build the bootstrap (not too sensitive to changes in the image).
Christophe.
------- demarey@193-51-236-143:~/dev/rmod/bootstrap/bootstrap-2016-01-19$
../pharo
bootstrap.image --no-default-preferences eval "1 + 1" 2 demarey@193-51-236-143:~/dev/rmod/bootstrap/bootstrap-2016-01-19$ ../pharo bootstrap.image --no-default-preferences eval "'a' , 'b'" 'ab' demarey@193-51-236-143:~/dev/rmod/bootstrap/bootstrap-2016-01-19$ ../pharo bootstrap.image --no-default-preferences eval "1 / 0" ZeroDivide SmallInteger>>/ UndefinedObject>>DoIt OpalCompiler>>evaluate OpalCompiler(AbstractCompiler)>>evaluate: SmalltalkImage>>evaluate:
EvaluateCommandLineHandler>>no (source is Undeclared) no source in EvaluateCommandLineHandler>>evaluate: in Block: no source BlockClosure>>on:do: EvaluateCommandLineHandler>>evaluate: EvaluateCommandLineHandler>>evaluateArguments EvaluateCommandLineHandler>>activate EvaluateCommandLineHandler class(CommandLineHandler class)>>activateWith:
BasicCommandLineHandler>>no (source is Undeclared) no source in PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: in Block: no source BlockClosure>>on:do: PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: PharoCommandLineHandler(BasicCommandLineHandler)>>handleSubcommand PharoCommandLineHandler(BasicCommandLineHandler)>>handleArgument:
BasicCommandLineHandler>>no (source is Undeclared) no source in PharoCommandLineHandler(BasicCommandLineHandler)>>activate in Block: no source BlockClosure>>on:do: PharoCommandLineHandler(BasicCommandLineHandler)>>activate PharoCommandLineHandler>>activate PharoCommandLineHandler class(CommandLineHandler class)>>activateWith:
PharoCommandLineHandler class>>no (source is Undeclared) no source in PharoCommandLineHandler class>>activateWith: in Block: no source NonInteractiveUIManager(UIManager)>>defer: PharoCommandLineHandler class>>activateWith: no source in BasicCommandLineHandler>>activateSubCommand: in Block: no source BlockClosure>>on:do: BasicCommandLineHandler>>activateSubCommand: BasicCommandLineHandler>>handleSubcommand BasicCommandLineHandler>>handleArgument: no source in BasicCommandLineHandler>>activate in Block: no source
SmallInteger>>no (source is Undeclared)
UndefinedObject>>no (source is Undeclared)
AbstractCompiler>>no (source is Undeclared)
SmalltalkImage>>no (source is Undeclared)
BlockClosure>>no (source is Undeclared)
EvaluateCommandLineHandler>>no (source is Undeclared)
EvaluateCommandLineHandler>>no (source is Undeclared)
CommandLineHandler class>>no (source is Undeclared)
BasicCommandLineHandler>>no (source is Undeclared)
BasicCommandLineHandler>>no (source is Undeclared)
PharoCommandLineHandler>>no (source is Undeclared)
UIManager>>no (source is Undeclared)
UndefinedObject>>no (source is Undeclared)
CommandLineUIManager>>no (source is Undeclared)
SmalltalkImage>>no (source is Undeclared)
DelayMicrosecondScheduler>>no (source is Undeclared)
BlockClosure>>no (source is Undeclared)
SmalltalkImage>>no (source is Undeclared)
WeakArray class>>no (source is Undeclared)
ps: source cannot be displayed because there is no formatter available in the bootstrap
Hi all-- Phil writes:
The point is not to clean but to start with an empty object engine and fill it in with the minimum required core. Currently, the image is the result of an evolutionary process and rebuilding it from zero was not possible.
Another approach is to modify the virtual machine so that it marks methods as they are run, and modify the garbage collector to reclaim methods that haven't been run. Then you can create systems that consist of only what is necessary to run unit tests, effectively imprinting the unit tests. You can interact with the target system from completely independent one over a remote messaging network connection, so your unit tests need not include graphics support, etc.[1] This seems much simpler to me than making a virtual machine that can run multiple object memories, and distributed object memories have several other important uses too.
This makes it possible to audit the whole build process, create different base image for various purposes (very small, headless with no UI objects at all, using the object engine in a completely different way that is used today).
That makes the image the result of a fully reproducible process, from a file of zero bytes, to a file with a working image.
You can do all of these things with imprinted unit tests as well.
Also, this allows to experiment with new low level mechanisms. This is currently not possible since it makes us like performing surgery on our own brain.
With remote messaging, which has been around for many years, you can perform this surgery from afar.
Also, the Oz-inspired system will help in exploring other image memories (small or large) and manipulate them as entities. At this point, this is also not common pratice.
This has been common practice since the introduction of the virtual machine simulator in 1996. The remote facilities I mentioned all work under simulation, too. There's a lot of prior art here. -C [1] http://netjam.org/context -- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Has anyone looked at the VASmalltalk Envy Packager. It let's one decide whether you want to package a headless or UI image. It then allows one to define packaging instructions, that are saved with the new application), that get fed into the packager. The instructions define prerequisite apps, un-referenced methods, etc. When one defines a new application (package), one can define prerequisite applications that have to be installed before the current application is installed. This prerequisite chain is used in loading the application from Envy and in packaging. Seems to work fairly well. Doesn't take too long to package an app and give the developer pretty fine control over the packaging and loading process. ----- Brad Selfridge -- View this message in context: http://forum.world.st/ANN-Pharo-bootstrap-tp4872633p4872989.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
Hi Brad, Envy is a package manager. It does not define the minimal kernel you need to install packages needed for your application. The goal of a bootstrap is to set up an environment from a know state. In the same time, this exercise forces us to define the minimal kernel needed to run Pharo (because you do not want to bootstrap the full system). Currently, we miss both the minimal kernel and a bootstrapped image of Pharo. Le 20 janv. 2016 à 20:17, Brad Selfridge a écrit :
Has anyone looked at the VASmalltalk Envy Packager. It let's one decide whether you want to package a headless or UI image. It then allows one to define packaging instructions, that are saved with the new application), that get fed into the packager. The instructions define prerequisite apps, un-referenced methods, etc. When one defines a new application (package), one can define prerequisite applications that have to be installed before the current application is installed. This prerequisite chain is used in loading the application from Envy and in packaging. Seems to work fairly well. Doesn't take too long to package an app and give the developer pretty fine control over the packaging and loading process.
----- Brad Selfridge -- View this message in context: http://forum.world.st/ANN-Pharo-bootstrap-tp4872633p4872989.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
Le 20 janv. 2016 à 20:09, Craig Latta a écrit :
Hi all--
Phil writes:
The point is not to clean but to start with an empty object engine and fill it in with the minimum required core. Currently, the image is the result of an evolutionary process and rebuilding it from zero was not possible.
Another approach is to modify the virtual machine so that it marks methods as they are run, and modify the garbage collector to reclaim methods that haven't been run. Then you can create systems that consist of only what is necessary to run unit tests, effectively imprinting the unit tests. You can interact with the target system from completely independent one over a remote messaging network connection, so your unit tests need not include graphics support, etc.[1] This seems much simpler to me than making a virtual machine that can run multiple object memories, and distributed object memories have several other important uses too.
Guillermo also implemented this kind of approach with Tornado. "Tornado, a technique to dynamically tailor applications to only embed code (classes and methods) they use. Tornado uses a run-fail-grow approach to prepare an application for deployment. It launches minimal version of an application and installs a minimal set of statements that will start the user's application. This ap- plication is run and these statements are executed. When the application fails because there are classes or methods missing, the necessary code is installed. Extended results of Tornado: A Run-Fail-Grow approach for Dynamic Application Tayloring (PDF Download Available). Available from: https://www.researchgate.net/publication/278629977_Extended_results_of_Torna... [accessed Jan 21, 2016]." But is not so easy ...
This makes it possible to audit the whole build process, create different base image for various purposes (very small, headless with no UI objects at all, using the object engine in a completely different way that is used today).
That makes the image the result of a fully reproducible process, from a file of zero bytes, to a file with a working image.
You can do all of these things with imprinted unit tests as well.
Also, this allows to experiment with new low level mechanisms. This is currently not possible since it makes us like performing surgery on our own brain.
With remote messaging, which has been around for many years, you can perform this surgery from afar.
Also, the Oz-inspired system will help in exploring other image memories (small or large) and manipulate them as entities. At this point, this is also not common pratice.
This has been common practice since the introduction of the virtual machine simulator in 1996. The remote facilities I mentioned all work under simulation, too.
There's a lot of prior art here.
-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Hi Christophe--
Another approach is to modify the virtual machine so that it marks methods as they are run, and modify the garbage collector to reclaim methods that haven't been run. Then you can create systems that consist of only what is necessary to run unit tests, effectively imprinting the unit tests. You can interact with the target system from completely independent one over a remote messaging network connection, so your unit tests need not include graphics support, etc.[1] This seems much simpler to me than making a virtual machine that can run multiple object memories, and distributed object memories have several other important uses too.
Guillermo also implemented this kind of approach with Tornado.
Right, I read that in his PhD thesis. That's why I mentioned my earlier work, when others claimed there was a precedent being set.
But is not so easy...
Sure, I learned that first-hand when I did it. One important thing I learned is that it's easier and more accurate to install code as a side-effect of actually running it live, and not through analysis. It's also useful to have the option of faulting code in when it's missing from a running target, or pushing it in from a running source. -C [1] http://netjam.org/context -- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Peter, The point is that âcleaning the imageâ is - not reproducible. A cleanup script may work with todayâs image but may not work with tomorrowâs one. - Itâs not only about cleaning. Imagine you observe that the tables keeping Unicode mappings are wrong. And there is no code to reproduce them. - it does not scale when it starts to become circular. For example, you want to refactor a part of morphic => but you are use morphic while the refactor is performed. We do that all the time, but at the cost of splitting and staging changes that require to be atomic, which adds more complexity to the process. Craig, Yes, Spoon was indeed an inspiration. However, some notes on top of your comments: - Oz/Espell's existing implementation works by introducing two object memories in the same VM, yes. However, the goal was to abstract Espellâs user from such detail and have other implementations such as a remote image one. For example, I wrote a prototype of Espell that manipulated an image loaded in the simulator. The result is that you had a high-level API provided by mirrors to manipulate an image without requiring a VM change. However it was also much slower, and required adapting the VM simulator to initialize it with an empty object memory, and being able to call primitives from outside the simulator while keeping the stack coherent. I dropped this prototype because it required a lot of engineering effort and at the same time Spur advanced really quickly, so I did not want to spend time in code that would soon be deprecated. I could however check if I can recover the code. - Also, I would not say that distributed object memories "are simpler" than keeping co-existing object memories, or not at least without exploring itâs flaws also :). A distributed object memory forces you to include in every peer that may receive code a distribution layer (at least sockets, class builder, compiler or serializer + dependencies). And we wanted to go further. We wanted to be able to easily manipulate an image that does not have those features. - The main point was to use the same infrastructure to build an image in an explicit manner (the bootstrap) as well as in an implicit on-demand manner (tornado). And I believe that the bootstrap is a more transparent process where we can control what happens. Guille
On 21 ene 2016, at 2:34 p.m., Craig Latta <craig@netjam.org> wrote:
Hi Christophe--
Another approach is to modify the virtual machine so that it marks methods as they are run, and modify the garbage collector to reclaim methods that haven't been run. Then you can create systems that consist of only what is necessary to run unit tests, effectively imprinting the unit tests. You can interact with the target system from completely independent one over a remote messaging network connection, so your unit tests need not include graphics support, etc.[1] This seems much simpler to me than making a virtual machine that can run multiple object memories, and distributed object memories have several other important uses too.
Guillermo also implemented this kind of approach with Tornado.
Right, I read that in his PhD thesis. That's why I mentioned my earlier work, when others claimed there was a precedent being set.
But is not so easy...
Sure, I learned that first-hand when I did it. One important thing I learned is that it's easier and more accurate to install code as a side-effect of actually running it live, and not through analysis. It's also useful to have the option of faulting code in when it's missing from a running target, or pushing it in from a running source.
-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Hi Guille--
The point is that âcleaning the imageâ is not reproducible. A cleanup script may work with todayâs image but may not work with tomorrowâs.
I consider this sort of cleaning something that gets done once per preexisting Smalltalk implementation (Pharo, VisualWorks, etc.). After it's done, I would build every release artifact up from the identified essentials (minimal object memory). You need a good module system for that, and I think one based on live remote messaging instead of source code stored in files would be best (easiest to build and use, and most accurate). With the knowledge of those essentials, I would create new Smalltalk implementations from a system tracer informed by them. Cleaning isn't something I would expect to do routinely as an app developer. It's primarily a research activity.
[Cleaning] does not scale when it starts to become circular. For example, you want to refactor a part of morphic => but you are use morphic while the refactor is performed. We do that all the time, but at the cost of splitting and staging changes that require to be atomic, which adds more complexity to the process.
Of course, this is another argument for using remote tools.
...I wrote a prototype of Espell that manipulated an image loaded in the simulator. The result is that you had a high-level API provided by mirrors to manipulate an image without requiring a VM change.
I think that's definitely something you want to be able to do, since there are some changes that can only be made when time is stopped for the target image. And the simulator is simply the most pleasant tool for it. The implementation of this that I did was especially useful for making complimentary documentation, like the directed-graph movies at [1]. True, the simulator is slow, and making movie frames with it is glacial. :) We should never be afraid to change the virtual machine, though, in the absence of any other constraint. :)
A distributed object memory forces you to include in every peer that may receive code a distribution layer (at least sockets, class builder, compiler or serializer + dependencies). And we wanted to go further. We wanted to be able to easily manipulate an image that does not have those features.
Right, I use the simulator stuff when I care about leaving those things out (e.g. [2]). But all of the systems I actually want to deploy do have those things, to the point that I do consider them to be as fundamental as anything else. And they don't include the class builder or compiler; the remote messaging protocol I wrote gets by with just sockets, a few collection classes, and the reflection primitives that the virtual machine needs anyway. There is never a need to compile source code.
The main point was to use the same infrastructure to build an image in an explicit manner (the bootstrap) as well as in an implicit on-demand manner (tornado). And I believe that the bootstrap is a more transparent process where we can control what happens.
It seems to me that push imprinting, where methods are transferred from one system to another as a side-effect of running them, is just as transparent and controllable, and gives you tools that you want anyway for other purposes. thanks, -C [1] http://netjam.org/context/viz [2] http://netjam.org/context/smallest -- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
On Thu, Jan 21, 2016 at 10:33 AM, Christophe Demarey < Christophe.Demarey@inria.fr> wrote:
Le 20 janv. 2016 à 20:09, Craig Latta a écrit :
Hi all--
Phil writes:
The point is not to clean but to start with an empty object engine and
fill it in with the minimum required core. Currently, the image is
the result of an evolutionary process and rebuilding it from zero was
not possible.
Another approach is to modify the virtual machine so that it marks methods as they are run, and modify the garbage collector to reclaim methods that haven't been run. Then you can create systems that consist of only what is necessary to run unit tests, effectively imprinting the unit tests. You can interact with the target system from completely independent one over a remote messaging network connection, so your unit tests need not include graphics support, etc.[1] This seems much simpler to me than making a virtual machine that can run multiple object memories, and distributed object memories have several other important uses too.
Guillermo also implemented this kind of approach with Tornado. "Tornado, a technique to dynamically tailor applications to only embed code (classes and methods) they use. Tornado uses a run-fail-grow approach to prepare an application for deployment. It launches minimal version of an application and installs a minimal set of statements that will start the user's application. This ap- plication is run and these statements are executed. When the application fails because there are classes or methods missing, the necessary code is installed. *Extended results of Tornado: A Run-Fail-Grow approach for Dynamic Application Tayloring (PDF Download Available)*. Available from: https://www.researchgate.net/publication/278629977_Extended_results_of_Torna... [accessed Jan 21, 2016]."
But is not so easy ...
This makes it possible to audit the whole build process, create
different base image for various purposes (very small, headless with
no UI objects at all, using the object engine in a completely
different way that is used today).
That makes the image the result of a fully reproducible process, from
a file of zero bytes, to a file with a working image.
You can do all of these things with imprinted unit tests as well.
Also, this allows to experiment with new low level mechanisms. This is
currently not possible since it makes us like performing surgery on
our own brain.
With remote messaging, which has been around for many years, you can perform this surgery from afar.
Not available in my Pharo environment. Which is what I use and care about.
Also, the Oz-inspired system will help in exploring other image
memories (small or large) and manipulate them as entities. At this
point, this is also not common pratice.
This has been common practice since the introduction of the virtual machine simulator in 1996. The remote facilities I mentioned all work under simulation, too.
Simulator not working under Pharo currently. Same issue.
There's a lot of prior art here.
Sure, but prior art is not running on my environment. So, it is irrelevant
for my current activities. Phil
-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Hi Phil--
...prior art is not running on my environment. So, it is irrelevant for my current activities.
You weren't talking about your current activities before, you were talking about some allegedly innovative capability. At any rate, I do have versions of Context for Pharo and other Smalltalks, which I intend to release this year. I wanted to deal with SqueakJS after Squeak, then Pharo, then VisualWorks, etc. -C -- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Craig, Apologies for being blunt, but supercool stuff that I know about but am not able to use is not of any use to me.
From my perspective, I am very frustrated to not be able to use:
- Context in Pharo - Your remote running of tiny images in my Pharo browser (including the sonar ping sound when messages are landing remotely) - Using WebDav for looking into my images using your WebDav component That's a hell of a lot of super great tech that hasn't seen the light of day in the Pharo community (think being able to explore the image as a filesystem, edit the methods in Vim in a terminal [yes, this is actually cool to me], drag and drop binaries in these folders to import/export binary resources into the image). It is frustrating to see that what could turn into a superb capability of the tooling is running on your box but is not available for use by us (me) the common people. It is your code, you do what you want. I'd live to see it running (even super badly, which I don't think would be the case) on Pharo. Peace, Phil On Thu, Jan 21, 2016 at 7:39 PM, Craig Latta <craig@netjam.org> wrote:
Hi Phil--
...prior art is not running on my environment. So, it is irrelevant for my current activities.
You weren't talking about your current activities before, you were talking about some allegedly innovative capability.
At any rate, I do have versions of Context for Pharo and other Smalltalks, which I intend to release this year. I wanted to deal with SqueakJS after Squeak, then Pharo, then VisualWorks, etc.
-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
From my perspective, I am very frustrated to not be able to use:
- Context in Pharo - Your remote running of tiny images in my Pharo browser (including the sonar ping sound when messages are landing remotely) - Using WebDav for looking into my images using your WebDav component
Cool, that's all planned for this year (in reverse order). thanks, -C -- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
On 20 January 2016 at 16:36, phil@highoctane.be <phil@highoctane.be> wrote:
On Wed, Jan 20, 2016 at 4:19 PM, Peter H. Meadows via Pharo-dev <pharo-dev@lists.pharo.org> wrote:
---------- Forwarded message ---------- From: "Peter H. Meadows" <peter.h.meadows@googlemail.com> To: Pharo Development List <pharo-dev@lists.pharo.org> Cc: Date: Wed, 20 Jan 2016 15:17:59 +0000 Subject: Re: [Pharo-dev] [ANN] Pharo bootstrap Cool. Any chance someone can explain for beginners to understand what's going on here? Why is cleaning the image not as simple as running it and deleting any object that wasn't used?
The point is not to clean but to start with an empty object engine and fill it in with the minimum required core.
But why should this not end up with the same end result? One way is to start with nothing, and only add stuff we need to make it run/boot, and the other way is to start with what we have, and remove anything that is not needed to make it run/boot. In both cases don't you end up with the same thing?
Currently, the image is the result of an evolutionary process and rebuilding it from zero was not possible.
I wonder if they thought about this when they created the first image. Didn't they think 'hey, we should really have a way for anyone to create new images from nothing'? Or was it just that they didn't get around to making this tool?
This maes it possible to audit the whole build process, create different base image for various purposes (very small, headless with no UI objects at all, using the object engine in a completely different way that is used today).
That makes the image the result of a fully reproducible process, from a file of zero bytes, to a file with a working image.
That would make us master the whole chain. Today, there is still black magic in the image, due to the history.
That's an exciting development. For example, in C, one compiles all files, and links, giving the exe. All that from a file of zero size. We will be able to do the same kind of thing at the image level.
Also, this allows to experiment with new low level mechanisms. This is currently not possible since it makes us like performing surgery on our own brain. Also, the Oz-inspired system will help in exploring other image memories (small or large) and manipulate them as entities. At this point, this is also not common pratice.
HTH Phil, wishing to master the mitosis competency...
(Thanks in advance)
On 20 January 2016 at 12:52, Guillermo Polito <guillermopolito@gmail.com> wrote:
Well, there is no formal specification⦠But I could summarize the features as follows (and as Christophe points mostly out)
OzVM supports having two images in logically separated spaces. This separation is achieved by using the free bit in the object header. Thus we support only two spaces by now.
- The main addition is actually a new primitive: #resumeFromSpecialObjectsArray that receives an special objects array as argument and: - resumes the execution from the active process of such array - on context switch, if the VM is running from a second special objects array, it will return to the first one instead.
- fixes to primitives/bytecodes that assume a single special objects array. e.g., - #someObject and #nextObject should only iterate objects from the correct âspaceâ - #isNil comparisons should take the correct nil - the GC should set the correct nil object on Weak containers
The extensions I made are about 20 overrides to StackPrimitive and NewObjectMemory methods.
The rest is implemented completely on image side. We use mirror primitives to manipulate objects from other space (as each image/space has itâs own selector table).
I have in my long TODO list see if the "clever hackâ I did can be supported in a nice manner on top of Spurâs segmented memory. However, I should stop commencing new projects and start finishing them :P.
On 20 ene 2016, at 9:03 a.m., Christophe Demarey <Christophe.Demarey@inria.fr> wrote:
Hi Eliot,
Le 19 janv. 2016 à 19:29, Eliot Miranda a écrit :
Hi All,
great news! Where can I read a specification of the Oz VM facilities?
I do not know all the details of the Oz VM but basically, it is a standard stack interpreter vm specialized to be able to run 2 images at the same time on top of it. Guillermo made it possible by using one available bit in the object header of objects to mark the ownership of an object (e.g. is my object from image1 or image2?). Then, he mainly had to specialize VM primitives dealing with objects retrieval from memory. By example, he had to modify the garbage collector to set the right nil instance (yes, we have 2 images so 2 nil instances. we need to take the one of the active image) when an object is garbage collected; the primitive someObject has to return an object from the active image, etc. There is also a way to switch execution from an image to the other just by switching the special objects array reference.
You can find some information in: https://guillep.github.io/files/publications/Poli15Thesis.pdf. The Oz code is in http://smalltalkhub.com/#!/~Guille/ObjectSpace and in https://github.com/guillep/OzVM The current implementation uses Cog 6.6.1and OzVM-GuillermoPolito.22. I do not know if Guille has a more formal specification of the Oz VM.
If you have advices on what is the best way to handle two distinct object (memory) spaces in Spur, they will be welcomed :)
Cheers, Christophe
_,,,^..^,,,_ (phone)
On Jan 19, 2016, at 6:29 AM, Christophe Demarey <christophe.demarey@inria.fr> wrote:
Hi all,
In case you do not know, we work on bootstrapping Pharo, i.e. create a Pharo image from sources, not based on a previous image (well, we use a pharo image to produce it but no code / state from it).
This process will allow to define a minimal Pharo kernel (currently 52 packages but we could have it far smaller) and to modularize the whole image (currently packages have too much dependencies on packages already loaded into the image). The bootstrap process also allows to write down the recipe to initialize a new image from scratch (some code is missing in the image or is wrong). In addition, I think we will clean a lot of historical objects that are not used anymore.
With the amazing work done by Guillermo Polito during his Phd (around Espell, Oz): https://guillep.github.io/files/publications/Poli15Thesis.pdf, we succeeded to get a first prototype of a bootstraped Pharo 5 image (from 5.392). This prototype is able to run an eval command line handler and to log output / errors. Not all classes are yet initialized and you cannot yet save / restart this image but it is a big step forward. It is a 4 mb image (could be half the size without unicode data). You can download it at:
http://chercheurs.lille.inria.fr/~demarey/pmwiki/pub/pharo-bootstrap/pharo-b....
Next steps are to have a bootstrapped image fully working, then to load packages on top of it (like network, sunit) to produce a minimal image. Then, we need to implement an Oz VM on top of Spur. After that, we need to work on a reliable way to build the bootstrap (not too sensitive to changes in the image).
Christophe.
------- demarey@193-51-236-143:~/dev/rmod/bootstrap/bootstrap-2016-01-19$ ../pharo bootstrap.image --no-default-preferences eval "1 + 1" 2 demarey@193-51-236-143:~/dev/rmod/bootstrap/bootstrap-2016-01-19$ ../pharo bootstrap.image --no-default-preferences eval "'a' , 'b'" 'ab' demarey@193-51-236-143:~/dev/rmod/bootstrap/bootstrap-2016-01-19$ ../pharo bootstrap.image --no-default-preferences eval "1 / 0" ZeroDivide SmallInteger>>/ UndefinedObject>>DoIt OpalCompiler>>evaluate OpalCompiler(AbstractCompiler)>>evaluate: SmalltalkImage>>evaluate:
EvaluateCommandLineHandler>>no (source is Undeclared) no source in EvaluateCommandLineHandler>>evaluate: in Block: no source BlockClosure>>on:do: EvaluateCommandLineHandler>>evaluate: EvaluateCommandLineHandler>>evaluateArguments EvaluateCommandLineHandler>>activate EvaluateCommandLineHandler class(CommandLineHandler class)>>activateWith:
BasicCommandLineHandler>>no (source is Undeclared) no source in PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: in Block: no source BlockClosure>>on:do: PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: PharoCommandLineHandler(BasicCommandLineHandler)>>handleSubcommand PharoCommandLineHandler(BasicCommandLineHandler)>>handleArgument:
BasicCommandLineHandler>>no (source is Undeclared) no source in PharoCommandLineHandler(BasicCommandLineHandler)>>activate in Block: no source BlockClosure>>on:do: PharoCommandLineHandler(BasicCommandLineHandler)>>activate PharoCommandLineHandler>>activate PharoCommandLineHandler class(CommandLineHandler class)>>activateWith:
PharoCommandLineHandler class>>no (source is Undeclared) no source in PharoCommandLineHandler class>>activateWith: in Block: no source NonInteractiveUIManager(UIManager)>>defer: PharoCommandLineHandler class>>activateWith: no source in BasicCommandLineHandler>>activateSubCommand: in Block: no source BlockClosure>>on:do: BasicCommandLineHandler>>activateSubCommand: BasicCommandLineHandler>>handleSubcommand BasicCommandLineHandler>>handleArgument: no source in BasicCommandLineHandler>>activate in Block: no source
SmallInteger>>no (source is Undeclared)
UndefinedObject>>no (source is Undeclared)
AbstractCompiler>>no (source is Undeclared)
SmalltalkImage>>no (source is Undeclared)
BlockClosure>>no (source is Undeclared)
EvaluateCommandLineHandler>>no (source is Undeclared)
EvaluateCommandLineHandler>>no (source is Undeclared)
CommandLineHandler class>>no (source is Undeclared)
BasicCommandLineHandler>>no (source is Undeclared)
BasicCommandLineHandler>>no (source is Undeclared)
PharoCommandLineHandler>>no (source is Undeclared)
UIManager>>no (source is Undeclared)
UndefinedObject>>no (source is Undeclared)
CommandLineUIManager>>no (source is Undeclared)
SmalltalkImage>>no (source is Undeclared)
DelayMicrosecondScheduler>>no (source is Undeclared)
BlockClosure>>no (source is Undeclared)
SmalltalkImage>>no (source is Undeclared)
WeakArray class>>no (source is Undeclared)
ps: source cannot be displayed because there is no formatter available in the bootstrap
participants (6)
-
Brad Selfridge -
Christophe Demarey -
Craig Latta -
Guillermo Polito -
Peter H. Meadows -
phil＠highoctane.be