[Pharo-users] Understanding the role of the sources file

Sven Van Caekenberghe sven at stfx.eu
Wed Jan 13 10:46:48 EST 2016


I like Ben's version too. (It helps to be a native English speaker as well ;-)

Please write more, Ben.

> On 13 Jan 2016, at 16:41, Dimitris Chloupis <kilon.alios at gmail.com> wrote:
> 
> great explanation Ben thanks for sharing, I agree 100% . ISO files are popular file formats used by other VMs , like VirtualBOX unlike pharo image they are not a memory dump but rather a hard disk dump which is later loaded in memory usually to emulate booting from a DVD or booting a HD drive
> 
> https://en.wikipedia.org/wiki/ISO_image
> 
> Of course because in the pharo image we store every live object that means everything the similarity in how both formats are used is very similar way. 
> 
> On Wed, Jan 13, 2016 at 5:31 PM Ben Coman <btc at openinworld.com> wrote:
> Oh! I should have guessed there'd be more alternative suggestions and held back.
> Anyway, its done now, so just pick out what you like...
> Now I tend to think of the Image as more than just the .image file, so...
> 
> The heart of Pharo is the *Image*.  This holds the live running state
> of a Pharo system. Tools like the IDE, language parser and interactive
> playground run *inside* the Image to write code and create interacting
> objects which comprise your application.  Saving the Image to a
> ".image" file creates a frozen-in-time snapshot of the running system
> state. This can be moved to another system and resumed running as if
> the Image had never stopped.  The Pharo download provides
> /Pharo.image/ file as the starting point for the live environment that
> you adapt to your needs.
> 
> The Image runs on a Virtual Machine.  This abstracts away operating
> system and CPU architecture differences to allow the same ".image"
> file (the snapshot of you live system) to open and run on any support
> platform. The VM executable file is a different for each platform:
> * pharo for Linux and Unix
> * Pharo.exe for Windows
> * Pharo for OSX
> 
> Your source-code is compiled to byte-code in Image to become part of
> the live system state.  This occurs as soon as a modified method is
> accepted.  Only the modified method is compiled, not the whole source
> code, and the bytecode is immediately runnable.  This facilitates a
> very fast edit-compile-run-debug loop at the root of Pharo's
> productivity.   The VM optimally interprets or just-in-time compiles
> the bytecode as appropriate for best performance.
> 
> As an implementation detail carried over from when ram was scarce, the
> source-code is not stored in-Image.  It is spread across two files:
> 
> * Sources for parts of Pharo (i.e. the tools) that don’t change
> frequently are stored in the "*.sources" files.   It is generated as a
> static file per major release. Thus for  Pharo 4.0 it is named
> PharoV40.sources, and keeps that name when the image file name
> changes.   The *.sources file is not essential for running the Image,
> but is important because without it, you can't examine and learn from
> the implementation of Pharo's tools (e.g Collections, Graphics
> libraries, IDE and compiler.)
> 
> * All source code changes are journal logged to the "*.changes" file.
> This facilitates easy access to per method history for diffs or
> reverting.  Further, if you close or crash your image without saving,
> an in-Image tool can replay your "lost" changes from this file. The
> "*.changes" file is tightly coupled to the "*.image" file because it
> records the source code changes for a particular Image.  The basename
> of these two files must always be the same (e.g. my.image &
> my.changes).  Each release provides a near empty file, for example
> Pharo4.changes to match Pharo4.image.
> 
> Typically the VM and .sources files are distributed/stored together,
> since both are static files (per machine per Pharo Release) and both
> can be shared between multiple Images; and the dynamic .image and
> .changes files are distributed/stored together.
> 
> 
> 
> 
> > The other components below are portable across operating systems, and
> 
> 
> The image file provides a frozen in time snapshot of a running Pharo
> > system. This is the file where the Pharo bytecode is stored and as such its
> > a cross platform format. This is the heart of Pharo, containing the live
> > state of all objects in the system (including classes and methods, since
> > they are objects too). The file is named for the release (like
> > Pharo4.0.image).
> >
> 
> 
> 
> On Wed, Jan 13, 2016 at 7:58 PM, Dimitris Chloupis
> <kilon.alios at gmail.com> wrote:
> > So I am correct that the image does not store the source code, and that the
> > source code is stored in sources and changes. The only diffirence is that
> > the objects have a source variable that points to the right place for
> > finding the source code.
> >
> > This is the final text if you find anything incorrect please correct me
> >
> > ---------------
> >
> > 1. The virtual machine (VM) is the only component that is different for each
> > operating system. The purpose of the VM is to take Pharo bytcode that is
> > generated each time user accepts a piece of code and convert it to machine
> > code in order to be executed. Pharo 4 comes with the Cog VM a very fast JIT
> > VM. The VM executable is named:
> >
> > • Pharo.exe for Windows; • pharo for Linux ; and
> >
> > • Pharo for OSX (inside a package also named Pharo.app).
> > The other components below are portable across operating systems, and
> >
> > can be copied and run on any appropriate virtual machine.
> >
> > 2. The sources file contains source code for parts of Pharo that don’t
> > change frequently. Sources file is important because the image file format
> > stores only the bytecode of live objects and not their source code.
> > Typically a new sources file is generated once per major release of Pharo.
> > For Pharo 4.0, this file is named PharoV40.sources.
> >
> > 3. The changes file logs of all source code modifications since the .sources
> > file was generated. This facilitates a per method history for diffs or re-
> > verting.That means that even if you dont manage to save the image file on a
> > crash or you just forgot you can recover your changes from this file. Each
> > release provides a near empty file named for the release, for example
> > Pharo4.0.changes.
> >
> > 4. The image file provides a frozen in time snapshot of a running Pharo
> > system. This is the file where the Pharo bytecode is stored and as such its
> > a cross platform format. This is the heart of Pharo, containing the live
> > state of all objects in the system (including classes and methods, since
> > they are objects too). The file is named for the release (like
> > Pharo4.0.image).
> >
> > The .image and .changes files provided by a Pharo release are the starting
> > point for a live environment that you adapt to your needs. Essentially the
> > image file containes the compiler of the language (not the VM) , the
> > language parser, the IDE tools, many libraries and acts a bit like a virtual
> > Operation System that runs on top of a Virtual Machine (VM), similarly to
> > ISO files.
> 
> I don't get this similarity with ISO files.
> 
> >
> > As you work in Pharo, these files are modified, so you need to make sure
> > that they are writable. The .image and .changes files are intimately linked
> > and should always be kept together, with matching base filenames. Never edit
> > them directly with a text editor, as .images holds your live object runtime
> > memory, which indexes into the .changes files for the source. It is a good
> > idea to keep a backup copy of the downloaded .image and .changes files so
> > you can always start from a fresh image and reload your code. However the
> > most efficient way for backing up code is to use a version control system
> > that will provide an easier and powerful way to back up and track your
> > changes.
> >
> > The four main component files above can be placed in the same directory,
> > although it’s also possible to put the Virtual Machine and sources file in a
> > separate directory where everyone has read-only access to them.
> >
> > If more than one image file is present in the same directory pharo will
> > prompt you to choose an image file you want to load.
> >
> > Do whatever works best for your style of working and your operating system.
> >
> >
> >
> >
> >
> > On Wed, Jan 13, 2016 at 12:13 PM Sven Van Caekenberghe <sven at stfx.eu> wrote:
> >>
> >>
> >> > On 13 Jan 2016, at 10:57, Dimitris Chloupis <kilon.alios at gmail.com>
> >> > wrote:
> >> >
> >> > I was adding a short description to the UPBE about sources file , I
> >> > always thought that the sources file is the file that contains the source
> >> > code of the image because the image file itself stores only the bytecode.
> >> >
> >> > However its just came to my attention that the sources file does not
> >> > contain code that is recently installed in the image.
> >> >
> >> > So how exactly the sources file works and what it is ?
> >>
> >> The main perspective is from the object point of view: methods are just
> >> objects like everything else. In order to be executable they know their byte
> >> codes (which might be JIT compiled on execution, but that is an
> >> implementation detail) and they know their source code.
> >>
> >> Today we would probably just store the source code strings in the image
> >> (maybe compressed) as memory is pretty cheap. But way back when Smalltalk
> >> started, that was not the case. So they decided to map the source code out
> >> to files.
> >>
> >> So method source code is a magic string (RemoteString) that points to some
> >> position in a file. There are 2 files in use: the sources file and the
> >> changes file.
> >>
> >> The sources file is a kind of snapshot of the source code of all methods
> >> at the point of release of a major new version. That is why there is a Vxy
> >> in their name. The source file never changes once created or renewed (a
> >> process called generating the sources, see PharoSourcesCondenser).
> >>
> >> While developing and creating new versions of methods, the new source code
> >> is appended to another file called the changes file, much like a transaction
> >> log. This is also a safety mechanism to recover 'lost' changes.
> >>
> >> The changes file can contain multiple versions of a method. This can be
> >> reduced in size using a process called condensing the changes, see
> >> PharoChangesCondenser.
> >>
> >> On a new release, the changes file will be (almost) empty.
> >>
> >> HTH,
> >>
> >> Sven
> >>
> >>
> >>
> >
> 





More information about the Pharo-users mailing list