[Pharo-dev] [IMPORTANT] Following changes in the bootstrapping process

Eliot Miranda eliot.miranda at gmail.com
Tue Aug 1 16:25:28 EDT 2017


Hi Pablo,

On Tue, Aug 1, 2017 at 12:36 PM, tesonep at gmail.com <tesonep at gmail.com>
wrote:

> Hi again,
>    thanks for the clarification, I haven't understood the question
> initially, but now I think I can answer it.
>
> Hermes is only exporting the classes, not objects. so it does not have to
> handle complex graphs of objects. Basically it serializes the definition of
> the classes, traits and methods. Then they are loaded in sequence. The only
> caring during the generation is to serialize first the superclasses, and
> then the subclasses. There is no way of serializing objects outside the
> classes, methods, traits and literals in a method.
>

Classes, traits, methods, literals *are* a graph of objects :-). The Parcel
architecture, from which Fuel derived its architecture, was designed for
loading code in VisualWorks.  In fact, last time I looked Parcels were used
only to store code and not as a general purpose (de)serializer.

So the answer is that is not using Parcels or Fuel architecture because
> they are intended for different uses.
>

Well, I don't think that's the reason ;-), but fine.  I understand that it
doesn't use this architecture.  Thanks.


> Also the format itself has no special design or considerations to be fast
> to generate or fast to read (as It happens with Fuel)
>
> Cheers,
> Pablo
>
> On Tue, Aug 1, 2017 at 9:16 PM, Eliot Miranda <eliot.miranda at gmail.com>
> wrote:
>
>> Hi Pablo,
>>
>>     I understand that Hermes has its own format.  My question was about
>> architecture. The Parcels/Fuel architecture is, AFAIA, the best
>> architecture for binary storage because it is virtual, is fast to load and
>> cleanly separates loading from initialization.  What do I mean?
>>
>> - concrete formats like ImageSegment are tied to internal representations
>> in the VM and do are difficult to read in forever FB systems and very hard
>> to write in foreign systems.  So a virtual format (like BOSS, Fuel,
>> Parcels, etc) is better but unless done well can be slow
>>
>> - a first generation system like BOSS is a flattening of a graph
>> traversal using a simple grammar.  Each phrase is either an object
>> reference (object id) or an object definition (object is followed by type
>> info and contents, contents being a sequence of phrases.  This is slow to
>> parse (because each phrase must be decoded), and has invalid intermediate
>> states (e.g. a Set whose contents are not all present, hence whose external
>> hash may change during loading, etc).  A second generation system like
>> Parcels and Fuel separates objects from references (nodes from arcs) and
>> batches object creation, grouping all instances of a given class for bulk,
>> and hence fast, instantiation.  Following my the objects are the
>> references.  So loading has three phases:
>> - instantiate the objects
>> - connect the graph by filling in object references
>> - initialize the objects that require it (e.g. rehash sets)
>>
>> Consequently the writer must be two pass, the first pass to collect the
>> objects in the graph and sort them by class, the second to write the nodes
>> followed by the references
>>
>> So let me ask again, does Hermes use the Parcels/Fuel architecture?
>>
>> _,,,^..^,,,_ (phone)
>>
>> On Aug 1, 2017, at 11:52 AM, "tesonep at gmail.com" <tesonep at gmail.com>
>> wrote:
>>
>> Hi Eliot,
>>
>> The last version of Hermes, that I have generated today and have to be
>> tested for real yet, is able to load and save from and to 32 and 64 bits,
>> handling the conversion. Because the Hermes representation (if we can call
>> it like that, because is really simple, does not care about the
>> architecture).
>>
>> Hermes is using a custom format completely separated from Fuel or any
>> other tool.
>>
>> This is a consequence of the design of Hermes. Hermes has been designed
>> to only work as light way of loading code during the bootstrap. So that, it
>> is heavily coupled with Pharo (it can be changed, for example implementing
>> another loader / installer) and uses only really core classes.
>>
>> Of course it can be extended and used in another Smalltalk dialects, but
>> I am not sure if the tool can be useful for them, or even for the Pharo
>> community outside the Pharo bootstrap process.
>>
>> Today to be useful outside the bootstrap process, some work has to be
>> done (perhaps a lot), but mainly new use cases have to be thinked. Once the
>> Compiler, Monticello or Metacello is available, there is no need for
>> Hermes.  It is only used to generate some binary loadable packages for the
>> compiler from the source code.
>>
>> Again, it was only thinked as a tool to improve the speed during the
>> bootstrap. However, if somebody has ideas to use it or want to collaborate
>> are  of course welcomed.
>>
>> Hermes is actually in Github, because to me is more comfortable to have
>> it there,  but if somebody is really wanting to use it.... we can manage to
>> have an export process to Smalltalkhub.
>>
>> I hope I have clarified a little the idea behind Hermes.
>>
>> Maybe you have better ideas of what can we do with it.
>>
>>
>> On Tue, Aug 1, 2017 at 7:59 PM, Eliot Miranda <eliot.miranda at gmail.com>
>> wrote:
>>
>>> Hi Guille,
>>>
>>> On Aug 1, 2017, at 4:29 AM, Guillermo Polito <guillermopolito at gmail.com>
>>> wrote:
>>>
>>>
>>>
>>> On Tue, Aug 1, 2017 at 12:57 PM, philippe.back at highoctane.be <
>>> philippe.back at gmail.com> wrote:
>>>
>>>> Massive.
>>>>
>>>> What is in the super small image?
>>>> Is Hermes going to be a generic binary content loader?
>>>>
>>>
>>> There is still work to do in this front.
>>>  - Hermes only works for 32bits format. We should adapt it to 64 bits
>>> (immediate floats and so on...)
>>>  - There is no format validation. We should add one for safety.
>>>
>>>
>>> Will Hermes be able to save on 32-bits and load on 64-bits and vice
>>> verse?
>>>
>>> Does Hermes use the Parcels/Fuel architecture of saving nodes, grouped
>>> by class, followed by the arcs?
>>>
>>> If yes to both of these, are you willing to keep it in Monticello and
>>> collaborate with the Squeak & Cuis communities in developing Hermes?
>>>
>>>
>>>
>>>> Phil
>>>>
>>>> On Aug 1, 2017 12:36, "Stephane Ducasse" <stepharo.self at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Pavel
>>>>>
>>>>> This is super excellent! IMPRESSIVE. An image without the compiler and
>>>>> a reloadable compiler.
>>>>> Super cool.
>>>>>
>>>>> Stef
>>>>>
>>>>>
>>>>> On Tue, Aug 1, 2017 at 11:57 AM, Pavel Krivanek
>>>>> <pavel.krivanek at gmail.com> wrote:
>>>>> > Hello,
>>>>> >
>>>>> > we are checking a huge pull request #177
>>>>> > (https://github.com/pharo-project/pharo/pull/177) that will change
>>>>> some
>>>>> > basics of the bootstrap process:
>>>>> >
>>>>> > Now we will bootstrap a smaller image that will not include
>>>>> compiler/parser.
>>>>> > Compiler and related packages are exported and loaded using a binary
>>>>> > exporter named Hermes.
>>>>> > The compiler is then used to load FileSystem and Monticello. The
>>>>> rest of the
>>>>> > bootstrap process will be the same as before.
>>>>> > As the result we will have faster bootstrap and better system
>>>>> modularization
>>>>> > and possibilities.
>>>>> >
>>>>> > It required some modularization efforts:
>>>>> >
>>>>> > - simplified initialization scripts
>>>>> > - Use Zinc converters and encoders instead of FilePathEncoder and old
>>>>> > TextConverter
>>>>> > - Use Stdio instead of FileStream
>>>>> > - Using File instead of FileSystem
>>>>> > - Deprecated FileStream & childs (Moved to Deprecated70)
>>>>> > - Extracted Path classes to their on package: FileSystem-Path
>>>>> > - Moved OpalEncoders to their own package. They are required by the
>>>>> runtime
>>>>> > (not only for compilation)
>>>>> > - Introduced AsciiCharset in the kernel to answer to #isLetter
>>>>> #isUppercase
>>>>> > and so on without requiring full Unicode tables from the beginning
>>>>> > - Cleaning up a bit the full exception logging infrastructure
>>>>> (streams,
>>>>> > transcript, files, stack printing...)
>>>>> > - Split Ring methods required for system navigation to the
>>>>> Ring-Navigation
>>>>> > package
>>>>> > - Remove usages of #translated in the kernel
>>>>> > - Refactored the bootstrapping classes to remove duplications
>>>>> > - Cleaning up dependencies in CompiledMethod>>printOn:
>>>>> > - fix path printing
>>>>> >
>>>>> > We need to merge these changes at once and of course it can cause
>>>>> some
>>>>> > conflicts with the existing pull requests or external code. Anyway,
>>>>> we need
>>>>> > to merge it as soon as possible.
>>>>> >
>>>>> > So please, try to look at the PR and test the resultant image [1] to
>>>>> avoid
>>>>> > some major problems.
>>>>> >
>>>>> > [1]
>>>>> > https://ci.inria.fr/pharo/view/7.0/job/70-PR-Check-Load/last
>>>>> SuccessfulBuild/artifact/bootstrap-cache/Pharo7.0-32bit-9c0691d.zip
>>>>> >
>>>>> > Cheers,
>>>>> > -- Pavel
>>>>> >
>>>>>
>>>>>
>>> --
>>>
>>>
>>>
>>> Guille Polito
>>>
>>>
>>> Research Engineer
>>>
>>> French National Center for Scientific Research - *http://www.cnrs.fr*
>>> <http://www.cnrs.fr>
>>>
>>>
>>>
>>> *Web:* *http://guillep.github.io* <http://guillep.github.io>
>>>
>>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>>>
>>>
>>
>>
>> --
>> Pablo Tesone.
>> tesonep at gmail.com
>>
>>
>
>
> --
> Pablo Tesone.
> tesonep at gmail.com
>



-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20170801/0b46415e/attachment.html>


More information about the Pharo-dev mailing list