[Pharo-dev] Tell me about your workflow

Goubier Thierry thierry.goubier at cea.fr
Wed Dec 11 11:09:01 EST 2013


Yes, you're right in the general case.

But a solution to that general problem will take time to be implemented 
(time I lack at the moment, sadly) and if the main gain is a few % 
because it's writing the version file and the metadata for methods which 
are the "slow" factors, then we'll have worked hard for nothing.

If you want to help, I'd really like to see either 2- or 3- confirmed. I 
can produce a special gitfiletree to remove writing the metadata, that 
you can try on a large project temporary copy; if the slow writing (and 
reading) is confirmed, then this is 3-

(But I'm leaving on a trip tomorrow early, so I have no idea of when 
I'll have the time to do that :( ).

Thierry

Le 11/12/2013 16:44, Sebastian Sastre a écrit :
> Without entering in details, a cause for slow package write is dumping
> all every time.
>
> For that strategy, we already have the image save which is magically fast.
>
> So, if we make something to scan the code and write only when it's
> different from what's on disk, then we would be preventing tons of
> redundant writes
>
> sebastian <https://about.me/sebastianconcept>
>
> o/
>
>
>
>
>
> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <thierry.goubier at cea.fr
> <mailto:thierry.goubier at cea.fr>> wrote:
>
>>
>>
>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit :
>>> ah, and IMHO the problem is not about reading... is about writing (if it
>>> has to write the metadata each time...).
>>
>> But, personnaly, I don't know if this is the reason for the lack of
>> performance...
>>
>> I have three hypothesis for Sebastian problem:
>> 1 - Slow read time for version metadata
>>  - Confirmed because of the 16 seconds wait time for reading the
>> package metadata in the repository browser.
>> 2 - Slow metadata write
>> 3 - Slow package write
>>
>> I have an implemented solution for 1-, a very easy to implement for
>> 2-, and none yet for 3-
>>
>> So I'd really like to check if 3- is confirmed ;)
>>
>> Thierry
>>
>>>
>>> Esteban
>>>
>>>
>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano
>>> <estebanlm at gmail.com <mailto:estebanlm at gmail.com>
>>> <mailto:estebanlm at gmail.com>> wrote:
>>>
>>>    Thierry, I know there is a working version... let me search...
>>>
>>>    (5 mins later)
>>>
>>>
>>>    here:
>>>
>>> https://github.com/rjsargent/CypressReferenceImplementation
>>>
>>>    Dale says Richard made a metadata-less version.
>>>
>>>    We should take a look at that.
>>>
>>>    Esteban
>>>
>>>
>>>    On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry
>>>    <thierry.goubier at cea.fr
>>> <mailto:thierry.goubier at cea.fr><mailto:thierry.goubier at cea.fr>> wrote:
>>>
>>>        Esteban, Sebastian,
>>>
>>>        In the filetree code, you will find a format without metadata,
>>>        but it's not in use anymore.
>>>
>>>        If you use gitfiletree, it will write the metadata for
>>>        compatibility reasons with filetree, but it will never read it
>>> back.
>>>
>>>        I'm pushing code to make filetree robust to absence of metadata,
>>>        but I haven't worked on it for a while.
>>>
>>>        gitfiletree has solved the problem of a slow metadata read. It
>>>        does not solve any performance problem associated with
>>> writing, yet.
>>>
>>>        Thierry
>>>
>>>        Le 11/12/2013 16:12, Esteban Lorenzano a écrit :
>>>
>>>            I know there is a version of filetree without metadata (more
>>>            compelling
>>>            for projects that will never use other formats).
>>>            Dale told me that there was a preview somewhere, but I
>>>            didn't tested yet
>>>            (lack of time) and now I cannot find the mail...
>>>            Dale, can you re-send the link?
>>>
>>>            cheers,
>>>            Esteban
>>>
>>>
>>>            On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre
>>>            <sebastian at flowingconcept.com
>>> <mailto:sebastian at flowingconcept.com>
>>>            <mailto:sebastian at flowingconcept.com>
>>>            <mailto:sebastian at __flowingconcept.com
>>>            <mailto:sebastian at flowingconcept.com>>> wrote:
>>>
>>>                 I should breath before I type, but you probably already
>>>            got that I
>>>                 meant /redundant writes/ (not reads)...
>>>
>>>
>>>                 Anyway.. I was talking with Esteban and he mentions
>>>            some kind of
>>>                 compatibility metadata.
>>>
>>>                 If I'm going to give a leap of faith to filetree repos
>>>            to save code
>>>                 why should I care about mcz compatibility? Paying a
>>>            toll for no
>>>                 reason is evil.
>>>
>>>                 Maybe we could make that optional so those who don't
>>>            extract value
>>>                 from that feature can opt-out?
>>>
>>>                 sebastian <https://about.me/__sebastianconcept
>>>            <https://about.me/sebastianconcept>>
>>>
>>>
>>>                 o/
>>>
>>>
>>>
>>>
>>>
>>>                 On Dec 11, 2013, at 12:44 PM, Sebastian Sastre
>>>                 <sebastian at flowingconcept.com
>>> <mailto:sebastian at flowingconcept.com>
>>>            <mailto:sebastian at flowingconcept.com>
>>>            <mailto:sebastian at __flowingconcept.com
>>>            <mailto:sebastian at flowingconcept.com>>>
>>>                 wrote:
>>>
>>>                     Hi Thierry
>>>
>>>                     On Dec 11, 2013, at 12:43 PM, Goubier Thierry
>>>                     <thierry.goubier at cea.fr
>>> <mailto:thierry.goubier at cea.fr>
>>>                <mailto:thierry.goubier at cea.fr>
>>>                <mailto:thierry.goubier at cea.fr
>>>                <mailto:thierry.goubier at cea.fr>__>> wrote:
>>>
>>>
>>>                             I have packages (in the order of hundreds
>>>                        of classes) and save
>>>                             delays
>>>                             and package click delays are starting to
>>>                        demand patience in a
>>>                             way that
>>>                             doesn't feel like the right path
>>>
>>>
>>>                         Which operations ? I didn't remember noticing
>>>                    much with 179
>>>                         classes on a laptop without a SSD.
>>>
>>>
>>>                     choose one. Just for clicking the package that will
>>>                should you
>>>                     UUID, version and author I need to wait ~16
>>>                seconds. Sounds like a
>>>                     lot of overhead for reading a small .json file.
>>>
>>>                     But the write is the most worrisome
>>>
>>>
>>>                             All that is with a SSD disk, otherwise save
>>>                        delays would be
>>>                             /way/ beyond
>>>                             unacceptable
>>>
>>>
>>>                         I'd like to know more, and understand the
>>>                    reason, for sure. As
>>>                         far as I know, filetree will rewrite the whole
>>>                    package to disk
>>>                         everytime... and maybe optimising that could be
>>>                    the solution.
>>>
>>>
>>>                     Well, that explains a lot. Writing all every time
>>>                is the lazy
>>>                     thing that's okay for a prototype and temporary
>>>                code in a proof of
>>>                     concept but that massive redundant reads certainly
>>>                doesn't sounds
>>>                     like pro software. Specially for SSD's which has a
>>>                limited
>>>                     quantity of writes
>>>
>>>
>>>                         Thierry
>>>
>>>                             sebastian
>>>                        <https://about.me/__sebastianconcept
>>>                        <https://about.me/sebastianconcept>>
>>>
>>>                             o/
>>>
>>>
>>>
>>>
>>>
>>>
>>>                         --
>>>                         Thierry Goubier
>>>                         CEA list
>>>                         Laboratoire des Fondations des Systèmes Temps
>>>                    Réel Embarqués
>>>                         91191 Gif sur Yvette Cedex
>>>                         France
>>>                         Phone/Fax: +33 (0) 1 69 08 32 92
>>>                    <tel:%2B33%20%280%29%201%2069%2008%2032%2092>
>>>                         <tel:%2B33%20%280%29%201%2069%__2008%2032%2092>
>>>                    / 83 95
>>>
>>>
>>>
>>>
>>>
>>>
>>>        --
>>>        Thierry Goubier
>>>        CEA list
>>>        Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>        91191 Gif sur Yvette Cedex
>>>        France
>>>        Phone/Fax: +33 (0) 1 69 08 32 92
>>>        <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95
>>>
>>>
>>>
>>
>> --
>> Thierry Goubier
>> CEA list
>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>> 91191 Gif sur Yvette Cedex
>> France
>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
>

-- 
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95




More information about the Pharo-dev mailing list