[Pharo-dev] Tell me about your workflow

Sebastian Sastre sebastian at flowingconcept.com
Wed Dec 11 11:09:38 EST 2013


ok, if saving is dumping all, then 3 is confirmed? After the first commit, I'd say so.

about 2, I don't know. I'm available to make tests and measure results

have a nice trip, keep us tuned about any progress







On Dec 11, 2013, at 2:09 PM, Goubier Thierry <thierry.goubier at cea.fr> wrote:

> Yes, you're right in the general case.
> 
> But a solution to that general problem will take time to be implemented (time I lack at the moment, sadly) and if the main gain is a few % because it's writing the version file and the metadata for methods which are the "slow" factors, then we'll have worked hard for nothing.
> 
> If you want to help, I'd really like to see either 2- or 3- confirmed. I can produce a special gitfiletree to remove writing the metadata, that you can try on a large project temporary copy; if the slow writing (and reading) is confirmed, then this is 3-
> 
> (But I'm leaving on a trip tomorrow early, so I have no idea of when I'll have the time to do that :( ).
> 
> Thierry
> 
> Le 11/12/2013 16:44, Sebastian Sastre a écrit :
>> Without entering in details, a cause for slow package write is dumping
>> all every time.
>> 
>> For that strategy, we already have the image save which is magically fast.
>> 
>> So, if we make something to scan the code and write only when it's
>> different from what's on disk, then we would be preventing tons of
>> redundant writes
>> 
>> sebastian <https://about.me/sebastianconcept>
>> 
>> o/
>> 
>> 
>> 
>> 
>> 
>> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <thierry.goubier at cea.fr
>> <mailto:thierry.goubier at cea.fr>> wrote:
>> 
>>> 
>>> 
>>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit :
>>>> ah, and IMHO the problem is not about reading... is about writing (if it
>>>> has to write the metadata each time...).
>>> 
>>> But, personnaly, I don't know if this is the reason for the lack of
>>> performance...
>>> 
>>> I have three hypothesis for Sebastian problem:
>>> 1 - Slow read time for version metadata
>>> - Confirmed because of the 16 seconds wait time for reading the
>>> package metadata in the repository browser.
>>> 2 - Slow metadata write
>>> 3 - Slow package write
>>> 
>>> I have an implemented solution for 1-, a very easy to implement for
>>> 2-, and none yet for 3-
>>> 
>>> So I'd really like to check if 3- is confirmed ;)
>>> 
>>> Thierry
>>> 
>>>> 
>>>> Esteban
>>>> 
>>>> 
>>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano
>>>> <estebanlm at gmail.com <mailto:estebanlm at gmail.com>
>>>> <mailto:estebanlm at gmail.com>> wrote:
>>>> 
>>>>   Thierry, I know there is a working version... let me search...
>>>> 
>>>>   (5 mins later)
>>>> 
>>>> 
>>>>   here:
>>>> 
>>>> https://github.com/rjsargent/CypressReferenceImplementation
>>>> 
>>>>   Dale says Richard made a metadata-less version.
>>>> 
>>>>   We should take a look at that.
>>>> 
>>>>   Esteban
>>>> 
>>>> 
>>>>   On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry
>>>>   <thierry.goubier at cea.fr
>>>> <mailto:thierry.goubier at cea.fr><mailto:thierry.goubier at cea.fr>> wrote:
>>>> 
>>>>       Esteban, Sebastian,
>>>> 
>>>>       In the filetree code, you will find a format without metadata,
>>>>       but it's not in use anymore.
>>>> 
>>>>       If you use gitfiletree, it will write the metadata for
>>>>       compatibility reasons with filetree, but it will never read it
>>>> back.
>>>> 
>>>>       I'm pushing code to make filetree robust to absence of metadata,
>>>>       but I haven't worked on it for a while.
>>>> 
>>>>       gitfiletree has solved the problem of a slow metadata read. It
>>>>       does not solve any performance problem associated with
>>>> writing, yet.
>>>> 
>>>>       Thierry
>>>> 
>>>>       Le 11/12/2013 16:12, Esteban Lorenzano a écrit :
>>>> 
>>>>           I know there is a version of filetree without metadata (more
>>>>           compelling
>>>>           for projects that will never use other formats).
>>>>           Dale told me that there was a preview somewhere, but I
>>>>           didn't tested yet
>>>>           (lack of time) and now I cannot find the mail...
>>>>           Dale, can you re-send the link?
>>>> 
>>>>           cheers,
>>>>           Esteban
>>>> 
>>>> 
>>>>           On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre
>>>>           <sebastian at flowingconcept.com
>>>> <mailto:sebastian at flowingconcept.com>
>>>>           <mailto:sebastian at flowingconcept.com>
>>>>           <mailto:sebastian at __flowingconcept.com
>>>>           <mailto:sebastian at flowingconcept.com>>> wrote:
>>>> 
>>>>                I should breath before I type, but you probably already
>>>>           got that I
>>>>                meant /redundant writes/ (not reads)...
>>>> 
>>>> 
>>>>                Anyway.. I was talking with Esteban and he mentions
>>>>           some kind of
>>>>                compatibility metadata.
>>>> 
>>>>                If I'm going to give a leap of faith to filetree repos
>>>>           to save code
>>>>                why should I care about mcz compatibility? Paying a
>>>>           toll for no
>>>>                reason is evil.
>>>> 
>>>>                Maybe we could make that optional so those who don't
>>>>           extract value
>>>>                from that feature can opt-out?
>>>> 
>>>>                sebastian <https://about.me/__sebastianconcept
>>>>           <https://about.me/sebastianconcept>>
>>>> 
>>>> 
>>>>                o/
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>                On Dec 11, 2013, at 12:44 PM, Sebastian Sastre
>>>>                <sebastian at flowingconcept.com
>>>> <mailto:sebastian at flowingconcept.com>
>>>>           <mailto:sebastian at flowingconcept.com>
>>>>           <mailto:sebastian at __flowingconcept.com
>>>>           <mailto:sebastian at flowingconcept.com>>>
>>>>                wrote:
>>>> 
>>>>                    Hi Thierry
>>>> 
>>>>                    On Dec 11, 2013, at 12:43 PM, Goubier Thierry
>>>>                    <thierry.goubier at cea.fr
>>>> <mailto:thierry.goubier at cea.fr>
>>>>               <mailto:thierry.goubier at cea.fr>
>>>>               <mailto:thierry.goubier at cea.fr
>>>>               <mailto:thierry.goubier at cea.fr>__>> wrote:
>>>> 
>>>> 
>>>>                            I have packages (in the order of hundreds
>>>>                       of classes) and save
>>>>                            delays
>>>>                            and package click delays are starting to
>>>>                       demand patience in a
>>>>                            way that
>>>>                            doesn't feel like the right path
>>>> 
>>>> 
>>>>                        Which operations ? I didn't remember noticing
>>>>                   much with 179
>>>>                        classes on a laptop without a SSD.
>>>> 
>>>> 
>>>>                    choose one. Just for clicking the package that will
>>>>               should you
>>>>                    UUID, version and author I need to wait ~16
>>>>               seconds. Sounds like a
>>>>                    lot of overhead for reading a small .json file.
>>>> 
>>>>                    But the write is the most worrisome
>>>> 
>>>> 
>>>>                            All that is with a SSD disk, otherwise save
>>>>                       delays would be
>>>>                            /way/ beyond
>>>>                            unacceptable
>>>> 
>>>> 
>>>>                        I'd like to know more, and understand the
>>>>                   reason, for sure. As
>>>>                        far as I know, filetree will rewrite the whole
>>>>                   package to disk
>>>>                        everytime... and maybe optimising that could be
>>>>                   the solution.
>>>> 
>>>> 
>>>>                    Well, that explains a lot. Writing all every time
>>>>               is the lazy
>>>>                    thing that's okay for a prototype and temporary
>>>>               code in a proof of
>>>>                    concept but that massive redundant reads certainly
>>>>               doesn't sounds
>>>>                    like pro software. Specially for SSD's which has a
>>>>               limited
>>>>                    quantity of writes
>>>> 
>>>> 
>>>>                        Thierry
>>>> 
>>>>                            sebastian
>>>>                       <https://about.me/__sebastianconcept
>>>>                       <https://about.me/sebastianconcept>>
>>>> 
>>>>                            o/
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>                        --
>>>>                        Thierry Goubier
>>>>                        CEA list
>>>>                        Laboratoire des Fondations des Systèmes Temps
>>>>                   Réel Embarqués
>>>>                        91191 Gif sur Yvette Cedex
>>>>                        France
>>>>                        Phone/Fax: +33 (0) 1 69 08 32 92
>>>>                   <tel:%2B33%20%280%29%201%2069%2008%2032%2092>
>>>>                        <tel:%2B33%20%280%29%201%2069%__2008%2032%2092>
>>>>                   / 83 95
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>       --
>>>>       Thierry Goubier
>>>>       CEA list
>>>>       Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>       91191 Gif sur Yvette Cedex
>>>>       France
>>>>       Phone/Fax: +33 (0) 1 69 08 32 92
>>>>       <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95
>>>> 
>>>> 
>>>> 
>>> 
>>> --
>>> Thierry Goubier
>>> CEA list
>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>> 91191 Gif sur Yvette Cedex
>>> France
>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
>> 
> 
> -- 
> Thierry Goubier
> CEA list
> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
> 91191 Gif sur Yvette Cedex
> France
> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20131211/d9bffd04/attachment-0002.html>


More information about the Pharo-dev mailing list