[Pharo-dev] Tell me about your workflow

Sebastian Sastre sebastian at flowingconcept.com
Thu Dec 12 06:12:03 EST 2013


gee the big code package is airflowing which I have, quite conservatively, running on #14438 images

 I load filetree like this:

Gofer new
      url: 'http://ss3.gemstone.com/ss/FileTree';
      package: 'ConfigurationOfFileTree';
      load.
((Smalltalk at: #ConfigurationOfFileTree) project version: #'stable') load.

and it never complained

let me know 

 



On Dec 12, 2013, at 3:53 AM, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:

> If you would be ready to profile a package save on your repository, it would be great. In the mean time, I'll make available a special gitfiletree package to test. Which version of Pharo you are using? 2.0 or 3.0?
> 
> Regards,
> 
> Thierry
> 
> 
> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Sebastian Sastre [sebastian at flowingconcept.com]
> Date d'envoi : mercredi 11 décembre 2013 17:09
> À : Pharo Development List
> Objet : Re: [Pharo-dev] Tell me about your workflow
> 
> ok, if saving is dumping all, then 3 is confirmed? After the first commit, I'd say so.
> 
> about 2, I don't know. I'm available to make tests and measure results
> 
> have a nice trip, keep us tuned about any progress
> 
> 
> 
> 
> 
> 
> 
> On Dec 11, 2013, at 2:09 PM, Goubier Thierry <thierry.goubier at cea.fr> wrote:
> 
>> Yes, you're right in the general case.
>> 
>> But a solution to that general problem will take time to be implemented (time I lack at the moment, sadly) and if the main gain is a few % because it's writing the version file and the metadata for methods which are the "slow" factors, then we'll have worked hard for nothing.
>> 
>> If you want to help, I'd really like to see either 2- or 3- confirmed. I can produce a special gitfiletree to remove writing the metadata, that you can try on a large project temporary copy; if the slow writing (and reading) is confirmed, then this is 3-
>> 
>> (But I'm leaving on a trip tomorrow early, so I have no idea of when I'll have the time to do that :( ).
>> 
>> Thierry
>> 
>> Le 11/12/2013 16:44, Sebastian Sastre a écrit :
>>> Without entering in details, a cause for slow package write is dumping
>>> all every time.
>>> 
>>> For that strategy, we already have the image save which is magically fast.
>>> 
>>> So, if we make something to scan the code and write only when it's
>>> different from what's on disk, then we would be preventing tons of
>>> redundant writes
>>> 
>>> sebastian <https://about.me/sebastianconcept>
>>> 
>>> o/
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <thierry.goubier at cea.fr
>>> <mailto:thierry.goubier at cea.fr>> wrote:
>>> 
>>>> 
>>>> 
>>>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit :
>>>>> ah, and IMHO the problem is not about reading... is about writing (if it
>>>>> has to write the metadata each time...).
>>>> 
>>>> But, personnaly, I don't know if this is the reason for the lack of
>>>> performance...
>>>> 
>>>> I have three hypothesis for Sebastian problem:
>>>> 1 - Slow read time for version metadata
>>>> - Confirmed because of the 16 seconds wait time for reading the
>>>> package metadata in the repository browser.
>>>> 2 - Slow metadata write
>>>> 3 - Slow package write
>>>> 
>>>> I have an implemented solution for 1-, a very easy to implement for
>>>> 2-, and none yet for 3-
>>>> 
>>>> So I'd really like to check if 3- is confirmed ;)
>>>> 
>>>> Thierry
>>>> 
>>>>> 
>>>>> Esteban
>>>>> 
>>>>> 
>>>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano
>>>>> <estebanlm at gmail.com <mailto:estebanlm at gmail.com>
>>>>> <mailto:estebanlm at gmail.com>> wrote:
>>>>> 
>>>>>   Thierry, I know there is a working version... let me search...
>>>>> 
>>>>>   (5 mins later)
>>>>> 
>>>>> 
>>>>>   here:
>>>>> 
>>>>> https://github.com/rjsargent/CypressReferenceImplementation
>>>>> 
>>>>>   Dale says Richard made a metadata-less version.
>>>>> 
>>>>>   We should take a look at that.
>>>>> 
>>>>>   Esteban
>>>>> 
>>>>> 
>>>>>   On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry
>>>>>   <thierry.goubier at cea.fr
>>>>> <mailto:thierry.goubier at cea.fr><mailto:thierry.goubier at cea.fr>> wrote:
>>>>> 
>>>>>       Esteban, Sebastian,
>>>>> 
>>>>>       In the filetree code, you will find a format without metadata,
>>>>>       but it's not in use anymore.
>>>>> 
>>>>>       If you use gitfiletree, it will write the metadata for
>>>>>       compatibility reasons with filetree, but it will never read it
>>>>> back.
>>>>> 
>>>>>       I'm pushing code to make filetree robust to absence of metadata,
>>>>>       but I haven't worked on it for a while.
>>>>> 
>>>>>       gitfiletree has solved the problem of a slow metadata read. It
>>>>>       does not solve any performance problem associated with
>>>>> writing, yet.
>>>>> 
>>>>>       Thierry
>>>>> 
>>>>>       Le 11/12/2013 16:12, Esteban Lorenzano a écrit :
>>>>> 
>>>>>           I know there is a version of filetree without metadata (more
>>>>>           compelling
>>>>>           for projects that will never use other formats).
>>>>>           Dale told me that there was a preview somewhere, but I
>>>>>           didn't tested yet
>>>>>           (lack of time) and now I cannot find the mail...
>>>>>           Dale, can you re-send the link?
>>>>> 
>>>>>           cheers,
>>>>>           Esteban
>>>>> 
>>>>> 
>>>>>           On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre
>>>>>           <sebastian at flowingconcept.com
>>>>> <mailto:sebastian at flowingconcept.com>
>>>>>           <mailto:sebastian at flowingconcept.com>
>>>>>           <mailto:sebastian at __flowingconcept.com
>>>>>           <mailto:sebastian at flowingconcept.com>>> wrote:
>>>>> 
>>>>>                I should breath before I type, but you probably already
>>>>>           got that I
>>>>>                meant /redundant writes/ (not reads)...
>>>>> 
>>>>> 
>>>>>                Anyway.. I was talking with Esteban and he mentions
>>>>>           some kind of
>>>>>                compatibility metadata.
>>>>> 
>>>>>                If I'm going to give a leap of faith to filetree repos
>>>>>           to save code
>>>>>                why should I care about mcz compatibility? Paying a
>>>>>           toll for no
>>>>>                reason is evil.
>>>>> 
>>>>>                Maybe we could make that optional so those who don't
>>>>>           extract value
>>>>>                from that feature can opt-out?
>>>>> 
>>>>>                sebastian <https://about.me/__sebastianconcept
>>>>>           <https://about.me/sebastianconcept>>
>>>>> 
>>>>> 
>>>>>                o/
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>                On Dec 11, 2013, at 12:44 PM, Sebastian Sastre
>>>>>                <sebastian at flowingconcept.com
>>>>> <mailto:sebastian at flowingconcept.com>
>>>>>           <mailto:sebastian at flowingconcept.com>
>>>>>           <mailto:sebastian at __flowingconcept.com
>>>>>           <mailto:sebastian at flowingconcept.com>>>
>>>>>                wrote:
>>>>> 
>>>>>                    Hi Thierry
>>>>> 
>>>>>                    On Dec 11, 2013, at 12:43 PM, Goubier Thierry
>>>>>                    <thierry.goubier at cea.fr
>>>>> <mailto:thierry.goubier at cea.fr>
>>>>>               <mailto:thierry.goubier at cea.fr>
>>>>>               <mailto:thierry.goubier at cea.fr
>>>>>               <mailto:thierry.goubier at cea.fr>__>> wrote:
>>>>> 
>>>>> 
>>>>>                            I have packages (in the order of hundreds
>>>>>                       of classes) and save
>>>>>                            delays
>>>>>                            and package click delays are starting to
>>>>>                       demand patience in a
>>>>>                            way that
>>>>>                            doesn't feel like the right path
>>>>> 
>>>>> 
>>>>>                        Which operations ? I didn't remember noticing
>>>>>                   much with 179
>>>>>                        classes on a laptop without a SSD.
>>>>> 
>>>>> 
>>>>>                    choose one. Just for clicking the package that will
>>>>>               should you
>>>>>                    UUID, version and author I need to wait ~16
>>>>>               seconds. Sounds like a
>>>>>                    lot of overhead for reading a small .json file.
>>>>> 
>>>>>                    But the write is the most worrisome
>>>>> 
>>>>> 
>>>>>                            All that is with a SSD disk, otherwise save
>>>>>                       delays would be
>>>>>                            /way/ beyond
>>>>>                            unacceptable
>>>>> 
>>>>> 
>>>>>                        I'd like to know more, and understand the
>>>>>                   reason, for sure. As
>>>>>                        far as I know, filetree will rewrite the whole
>>>>>                   package to disk
>>>>>                        everytime... and maybe optimising that could be
>>>>>                   the solution.
>>>>> 
>>>>> 
>>>>>                    Well, that explains a lot. Writing all every time
>>>>>               is the lazy
>>>>>                    thing that's okay for a prototype and temporary
>>>>>               code in a proof of
>>>>>                    concept but that massive redundant reads certainly
>>>>>               doesn't sounds
>>>>>                    like pro software. Specially for SSD's which has a
>>>>>               limited
>>>>>                    quantity of writes
>>>>> 
>>>>> 
>>>>>                        Thierry
>>>>> 
>>>>>                            sebastian
>>>>>                       <https://about.me/__sebastianconcept
>>>>>                       <https://about.me/sebastianconcept>>
>>>>> 
>>>>>                            o/
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>                        --
>>>>>                        Thierry Goubier
>>>>>                        CEA list
>>>>>                        Laboratoire des Fondations des Systèmes Temps
>>>>>                   Réel Embarqués
>>>>>                        91191 Gif sur Yvette Cedex
>>>>>                        France
>>>>>                        Phone/Fax: +33 (0) 1 69 08 32 92
>>>>>                   <tel:%2B33%20%280%29%201%2069%2008%2032%2092>
>>>>>                        <tel:%2B33%20%280%29%201%2069%__2008%2032%2092>
>>>>>                   / 83 95
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>       --
>>>>>       Thierry Goubier
>>>>>       CEA list
>>>>>       Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>>       91191 Gif sur Yvette Cedex
>>>>>       France
>>>>>       Phone/Fax: +33 (0) 1 69 08 32 92
>>>>>       <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> --
>>>> Thierry Goubier
>>>> CEA list
>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>> 91191 Gif sur Yvette Cedex
>>>> France
>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
>>> 
>> 
>> -- 
>> Thierry Goubier
>> CEA list
>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>> 91191 Gif sur Yvette Cedex
>> France
>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20131212/8ec793e6/attachment-0002.html>


More information about the Pharo-dev mailing list