[Pharo-dev] Tell me about your workflow

Stéphane Ducasse stephane.ducasse at inria.fr
Fri Dec 13 07:47:24 EST 2013


try with Moose.

Stef

On Dec 12, 2013, at 4:20 PM, Yuriy Tymchuk <yuriy.tymchuk at me.com> wrote:

> Pharo10 on SmalltalkHub is humongous. You can definitely do a stress test with it :)
> 
> Uko
> 
> On 12 Dec 2013, at 15:43, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:
> 
>> I would need a large project, composed of one or more packages, with more than 150~200 classes, which triggers the slow read and writing times Sebastian experience. And, probably, to be complete, a long and complex commit history in git (> 100 commits).
>> 
>> I'll keep in mind the idea of creating one randomly ;)
>> 
>> Thierry
>> 
>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Yuriy Tymchuk [yuriy.tymchuk at me.com]
>> Date d'envoi : jeudi 12 décembre 2013 15:37
>> À : Pharo Development List
>> Objet : Re: [Pharo-dev] Tell me about your workflow
>> 
>> Are you interested in a package or a project? I can provide you information based on size, etc…
>> 
>> Uko
>> 
>> On 12 Dec 2013, at 15:30, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:
>> 
>>> I gave up running gitfiletree on 1.4 :(
>>> 
>>> It's possible to use gitfiletree from a 2.0 or a 3.0 image to browse your git repository, but testing the writing will be an issue.
>>> 
>>> My best chance would be to find a large enough package I can use on 2.0 or 3.0 to test and profile. Does anybody has a large enough package which could fit? Anything that doesn't require a NDA to read it, of course. Is Roassal large enough?
>>> 
>>> Thierry
>>> 
>>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Sebastian Sastre [sebastian at flowingconcept.com]
>>> Date d'envoi : jeudi 12 décembre 2013 12:12
>>> À : Pharo Development List
>>> Objet : Re: [Pharo-dev] Tell me about your workflow
>>> 
>>> gee the big code package is airflowing which I have, quite conservatively, running on #14438 images
>>> 
>>>  I load filetree like this:
>>> 
>>> Gofer new
>>>       url: 'http://ss3.gemstone.com/ss/FileTree';
>>>       package: 'ConfigurationOfFileTree';
>>>       load.
>>> ((Smalltalk at: #ConfigurationOfFileTree) project version: #'stable') load.
>>> 
>>> and it never complained
>>> 
>>> let me know 
>>> 
>>>  
>>> 
>>> 
>>> 
>>> On Dec 12, 2013, at 3:53 AM, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:
>>> 
>>>> If you would be ready to profile a package save on your repository, it would be great. In the mean time, I'll make available a special gitfiletree package to test. Which version of Pharo you are using? 2.0 or 3.0?
>>>> 
>>>> Regards,
>>>> 
>>>> Thierry
>>>> 
>>>> 
>>>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Sebastian Sastre [sebastian at flowingconcept.com]
>>>> Date d'envoi : mercredi 11 décembre 2013 17:09
>>>> À : Pharo Development List
>>>> Objet : Re: [Pharo-dev] Tell me about your workflow
>>>> 
>>>> ok, if saving is dumping all, then 3 is confirmed? After the first commit, I'd say so.
>>>> 
>>>> about 2, I don't know. I'm available to make tests and measure results
>>>> 
>>>> have a nice trip, keep us tuned about any progress
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Dec 11, 2013, at 2:09 PM, Goubier Thierry <thierry.goubier at cea.fr> wrote:
>>>> 
>>>>> Yes, you're right in the general case.
>>>>> 
>>>>> But a solution to that general problem will take time to be implemented (time I lack at the moment, sadly) and if the main gain is a few % because it's writing the version file and the metadata for methods which are the "slow" factors, then we'll have worked hard for nothing.
>>>>> 
>>>>> If you want to help, I'd really like to see either 2- or 3- confirmed. I can produce a special gitfiletree to remove writing the metadata, that you can try on a large project temporary copy; if the slow writing (and reading) is confirmed, then this is 3-
>>>>> 
>>>>> (But I'm leaving on a trip tomorrow early, so I have no idea of when I'll have the time to do that :( ).
>>>>> 
>>>>> Thierry
>>>>> 
>>>>> Le 11/12/2013 16:44, Sebastian Sastre a écrit :
>>>>>> Without entering in details, a cause for slow package write is dumping
>>>>>> all every time.
>>>>>> 
>>>>>> For that strategy, we already have the image save which is magically fast.
>>>>>> 
>>>>>> So, if we make something to scan the code and write only when it's
>>>>>> different from what's on disk, then we would be preventing tons of
>>>>>> redundant writes
>>>>>> 
>>>>>> sebastian <https://about.me/sebastianconcept>
>>>>>> 
>>>>>> o/
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <thierry.goubier at cea.fr
>>>>>> <mailto:thierry.goubier at cea.fr>> wrote:
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit :
>>>>>>>> ah, and IMHO the problem is not about reading... is about writing (if it
>>>>>>>> has to write the metadata each time...).
>>>>>>> 
>>>>>>> But, personnaly, I don't know if this is the reason for the lack of
>>>>>>> performance...
>>>>>>> 
>>>>>>> I have three hypothesis for Sebastian problem:
>>>>>>> 1 - Slow read time for version metadata
>>>>>>> - Confirmed because of the 16 seconds wait time for reading the
>>>>>>> package metadata in the repository browser.
>>>>>>> 2 - Slow metadata write
>>>>>>> 3 - Slow package write
>>>>>>> 
>>>>>>> I have an implemented solution for 1-, a very easy to implement for
>>>>>>> 2-, and none yet for 3-
>>>>>>> 
>>>>>>> So I'd really like to check if 3- is confirmed ;)
>>>>>>> 
>>>>>>> Thierry
>>>>>>> 
>>>>>>>> 
>>>>>>>> Esteban
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano
>>>>>>>> <estebanlm at gmail.com <mailto:estebanlm at gmail.com>
>>>>>>>> <mailto:estebanlm at gmail.com>> wrote:
>>>>>>>> 
>>>>>>>>   Thierry, I know there is a working version... let me search...
>>>>>>>> 
>>>>>>>>   (5 mins later)
>>>>>>>> 
>>>>>>>> 
>>>>>>>>   here:
>>>>>>>> 
>>>>>>>> https://github.com/rjsargent/CypressReferenceImplementation
>>>>>>>> 
>>>>>>>>   Dale says Richard made a metadata-less version.
>>>>>>>> 
>>>>>>>>   We should take a look at that.
>>>>>>>> 
>>>>>>>>   Esteban
>>>>>>>> 
>>>>>>>> 
>>>>>>>>   On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry
>>>>>>>>   <thierry.goubier at cea.fr
>>>>>>>> <mailto:thierry.goubier at cea.fr><mailto:thierry.goubier at cea.fr>> wrote:
>>>>>>>> 
>>>>>>>>       Esteban, Sebastian,
>>>>>>>> 
>>>>>>>>       In the filetree code, you will find a format without metadata,
>>>>>>>>       but it's not in use anymore.
>>>>>>>> 
>>>>>>>>       If you use gitfiletree, it will write the metadata for
>>>>>>>>       compatibility reasons with filetree, but it will never read it
>>>>>>>> back.
>>>>>>>> 
>>>>>>>>       I'm pushing code to make filetree robust to absence of metadata,
>>>>>>>>       but I haven't worked on it for a while.
>>>>>>>> 
>>>>>>>>       gitfiletree has solved the problem of a slow metadata read. It
>>>>>>>>       does not solve any performance problem associated with
>>>>>>>> writing, yet.
>>>>>>>> 
>>>>>>>>       Thierry
>>>>>>>> 
>>>>>>>>       Le 11/12/2013 16:12, Esteban Lorenzano a écrit :
>>>>>>>> 
>>>>>>>>           I know there is a version of filetree without metadata (more
>>>>>>>>           compelling
>>>>>>>>           for projects that will never use other formats).
>>>>>>>>           Dale told me that there was a preview somewhere, but I
>>>>>>>>           didn't tested yet
>>>>>>>>           (lack of time) and now I cannot find the mail...
>>>>>>>>           Dale, can you re-send the link?
>>>>>>>> 
>>>>>>>>           cheers,
>>>>>>>>           Esteban
>>>>>>>> 
>>>>>>>> 
>>>>>>>>           On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre
>>>>>>>>           <sebastian at flowingconcept.com
>>>>>>>> <mailto:sebastian at flowingconcept.com>
>>>>>>>>           <mailto:sebastian at flowingconcept.com>
>>>>>>>>           <mailto:sebastian at __flowingconcept.com
>>>>>>>>           <mailto:sebastian at flowingconcept.com>>> wrote:
>>>>>>>> 
>>>>>>>>                I should breath before I type, but you probably already
>>>>>>>>           got that I
>>>>>>>>                meant /redundant writes/ (not reads)...
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                Anyway.. I was talking with Esteban and he mentions
>>>>>>>>           some kind of
>>>>>>>>                compatibility metadata.
>>>>>>>> 
>>>>>>>>                If I'm going to give a leap of faith to filetree repos
>>>>>>>>           to save code
>>>>>>>>                why should I care about mcz compatibility? Paying a
>>>>>>>>           toll for no
>>>>>>>>                reason is evil.
>>>>>>>> 
>>>>>>>>                Maybe we could make that optional so those who don't
>>>>>>>>           extract value
>>>>>>>>                from that feature can opt-out?
>>>>>>>> 
>>>>>>>>                sebastian <https://about.me/__sebastianconcept
>>>>>>>>           <https://about.me/sebastianconcept>>
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                o/
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                On Dec 11, 2013, at 12:44 PM, Sebastian Sastre
>>>>>>>>                <sebastian at flowingconcept.com
>>>>>>>> <mailto:sebastian at flowingconcept.com>
>>>>>>>>           <mailto:sebastian at flowingconcept.com>
>>>>>>>>           <mailto:sebastian at __flowingconcept.com
>>>>>>>>           <mailto:sebastian at flowingconcept.com>>>
>>>>>>>>                wrote:
>>>>>>>> 
>>>>>>>>                    Hi Thierry
>>>>>>>> 
>>>>>>>>                    On Dec 11, 2013, at 12:43 PM, Goubier Thierry
>>>>>>>>                    <thierry.goubier at cea.fr
>>>>>>>> <mailto:thierry.goubier at cea.fr>
>>>>>>>>               <mailto:thierry.goubier at cea.fr>
>>>>>>>>               <mailto:thierry.goubier at cea.fr
>>>>>>>>               <mailto:thierry.goubier at cea.fr>__>> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                            I have packages (in the order of hundreds
>>>>>>>>                       of classes) and save
>>>>>>>>                            delays
>>>>>>>>                            and package click delays are starting to
>>>>>>>>                       demand patience in a
>>>>>>>>                            way that
>>>>>>>>                            doesn't feel like the right path
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                        Which operations ? I didn't remember noticing
>>>>>>>>                   much with 179
>>>>>>>>                        classes on a laptop without a SSD.
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                    choose one. Just for clicking the package that will
>>>>>>>>               should you
>>>>>>>>                    UUID, version and author I need to wait ~16
>>>>>>>>               seconds. Sounds like a
>>>>>>>>                    lot of overhead for reading a small .json file.
>>>>>>>> 
>>>>>>>>                    But the write is the most worrisome
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                            All that is with a SSD disk, otherwise save
>>>>>>>>                       delays would be
>>>>>>>>                            /way/ beyond
>>>>>>>>                            unacceptable
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                        I'd like to know more, and understand the
>>>>>>>>                   reason, for sure. As
>>>>>>>>                        far as I know, filetree will rewrite the whole
>>>>>>>>                   package to disk
>>>>>>>>                        everytime... and maybe optimising that could be
>>>>>>>>                   the solution.
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                    Well, that explains a lot. Writing all every time
>>>>>>>>               is the lazy
>>>>>>>>                    thing that's okay for a prototype and temporary
>>>>>>>>               code in a proof of
>>>>>>>>                    concept but that massive redundant reads certainly
>>>>>>>>               doesn't sounds
>>>>>>>>                    like pro software. Specially for SSD's which has a
>>>>>>>>               limited
>>>>>>>>                    quantity of writes
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                        Thierry
>>>>>>>> 
>>>>>>>>                            sebastian
>>>>>>>>                       <https://about.me/__sebastianconcept
>>>>>>>>                       <https://about.me/sebastianconcept>>
>>>>>>>> 
>>>>>>>>                            o/
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>                        --
>>>>>>>>                        Thierry Goubier
>>>>>>>>                        CEA list
>>>>>>>>                        Laboratoire des Fondations des Systèmes Temps
>>>>>>>>                   Réel Embarqués
>>>>>>>>                        91191 Gif sur Yvette Cedex
>>>>>>>>                        France
>>>>>>>>                        Phone/Fax: +33 (0) 1 69 08 32 92
>>>>>>>>                   <tel:%2B33%20%280%29%201%2069%2008%2032%2092>
>>>>>>>>                        <tel:%2B33%20%280%29%201%2069%__2008%2032%2092>
>>>>>>>>                   / 83 95
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>       --
>>>>>>>>       Thierry Goubier
>>>>>>>>       CEA list
>>>>>>>>       Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>>>>>       91191 Gif sur Yvette Cedex
>>>>>>>>       France
>>>>>>>>       Phone/Fax: +33 (0) 1 69 08 32 92
>>>>>>>>       <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Thierry Goubier
>>>>>>> CEA list
>>>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>>>> 91191 Gif sur Yvette Cedex
>>>>>>> France
>>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
>>>>>> 
>>>>> 
>>>>> -- 
>>>>> Thierry Goubier
>>>>> CEA list
>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>> 91191 Gif sur Yvette Cedex
>>>>> France
>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20131213/0ad0e76e/attachment-0002.html>


More information about the Pharo-dev mailing list