[Pharo-dev] Tell me about your workflow

Sebastian Sastre sebastian at flowingconcept.com
Fri Dec 13 08:57:36 EST 2013


I can try a forced load on a 3.0 ignoring requisites for testing purposes

but I'm using this:
https://github.com/dalehenrich/filetree

I'm not sure what you're are using

can you clarify?



On Dec 13, 2013, at 11:43 AM, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:

> Ok. I'll try Roassal on a slow netbook to see. I don't see a factor of 10 difference between yours and Roassal, so I'll have a look.
> 
> Thierry
> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Sebastian Sastre [sebastian at flowingconcept.com]
> Date d'envoi : vendredi 13 décembre 2013 14:32
> À : Pharo Development List
> Objet : Re: [Pharo-dev] Tell me about your workflow
> 
> A bit.
> 
> This is from today's current version (and is not all, it's only the two biggest packages):
> 
> (MCPackage named: 'flow') workingCopy packageInfo classes size. 363.
> (MCPackage named: 'flow') workingCopy packageInfo coreMethods size. 4585.
> 
> (MCPackage named: 'airflowing') workingCopy packageInfo classes size. 377.
> (MCPackage named: 'airflowing') workingCopy packageInfo coreMethods size. 5818.
> 
> 
> 
> 
> 
> 
> 
> On Dec 13, 2013, at 11:25 AM, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:
> 
>> Roassal: 3493
>> 
>> Number of versions in the package history: 733. Size of the version file: 202796.
>> 
>> Is that a lot lower than your count?
>> 
>> Thierry
>> 
>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Sebastian Sastre [sebastian at flowingconcept.com]
>> Date d'envoi : vendredi 13 décembre 2013 13:34
>> À : Pharo Development List
>> Objet : Re: [Pharo-dev] Tell me about your workflow
>> 
>> how many coreMethods?
>> 
>> 
>> 
>> 
>> On Dec 13, 2013, at 7:00 AM, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:
>> 
>>> Bad news. Roassal package directory has 355 entries (343 classes + a few extensions) and I don't see much of a slow down (on 3.0). It's not instantaneous, but with a bit of feedback, it doesn't seems long.
>>> 
>>> I'll do some profiling.
>>> 
>>> Thierry
>>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de GOUBIER Thierry
>>> Date d'envoi : jeudi 12 décembre 2013 17:07
>>> À : Pharo Development List
>>> Objet : [PROVENANCE INTERNET] Re: [Pharo-dev] Tell me about your workflow
>>> 
>>> Thanks for the pointers.
>>> 
>>> I'll look at Seaside/Moose/Mondrian and Roassal, because I need code I can load and save in an image without destroying the very image I use to test  (which would happen if I load Pharo10 stuff in a 3.0 image ;) ).
>>> 
>>> Thierry
>>> 
>>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Yuriy Tymchuk [yuriy.tymchuk at me.com]
>>> Date d'envoi : jeudi 12 décembre 2013 16:24
>>> À : Pharo Development List
>>> Objet : Re: [Pharo-dev] Tell me about your workflow
>>> 
>>> So if you want something big and with a lot of commits you can use Pharo* in general. Pharo10 has the most versions and Pharo30Inbox is the largest one. If you want some other projects then you heve to take a look at Seaside30, Mondrian, Moose, Glamour or Roassal.
>>> 
>>> Uko
>>> 
>>> On 12 Dec 2013, at 16:20, Yuriy Tymchuk <yuriy.tymchuk at me.com> wrote:
>>> 
>>>> Pharo10 on SmalltalkHub is humongous. You can definitely do a stress test with it :)
>>>> 
>>>> Uko
>>>> 
>>>> On 12 Dec 2013, at 15:43, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:
>>>> 
>>>>> I would need a large project, composed of one or more packages, with more than 150~200 classes, which triggers the slow read and writing times Sebastian experience. And, probably, to be complete, a long and complex commit history in git (> 100 commits).
>>>>> 
>>>>> I'll keep in mind the idea of creating one randomly ;)
>>>>> 
>>>>> Thierry
>>>>> 
>>>>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Yuriy Tymchuk [yuriy.tymchuk at me.com]
>>>>> Date d'envoi : jeudi 12 décembre 2013 15:37
>>>>> À : Pharo Development List
>>>>> Objet : Re: [Pharo-dev] Tell me about your workflow
>>>>> 
>>>>> Are you interested in a package or a project? I can provide you information based on size, etc…
>>>>> 
>>>>> Uko
>>>>> 
>>>>> On 12 Dec 2013, at 15:30, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:
>>>>> 
>>>>>> I gave up running gitfiletree on 1.4 :(
>>>>>> 
>>>>>> It's possible to use gitfiletree from a 2.0 or a 3.0 image to browse your git repository, but testing the writing will be an issue.
>>>>>> 
>>>>>> My best chance would be to find a large enough package I can use on 2.0 or 3.0 to test and profile. Does anybody has a large enough package which could fit? Anything that doesn't require a NDA to read it, of course. Is Roassal large enough?
>>>>>> 
>>>>>> Thierry
>>>>>> 
>>>>>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Sebastian Sastre [sebastian at flowingconcept.com]
>>>>>> Date d'envoi : jeudi 12 décembre 2013 12:12
>>>>>> À : Pharo Development List
>>>>>> Objet : Re: [Pharo-dev] Tell me about your workflow
>>>>>> 
>>>>>> gee the big code package is airflowing which I have, quite conservatively, running on #14438 images
>>>>>> 
>>>>>>  I load filetree like this:
>>>>>> 
>>>>>> Gofer new
>>>>>>       url: 'http://ss3.gemstone.com/ss/FileTree';
>>>>>>       package: 'ConfigurationOfFileTree';
>>>>>>       load.
>>>>>> ((Smalltalk at: #ConfigurationOfFileTree) project version: #'stable') load.
>>>>>> 
>>>>>> and it never complained
>>>>>> 
>>>>>> let me know 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Dec 12, 2013, at 3:53 AM, GOUBIER Thierry <thierry.goubier at cea.fr> wrote:
>>>>>> 
>>>>>>> If you would be ready to profile a package save on your repository, it would be great. In the mean time, I'll make available a special gitfiletree package to test. Which version of Pharo you are using? 2.0 or 3.0?
>>>>>>> 
>>>>>>> Regards,
>>>>>>> 
>>>>>>> Thierry
>>>>>>> 
>>>>>>> 
>>>>>>> De : Pharo-dev [pharo-dev-bounces at lists.pharo.org] de la part de Sebastian Sastre [sebastian at flowingconcept.com]
>>>>>>> Date d'envoi : mercredi 11 décembre 2013 17:09
>>>>>>> À : Pharo Development List
>>>>>>> Objet : Re: [Pharo-dev] Tell me about your workflow
>>>>>>> 
>>>>>>> ok, if saving is dumping all, then 3 is confirmed? After the first commit, I'd say so.
>>>>>>> 
>>>>>>> about 2, I don't know. I'm available to make tests and measure results
>>>>>>> 
>>>>>>> have a nice trip, keep us tuned about any progress
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Dec 11, 2013, at 2:09 PM, Goubier Thierry <thierry.goubier at cea.fr> wrote:
>>>>>>> 
>>>>>>>> Yes, you're right in the general case.
>>>>>>>> 
>>>>>>>> But a solution to that general problem will take time to be implemented (time I lack at the moment, sadly) and if the main gain is a few % because it's writing the version file and the metadata for methods which are the "slow" factors, then we'll have worked hard for nothing.
>>>>>>>> 
>>>>>>>> If you want to help, I'd really like to see either 2- or 3- confirmed. I can produce a special gitfiletree to remove writing the metadata, that you can try on a large project temporary copy; if the slow writing (and reading) is confirmed, then this is 3-
>>>>>>>> 
>>>>>>>> (But I'm leaving on a trip tomorrow early, so I have no idea of when I'll have the time to do that :( ).
>>>>>>>> 
>>>>>>>> Thierry
>>>>>>>> 
>>>>>>>> Le 11/12/2013 16:44, Sebastian Sastre a écrit :
>>>>>>>>> Without entering in details, a cause for slow package write is dumping
>>>>>>>>> all every time.
>>>>>>>>> 
>>>>>>>>> For that strategy, we already have the image save which is magically fast.
>>>>>>>>> 
>>>>>>>>> So, if we make something to scan the code and write only when it's
>>>>>>>>> different from what's on disk, then we would be preventing tons of
>>>>>>>>> redundant writes
>>>>>>>>> 
>>>>>>>>> sebastian <https://about.me/sebastianconcept>
>>>>>>>>> 
>>>>>>>>> o/
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <thierry.goubier at cea.fr
>>>>>>>>> <mailto:thierry.goubier at cea.fr>> wrote:
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit :
>>>>>>>>>>> ah, and IMHO the problem is not about reading... is about writing (if it
>>>>>>>>>>> has to write the metadata each time...).
>>>>>>>>>> 
>>>>>>>>>> But, personnaly, I don't know if this is the reason for the lack of
>>>>>>>>>> performance...
>>>>>>>>>> 
>>>>>>>>>> I have three hypothesis for Sebastian problem:
>>>>>>>>>> 1 - Slow read time for version metadata
>>>>>>>>>> - Confirmed because of the 16 seconds wait time for reading the
>>>>>>>>>> package metadata in the repository browser.
>>>>>>>>>> 2 - Slow metadata write
>>>>>>>>>> 3 - Slow package write
>>>>>>>>>> 
>>>>>>>>>> I have an implemented solution for 1-, a very easy to implement for
>>>>>>>>>> 2-, and none yet for 3-
>>>>>>>>>> 
>>>>>>>>>> So I'd really like to check if 3- is confirmed ;)
>>>>>>>>>> 
>>>>>>>>>> Thierry
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Esteban
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano
>>>>>>>>>>> <estebanlm at gmail.com <mailto:estebanlm at gmail.com>
>>>>>>>>>>> <mailto:estebanlm at gmail.com>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>   Thierry, I know there is a working version... let me search...
>>>>>>>>>>> 
>>>>>>>>>>>   (5 mins later)
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>   here:
>>>>>>>>>>> 
>>>>>>>>>>> https://github.com/rjsargent/CypressReferenceImplementation
>>>>>>>>>>> 
>>>>>>>>>>>   Dale says Richard made a metadata-less version.
>>>>>>>>>>> 
>>>>>>>>>>>   We should take a look at that.
>>>>>>>>>>> 
>>>>>>>>>>>   Esteban
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>   On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry
>>>>>>>>>>>   <thierry.goubier at cea.fr
>>>>>>>>>>> <mailto:thierry.goubier at cea.fr><mailto:thierry.goubier at cea.fr>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>       Esteban, Sebastian,
>>>>>>>>>>> 
>>>>>>>>>>>       In the filetree code, you will find a format without metadata,
>>>>>>>>>>>       but it's not in use anymore.
>>>>>>>>>>> 
>>>>>>>>>>>       If you use gitfiletree, it will write the metadata for
>>>>>>>>>>>       compatibility reasons with filetree, but it will never read it
>>>>>>>>>>> back.
>>>>>>>>>>> 
>>>>>>>>>>>       I'm pushing code to make filetree robust to absence of metadata,
>>>>>>>>>>>       but I haven't worked on it for a while.
>>>>>>>>>>> 
>>>>>>>>>>>       gitfiletree has solved the problem of a slow metadata read. It
>>>>>>>>>>>       does not solve any performance problem associated with
>>>>>>>>>>> writing, yet.
>>>>>>>>>>> 
>>>>>>>>>>>       Thierry
>>>>>>>>>>> 
>>>>>>>>>>>       Le 11/12/2013 16:12, Esteban Lorenzano a écrit :
>>>>>>>>>>> 
>>>>>>>>>>>           I know there is a version of filetree without metadata (more
>>>>>>>>>>>           compelling
>>>>>>>>>>>           for projects that will never use other formats).
>>>>>>>>>>>           Dale told me that there was a preview somewhere, but I
>>>>>>>>>>>           didn't tested yet
>>>>>>>>>>>           (lack of time) and now I cannot find the mail...
>>>>>>>>>>>           Dale, can you re-send the link?
>>>>>>>>>>> 
>>>>>>>>>>>           cheers,
>>>>>>>>>>>           Esteban
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>           On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre
>>>>>>>>>>>           <sebastian at flowingconcept.com
>>>>>>>>>>> <mailto:sebastian at flowingconcept.com>
>>>>>>>>>>>           <mailto:sebastian at flowingconcept.com>
>>>>>>>>>>>           <mailto:sebastian at __flowingconcept.com
>>>>>>>>>>>           <mailto:sebastian at flowingconcept.com>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>                I should breath before I type, but you probably already
>>>>>>>>>>>           got that I
>>>>>>>>>>>                meant /redundant writes/ (not reads)...
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                Anyway.. I was talking with Esteban and he mentions
>>>>>>>>>>>           some kind of
>>>>>>>>>>>                compatibility metadata.
>>>>>>>>>>> 
>>>>>>>>>>>                If I'm going to give a leap of faith to filetree repos
>>>>>>>>>>>           to save code
>>>>>>>>>>>                why should I care about mcz compatibility? Paying a
>>>>>>>>>>>           toll for no
>>>>>>>>>>>                reason is evil.
>>>>>>>>>>> 
>>>>>>>>>>>                Maybe we could make that optional so those who don't
>>>>>>>>>>>           extract value
>>>>>>>>>>>                from that feature can opt-out?
>>>>>>>>>>> 
>>>>>>>>>>>                sebastian <https://about.me/__sebastianconcept
>>>>>>>>>>>           <https://about.me/sebastianconcept>>
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                o/
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                On Dec 11, 2013, at 12:44 PM, Sebastian Sastre
>>>>>>>>>>>                <sebastian at flowingconcept.com
>>>>>>>>>>> <mailto:sebastian at flowingconcept.com>
>>>>>>>>>>>           <mailto:sebastian at flowingconcept.com>
>>>>>>>>>>>           <mailto:sebastian at __flowingconcept.com
>>>>>>>>>>>           <mailto:sebastian at flowingconcept.com>>>
>>>>>>>>>>>                wrote:
>>>>>>>>>>> 
>>>>>>>>>>>                    Hi Thierry
>>>>>>>>>>> 
>>>>>>>>>>>                    On Dec 11, 2013, at 12:43 PM, Goubier Thierry
>>>>>>>>>>>                    <thierry.goubier at cea.fr
>>>>>>>>>>> <mailto:thierry.goubier at cea.fr>
>>>>>>>>>>>               <mailto:thierry.goubier at cea.fr>
>>>>>>>>>>>               <mailto:thierry.goubier at cea.fr
>>>>>>>>>>>               <mailto:thierry.goubier at cea.fr>__>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                            I have packages (in the order of hundreds
>>>>>>>>>>>                       of classes) and save
>>>>>>>>>>>                            delays
>>>>>>>>>>>                            and package click delays are starting to
>>>>>>>>>>>                       demand patience in a
>>>>>>>>>>>                            way that
>>>>>>>>>>>                            doesn't feel like the right path
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                        Which operations ? I didn't remember noticing
>>>>>>>>>>>                   much with 179
>>>>>>>>>>>                        classes on a laptop without a SSD.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                    choose one. Just for clicking the package that will
>>>>>>>>>>>               should you
>>>>>>>>>>>                    UUID, version and author I need to wait ~16
>>>>>>>>>>>               seconds. Sounds like a
>>>>>>>>>>>                    lot of overhead for reading a small .json file.
>>>>>>>>>>> 
>>>>>>>>>>>                    But the write is the most worrisome
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                            All that is with a SSD disk, otherwise save
>>>>>>>>>>>                       delays would be
>>>>>>>>>>>                            /way/ beyond
>>>>>>>>>>>                            unacceptable
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                        I'd like to know more, and understand the
>>>>>>>>>>>                   reason, for sure. As
>>>>>>>>>>>                        far as I know, filetree will rewrite the whole
>>>>>>>>>>>                   package to disk
>>>>>>>>>>>                        everytime... and maybe optimising that could be
>>>>>>>>>>>                   the solution.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                    Well, that explains a lot. Writing all every time
>>>>>>>>>>>               is the lazy
>>>>>>>>>>>                    thing that's okay for a prototype and temporary
>>>>>>>>>>>               code in a proof of
>>>>>>>>>>>                    concept but that massive redundant reads certainly
>>>>>>>>>>>               doesn't sounds
>>>>>>>>>>>                    like pro software. Specially for SSD's which has a
>>>>>>>>>>>               limited
>>>>>>>>>>>                    quantity of writes
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                        Thierry
>>>>>>>>>>> 
>>>>>>>>>>>                            sebastian
>>>>>>>>>>>                       <https://about.me/__sebastianconcept
>>>>>>>>>>>                       <https://about.me/sebastianconcept>>
>>>>>>>>>>> 
>>>>>>>>>>>                            o/
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                        --
>>>>>>>>>>>                        Thierry Goubier
>>>>>>>>>>>                        CEA list
>>>>>>>>>>>                        Laboratoire des Fondations des Systèmes Temps
>>>>>>>>>>>                   Réel Embarqués
>>>>>>>>>>>                        91191 Gif sur Yvette Cedex
>>>>>>>>>>>                        France
>>>>>>>>>>>                        Phone/Fax: +33 (0) 1 69 08 32 92
>>>>>>>>>>>                   <tel:%2B33%20%280%29%201%2069%2008%2032%2092>
>>>>>>>>>>>                        <tel:%2B33%20%280%29%201%2069%__2008%2032%2092>
>>>>>>>>>>>                   / 83 95
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>       --
>>>>>>>>>>>       Thierry Goubier
>>>>>>>>>>>       CEA list
>>>>>>>>>>>       Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>>>>>>>>       91191 Gif sur Yvette Cedex
>>>>>>>>>>>       France
>>>>>>>>>>>       Phone/Fax: +33 (0) 1 69 08 32 92
>>>>>>>>>>>       <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Thierry Goubier
>>>>>>>>>> CEA list
>>>>>>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>>>>>>> 91191 Gif sur Yvette Cedex
>>>>>>>>>> France
>>>>>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> -- 
>>>>>>>> Thierry Goubier
>>>>>>>> CEA list
>>>>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>>>>> 91191 Gif sur Yvette Cedex
>>>>>>>> France
>>>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20131213/2a36a46c/attachment-0002.html>


More information about the Pharo-dev mailing list