- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Spur Garbage collection takes 4 times more when loading MSE (WAS: mse loading looks slower :()

by Eliot Miranda

Hi Vincent, I'll take a look early this week. There's clearly a bug; the Spur GC is /not/ collecting those dictionaries :-( (thanks Stephan!). Assuming the bug is fixed times should come down (see below). It may be that the bug in Slang that I introduced in December has broken things because I don't see these symptoms in my daily work (but I use the most up-to-date VM version possible ;-) ). But I'm not in denial and look forward to using MOOSE as a good stress case. I do want to say that the GC is not complete. Right now we have a scavenger that works well, and a global GC that has a slow compaction algorithm, and hence there are significant pauses. For example here's what I see as typical in using SPur for VMMaker work: memory 160,432,128 bytes old 153,658,624 bytes (95.8%) young 4,838,224 bytes (3%) used 127,009,928 bytes (79.2%) free 28,126,456 bytes (17.5%) GCs 7,265 (?? ms between GCs) full 36 totalling 13,229 ms (0% uptime), avg 367.5 ms incr 7,229 totalling 6,546 ms (0% uptime), avg 0.9 ms tenures 3,589,063 (avg 0 GCs/tenure) (There's no uptime in the above stats because we're still transitioning Squeak to the 64-bit clock and there are consequently bugs in computing uptime). The plan is to add an incremental global GC so this work is broken up into much smaller pieces. I don't want to see 700ms pauses in global GC; one can't do game animation with that. So an incremental mark-sweep is needed. There are two nice papers we're considering, one from Lua and one for a truly concurrent collector. But time is pressing, so if anyone out there knows GC and is interested in helping this is a nicely self-contained project for which we'd love to have volunteers. _,,,^..^,,,_ (phone) > On Jan 24, 2016, at 3:43 AM, Vincent BLONDEAU <vincent.blondeau(a)polytech-lille.net> wrote: > > Hi, > > I made the benchmarks with the files you provided. I have more or less the > same magnitude: > Version 504: 0:00:01:09.021 > Version 1175: 0:00:02:37.507 > > However, by launching it in the time profiler (MooseModel new > importFromMSEStream: (StandardFileStream readOnlyFileNamed: > 'd:/ArgoUML-0-34.mse')), it takes > 504: 1 min 55 > 1175: 4 min 25 > Well there is a delta... > > After investigation, the standard process has almost the same duration (120 > secs for prespur and 140 secs for spur). > But, there is a large difference in GC time: > > 504: not spur > **Memory** > old +144,822,000 bytes > young -8,293,660 bytes > used +136,528,340 bytes > free -104,186,788 bytes > > **GCs** > full 1 totalling 965ms (1.0% uptime), avg 965.0ms > incr 3264 totalling 42,279ms (33.0% uptime), avg 13.0ms > tenures 2,497 (avg 1 GCs/tenure) > root table 0 overflows > > 1175: spur > **Memory** > old +0 bytes > young +340,048 bytes > used +340,048 bytes > free -340,048 bytes > **GCs** > full 7 totalling 145,003ms (66.0% uptime), avg > 20715.0ms > incr 3288 totalling 30,912ms (14.0% uptime), avg 9.0ms > tenures 7,146,505 (avg 0 GCs/tenure) > root table 0 overflows > > Total GC time > 504: 43 secs > 1175: 176 secs > > See the performance reports attached. > > I let VM people take care of the issue ;) > > Cheers, > Vincent > > -----Original Message----- > From: moose-dev-bounces(a)list.inf.unibe.ch > [mailto:moose-dev-bounces@list.inf.unibe.ch] On Behalf Of Tudor Girba > Sent: dimanche 24 janvier 2016 09:08 > To: Moose-related development > Subject: [Moose-dev] Re: mse loading looks slower :( > > Hi, > > I am talking about the difference between Moose 6 images: > - October 7: > https://ci.inria.fr/moose/job/moose-6.0/504/artifact/moose-6.0.zip > > - yesterday: > https://ci.inria.fr/moose/job/moose-6.0/1175/artifact/moose-6.0.zip > > Multiple things did change, but not in Moose. In the end, I would like to > understand where the slowness comes. Maybe it comes from Spur itself, but > maybe it comes from somewhere else. > > Cheers, > Doru > > > >>> On Jan 24, 2016, at 1:41 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> >> wrote: >> >> Doru...just to be sure it is not a Pharo (image change), when you said > before and after Spur, do you mean a Pharo 5.0 exactly (just before Spur) > and a Pharo JUST after it? Otherwise, the slowness may come from the > difference between the 2 Pharos you are running. >> >> Cheers, >> >> On Sat, Jan 23, 2016 at 5:55 PM, Tudor Girba <tudor(a)tudorgirba.com> wrote: >> Hi, >> >> I am doing some performance testing of Moose with the Spur VM on Mac. >> >> I tried to load an MSE file with ArgoUML 0.34 and on my machine it loads > twice as slow with Spur than before: >> - PreSpur: 0:00:01:07.272 >> - Spur: 0:00:02:10.508 >> >> Here is the reference file: >> https://dl.dropboxusercontent.com/u/18323746/Tmp/ArgoUML-0-34.mse.zip >> >> And here is the script: >> [ >> MooseModel new >> importFromMSEStream: (StandardFileStream > readOnlyFileNamed: >> (FileSystem workingDirectory / 'src' / > 'ArgoUML-0-34' / 'ArgoUML-0-34.mse') fullName). >> ] timeToRun >> >> Do you get the same? >> >> Cheers, >> Doru >> >> >> -- >> www.tudorgirba.com >> www.feenk.com >> >> "Problem solving should be focused on describing the problem in a way >> that makes the solution obvious." >> >> >> >> >> >> _______________________________________________ >> Moose-dev mailing list >> Moose-dev(a)list.inf.unibe.ch >> https://www.list.inf.unibe.ch/listinfo/moose-dev >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> _______________________________________________ >> Moose-dev mailing list >> Moose-dev(a)list.inf.unibe.ch >> https://www.list.inf.unibe.ch/listinfo/moose-dev > > -- > www.tudorgirba.com > www.feenk.com > > "What is more important: To be happy, or to make happy?" > > _______________________________________________ > Moose-dev mailing list > Moose-dev(a)list.inf.unibe.ch > https://www.list.inf.unibe.ch/listinfo/moose-dev > <report504.txt> > <report1175.txt>

Jan. 24, 2016

Re: [Pharo-dev] Spur Garbage collection takes 4 times more when loading MSE (WAS: mse loading looks slower :()

by Stephan Eggermont

I think the use of SmallDictionaries for storing Famix state is not such a good idea. After loading the model: bag := Bag new . SmallDictionary allInstances do: [ :each | bag add: each size ]. bag sortedCounts "{530387->0. 19253->3. 18292->1. 20->2. 2->4. 2->11. 1->12}" bag := Bag new . SmallDictionary allInstances do: [ :each | bag add: (each keys) ]. bag sortedCounts an Array(530387->#() 19253->#(#cyclomaticComplexity #numberOfStatements #mooseName) 9311->#(#binding) 8707->#(#mooseName) 426->#(#comments) 20->#(#comments #ir) 13->#(#ir) 4->#(#comments #binding) 3->#(#binding #semanticWarning) 2->#('charset') 1->#(1 2 3 4 5 6 7 8 9 10 11 12) 1->#(14 15 16 17 18 19 20 21 22 23 24) 1->#(9 10 11 12 13 14 15 16) 1->an Array('-source-' #'`@method:' RBPatternVariableNode(`@args) '-pragmas-') 1->#(1 2 3 4 5 6 7 8 9 10 11) 1->{MooseModel} 1->{FAMIXAnnotationType. FAMIXClass. FAMIXNamespace. FAMIXMethod} 1->#(1 2))

Jan. 24, 2016

Moving large classes in Pharo5 between packages makes me sad

by Aliaksei Syrel

[image: Inline image 1] Cheers, Alex

Jan. 24, 2016

3rd Chapter of Updated Pharo by Example ready for review.

by stepharo

3rd Chapter of Updated Pharo by Example ready for review. https://ci.inria.fr/pharo-contribution/view/Books/job/UpdatedPharoByExample… Stef Not thinking too much, just doing.

Jan. 24, 2016

Re: [Pharo-dev] Fix To Include

by stepharo

>> Fix to include are marked when we do not want that the monkey recheck them to avoid that >> they get unttaged for integration. > No, for me they are the ones to get integrated next. I always start with them. i never integrate anything > that is not âfix to includeâ. > > I configured my integrator tool to not show my anything else. This way I am sure that I never integrate > something wrong by mistake. I do not have the same process.

Jan. 24, 2016

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/50545 Home: https://github.com/pharo-project/pharo-core

Jan. 24, 2016

[pharo-project/pharo-core] e29965: 50545

by GitHub

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: e29965eecb03b444a689c55aeb098e9f0fbe09ad https://github.com/pharo-project/pharo-core/commit/e29965eecb03b444a689c55a… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2016-01-24 (Sun, 24 Jan 2016) Changed paths: A Deprecated50.package/extension/Delay/instance/waitOtherwise_.st R Kernel.package/Delay.class/instance/delaying/waitOtherwise_.st M Kernel.package/Delay.class/instance/private/schedule.st M Kernel.package/DelayExperimentalCourageousScheduler.class/instance/low priority processes/schedule_.st M Kernel.package/DelayExperimentalSemaphoreScheduler.class/instance/low priority processes/schedule_.st M Kernel.package/DelayExperimentalSpinScheduler.class/instance/low priority processes/schedule_.st M Kernel.package/DelayMicrosecondScheduler.class/instance/low priority processes/schedule_.st M Kernel.package/DelayMillisecondScheduler.class/instance/low priority processes/schedule_.st M Kernel.package/DelayNullScheduler.class/instance/low priority processes/schedule_.st A Reflectivity.package/RFReification.class/instance/generate/genForRBArgumentNode.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50544.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50545.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50544.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50545.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 50545 17296 Remove Delay>>otherwise: https://pharo.fogbugz.com/f/cases/17296 17453 add #genForRBArgumentNode https://pharo.fogbugz.com/f/cases/17453 http://files.pharo.org/image/50/50545.zip

Jan. 24, 2016

Re: [Pharo-dev] Fix To Include

by Marcus Denker

> On 24 Jan 2016, at 17:07, stepharo <stepharo(a)free.fr> wrote: > > Hi ben >> Is something stuck with integration by the CI monkey. > > Sometimes yes. Sometimes tests randomly failed on windows or something like that :) >> I thought this automatically picked and integrated any Fix To Include status slices. >> But these two cases were marked Fix To include a fews days ago, and >> have not yet been integrated. > No we (either esteban marcus and me look at the code) at least I try to browse the code, > and click on a button. >> Case 17296 --> Fix To Include 2016-01-21 22:29 >> Case 17435 --> Fix To Include 2016-01-22 16:34 > Fix to include are marked when we do not want that the monkey recheck them to avoid that > they get unttaged for integration. No, for me they are the ones to get integrated next. I always start with them. i never integrate anything that is not âfix to includeâ. I configured my integrator tool to not show my anything else. This way I am sure that I never integrate something wrong by mistake. Marcus

Jan. 24, 2016

Re: [Pharo-dev] Fix To Include

by stepharo

Hi ben > Is something stuck with integration by the CI monkey. Sometimes yes. Sometimes tests randomly failed on windows or something like that :) > I thought this automatically picked and integrated any Fix To Include status slices. > But these two cases were marked Fix To include a fews days ago, and > have not yet been integrated. No we (either esteban marcus and me look at the code) at least I try to browse the code, and click on a button. > Case 17296 --> Fix To Include 2016-01-21 22:29 > Case 17435 --> Fix To Include 2016-01-22 16:34 Fix to include are marked when we do not want that the monkey recheck them to avoid that they get unttaged for integration. When I do an integration session like today for example. I tend to go slowly and proceed with the domain I can understand. So now I will do Case 17296 > > But I see several cases integrated since then didn't pass through Fix > To Include, so I'm curious what the process is. Is it still a manual > pick of items to integrate? Fix to include is just a marker to avoid endless verification of an issue. Now this is not because a fix is reviewed by the monkey that he should be automatically integrated. In addition we pay attention to some fast paths when people need it. Do I answer your questions? Stef > In that case why are the above passed > over? There is no indication on the tracker. > > Case 17429 --> Fix To include 2016-01-22 16:52 > --> Integrated 2016-01-22 21:05 Build 50539 > > Case 17446 --> Fix To Include 2016-01-22 21:40 > --> Integrated 2016-01-22 23:40 Build 50540 > > > Case 17404 --> Fix Reviewed By Monkey 2016-01-24 03:42 > Case 17443 --> Fix Reviewed By Monkey 2016-01-24 04:57 > Case 17457 --> Fix Reviewed By Monkey 2016-01-24 05:03 > --> Integrated 2016-01-24 17:55 Build 50541 > > Case 17450 --> Fix Review Needed 2016-01-24 19:37 > --> Integrated 2016-01-24 17:37 Build 50542 > > Case 17301 --> Fix Reviewed by the Monkey 2016-01-24 20:39 > Case 17384 --> Fix To Include 2016-01-24 16:50 > --> Integrated 2016-01-24 20:39 > >

Jan. 24, 2016

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/50544 Home: https://github.com/pharo-project/pharo-core

Jan. 24, 2016