- Pharo-dev - lists.pharo.org

ZeroConf and Launcher

by Torsten Bergmann

ZeroConf (get.pharo.org) is nice - as it allows to easily setup a new machine with a fresh image. What about having the ability to download the latest Launcher using a zeroconf script too?

May 27, 2018

[Pharo 7.0-dev] Build #961: 21995-HeapTest-examples-should-use-String-streamContents-instead-of-Transcript

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #961 was: FAILURE. The Pull Request #1438 was integrated: "21995-HeapTest-examples-should-use-String-streamContents-instead-of-Transcript" Pull request url: https://github.com/pharo-project/pharo/pull/1438 Issue Url: https://pharo.fogbugz.com/f/cases/21995 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

May 27, 2018

[Pharo 7.0-dev] Build #960: 21994-Generator-class-example-method-should-use-streamContents-instead-of-Transcript

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #960 was: SUCCESS. The Pull Request #1437 was integrated: "21994-Generator-class-example-method-should-use-streamContents-instead-of-Transcript" Pull request url: https://github.com/pharo-project/pharo/pull/1437 Issue Url: https://pharo.fogbugz.com/f/cases/21994 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

May 27, 2018

[Pharo 7.0-dev] Build #959: 21993-ExternalBrowser-reference-to-Transcript-does-not-make-sense

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #959 was: FAILURE. The Pull Request #1436 was integrated: "21993-ExternalBrowser-reference-to-Transcript-does-not-make-sense" Pull request url: https://github.com/pharo-project/pharo/pull/1436 Issue Url: https://pharo.fogbugz.com/f/cases/21993 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

May 27, 2018

Re: [Pharo-dev] Curious context change latency experiment results (wrt DelayScheduler)

by Ben Coman

On 6 May 2018 at 04:20, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Ben, > > > On May 5, 2018, at 7:25 AM, Ben Coman <btc(a)openinworld.com> wrote: > > > > On 5 May 2018 at 22:10, Ben Coman <btc(a)openinworld.com> wrote: > >> One of the key parts of Delay scheduling is setting the resumption time. >> There are two places this could be done. >> a. In Delay>>#schedule, running (for example) at userSchedulingPriority=40 >> b. In DelayMicrosecondScheduler>>#handleTImerEvent:, running at >> timingPriority=80 >> >> When we were using the millisecond clock for delay scheduling, >> it made sense to use (b.) since the clock might(?) roll over >> between when resumption time was set, and when #handleTImerEvent: >> expires delays. >> >> This should not be a problem now that we are using the microsecond clock, >> so I wondered what the latency penalty might be between (a.) and (b.). >> I ran a little experiment that indicates the cost/latency of switching >> threads, >> and was curious if anyone can comment on the validity of the experiment >> and interpretation of results. >> >> I made the following three changes... >> >> DelayScheduler subclass: #DelayMicrosecondScheduler >> instanceVariableNames: 'latencyStart countSinceLastZero' >> classVariableNames: 'LatencyCounts' >> poolDictionaries: '' >> category: 'Kernel-Processes' >> >> >> DelayMicrosecondScheduler>>#schedule: aDelay >> latencyStart:= Time primUTCMicrosecondsClock. "This is position (a.)" >> aDelay schedulerBeingWaitedOn ifTrue: [^self error: 'This Delay has >> already been scheduled.']. >> accessProtect critical: [ >> scheduledDelay := aDelay. >> timingSemaphore signal. "#handleTimerEvent: sets scheduledDelay:=nil" >> ]. >> >> >> DelayMicrosecondScheduler>>#handleTimerEvent: microsecondNowTick >> | microsecondNextTick | >> "Process any schedule requests" "This is position (b.)" >> scheduledDelay ifNotNil: [ >> |latency| >> latency := Time primUTCMicrosecondsClock - latencyStart. >> LatencyCounts ifNil: [ LatencyCounts := Bag new ]. >> LatencyCounts add: latency. >> latency = 0 >> ifTrue: [ countSinceLastZero := 1 + (countSinceLastZero ifNil: [0])] >> ifFalse: [Transcript >> crShow: 'zero latency count ' , countSinceLastZero printString ; >> show: ' before latency ', latency printString. >> countSinceLastZero := 0]. >> "Schedule the given delay." >> scheduledDelay scheduler: self resumptionTime: microsecondNowTick + (1000 >> * scheduledDelay millisecondDelayDuration). >> self scheduleDelay: scheduledDelay. >> scheduledDelay := nil ]. >> <snip> >> rest of method unchanged >> >> >> Then opened the Transcript and in Playground evaluated... >> Delay delaySchedulerClass: DelayMicrosecondScheduler. >> >> >> The Transcript results are shown below with some comments inserted. >> >> Now I guess the latency is affected by garbage collection. >> But one thing I was curious about is why the latency's were quantised in >> 1000s. >> >> Another interesting thing is that vast majority of the latency's were >> zero, >> which was a nice surprise, but can it be true? Or is it a consequence >> of the quantitisation rounding down? >> >> It seems that the idle-ness of the image affected how often a non-zero >> latency occurred. >> After I left the house for a while, the count of zero latency was very >> high, but a few still occurred. It would make sense there was less GC >> while idle. What is a good snippet of code to stress GC. I presume the >> latency might increase. >> >> >> zero latency count 2273 before latency 1000 >> zero latency count 943 before latency 1000 >> zero latency count 3666 before latency 1000 >> zero latency count 1643 before latency 1000 >> zero latency count 27 before latency 1000 >> "Left house for 20 minutes" >> zero latency count 12022 before latency 1000 >> zero latency count 15195 before latency 1000 >> zero latency count 41998 before latency 1000 >> "Returned from outing" >> zero latency count 128 before latency 1000 >> zero latency count 116 before latency 1000 >> zero latency count 555 before latency 1000 >> zero latency count 2377 before latency 1000 >> zero latency count 5423 before latency 1000 >> zero latency count 3178 before latency 1000 >> zero latency count 47 before latency 1000 >> zero latency count 2276 before latency 1000 >> "Left house to go shopping" >> zero latency count 6708 before latency 3000 >> zero latency count 4896 before latency 1000 >> zero latency count 433 before latency 1000 >> zero latency count 7106 before latency 1000 >> zero latency count 2195 before latency 1000 >> zero latency count 12397 before latency 1000 >> zero latency count 4815 before latency 2000 >> zero latency count 3480 before latency 1000 >> zero latency count 5690 before latency 1000 >> zero latency count 1670 before latency 1000 >> zero latency count 320 before latency 1000 >> zero latency count 1641 before latency 2000 >> zero latency count 290 before latency 1000 >> zero latency count 3835 before latency 1000 >> zero latency count 3818 before latency 1000 >> zero latency count 1132 before latency 1000 >> "Returned from shopping." >> zero latency count 2045 before latency 1000 >> zero latency count 671 before latency 1000 >> zero latency count 167 before latency 1000 >> zero latency count 147 before latency 1000 >> >> From the System Browser, inspecting the class variable LatencyCounts >> showed... >> 0 171329 >> 3000 1 >> 2000 3 >> 1000 49 >> > > P.S. after running a few more hours, LatencyCounts shows... > 0 1102691 > 1000 219 > 2000 12 > 3000 29 > 4000 21 > 5000 5 > 49000 1 > 75000 1 > > > It's likely that the vm code is using the cheap microsecond click which is > updated by the heartbeat, not the expensive one that interrogates the OS at > the instant. If not, it's the OS. I'll check soon. > I went looking and found an obvious source of quantitization multiplying by MicrosecondsPerMillisecond in the Win32 currentUTCMicroseconds() line 109 here... https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/ e17db79411cfec767e04f3d94d12a642d920a30e/platforms/win32/vm/ sqWin32Time.c#L109 Running the same experiment (for 20 mins) on Linux produced a latency distribution in line with what I was originally expecting... usec count 1 135 2 992 3 3250 4 21161 5 17309 6 5341 7 1230 8 313 9 79 10 44 11 46 12 50 13 36 14 58 15 73 16 38 17 12 18 7 19 5 20 1 21 3 23 2 24 4 25 4 26 5 27 4 29 3 30 6 31 5 32 6 33 5 34 2 35 2 36 5 37 4 39 4 40 2 41 2 44 3 46 2 47 1 48 4 49 3 51 2 53 1 54 1 56 2 58 1 59 1 61 1 66 2 70 1 82 1 86 1 184 1 544 1 576 1 1212 1 1402 1 2016 1 self average. "4.8"microseconds self stdev. "12.9"microseconds self average + (5 * self stdev) "69.1"microseconds So there are nine outliers outside five sigma. @Clement, Would you expect your incremental GC to have much impact on those outliers? or its target pause is ~1000usecs anyway? cheers -ben

May 27, 2018

CI memory leak testing

by Ben Coman

On 26 May 2018 at 18:21, Tudor Girba <tudor(a)tudorgirba.com> wrote: > Hi, > > Yes, Glamour is stable since years. Yes, we did have some leak issues that > appeared along the way. Last year, we discovered that a memory leak came > from Rubric (specifically there was a global variable in it). People saw > inspectors laying around, so naturally, the issue was seen as being due to > GT. And now there is another one, which is still under investigation. For > me, the main issue is not the memory leak, but the fact that we have no > tool to reason about them. > I wonder if its possible/useful to define a generic mechanism that at the end of a CI test run all windows are force closed, garbage collected and then tallied objects the check a zero count for all classes except those explicitly defined, perhaps in a class manifest. May have picked this up earlier. cheers -ben

May 26, 2018

Re: [Pharo-dev] Crash on GC garbageCollect

by Ben Coman

On 26 May 2018 at 08:27, Stephan Eggermont <stephan(a)stack.nl> wrote: > Esteban Lorenzano <estebanlm(a)gmail.com> > wrote: > > > > > >> On 25 May 2018, at 23:48, Stephan Eggermont <stephan(a)stack.nl> wrote: > >> > >> Esteban Lorenzano <estebanlm(a)gmail.com> > >> wrote: > >>> thatâs because 6.1 images still comes with an old VM. > >>> I think I will backport the newer, but next week (testing time has > passed, I think). > >> > >> You probably want to include the TLS fix from this week for Win7 so > github > >> repos eork > > > > no I do not want :) > > because in that case I will need to repeat the full cycle: 2 weeks as > > latest, one month as stable in 7.0, than backport. > > and probably in that moment there will be another fix to include. > > I understand that you donât want, but releasing a stable vm for Pharo 6 > where Iceberg (that is, github) doesnât work on Win7 is also not really a > solution. > Waiting for the another stable vm is in the realm of "someday", chasing the length-of-a-piece-of-string. Better to make the improvement that can be made today, and follow up with the next asap after that. cheers -ben

May 26, 2018

[Pharo 7.0-dev] Build #958: 21981-RBBasicLintRuleTest-class--usesTrue-should-be-removed

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #958 was: SUCCESS. The Pull Request #1421 was integrated: "21981-RBBasicLintRuleTest-class--usesTrue-should-be-removed" Pull request url: https://github.com/pharo-project/pharo/pull/1421 Issue Url: https://pharo.fogbugz.com/f/cases/21981 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

May 26, 2018

Re: [Pharo-dev] Beacon for Pharo 7

by Stephan Eggermont

Norbert Hartl <norbert(a)hartl.name> wrote: > How can pillar kill pier? They can not both be loaded. Pillar should have just forked: copy and rename the parts needed. Stephan

May 26, 2018

Re: [Pharo-dev] Beacon for Pharo 7

by Norbert Hartl

To be honest you sound a bit harsh. And Iâm not sure I can remember that such kind of arguments have ever been exchanged before something was integrated. You want to have a concrete definition. But you just said you donât want the core the rely neither on Beacon nor on Announcements. So maybe it would be good to define what is in the core so we can understand? I hope we can agree that having a logging system is a good thing. And I hope Beacon is considered the best candidate for it. In order to make it so we designed the core to be minimal for easy integration. The only consumers included in the system are the memory logger and the transcript logger. So the dependencies are none because Transcript is here to stay for the time being. Transcript logger can also be the default fallback so the system appears like it is today. One of the biggest decisions might be what method names are allowed to be highjacked by a logging framework. Will there be something like Object>>#log: (#emit:) ??? If beacon is considered to be close to the core this will be ok. If it is considered just a logger to be loaded it is not. Designing something that makes beacon replacable will most likely exceed the code base beacon is. It is kept small for that reason. Regarding Announcements. Are Announcements to be removed from the system? If not it is fine Beacon is using them. Btw. the global usage of beacon is just one use case. And a needed one because you want to have it usable from a playground, too. So it is a good fallback. The problems of Announcements should be solved in Announcements not in Beacon. Or Announcements should be deprecated. So please define what is the core of pharo and we can talk about how Beacon is best integrated. Norbert > Am 26.05.2018 um 11:26 schrieb Guillermo Polito <guillermopolito(a)gmail.com>: > > Hi, > >> On Sat, May 26, 2018 at 10:57 AM, Tudor Girba <tudor(a)tudorgirba.com> wrote: >> Hi, >> >> I do not know what you mean by Glamour. Glamour is stable since years. If you refer to the Glamorous Toolkit, then perhaps the problem comes from the fact that the existing tools and the new generation share the same name. However, what is integrated now has nothing to do with what is being developed separately. > > Really? Since years we are chasing bugs and memory leaks created by glamour and spotter... And what I see is a lot of effort on debuggging those from the people in RMod to fix those. Is that what you call stable? What I'd like is that the effort of maintaining Glamour or even the Glamorous Toolkit is shared, because you have the knowledge to debug it while me, I have to develop that knowledge... > > Right now in a vanilla Pharo 6 I've been using for some hours I do > > GTInspector allInstances. > > And I have three instances, while only one inspector is open. > >> As for Beacon, I am using it since two years, and it is as stable as software can get. >> >> The problems you describe have nothing to do with Beacon, but with Announcements. If Announcements are not safe to use, we should work on that. Beacon is an application that uses that mechanism. > > No, the problem will be there even without Announcements. Beacon relies on a global registration mechanism, otherwise it doesn't make sense. > If I register a logger in Beacon, that logger will be hanging around for ever. So we need tools to handle those. > > Announcements have other problems on their own, as we were talking with Pablo the other day. They are not reentrant (what happens if an announcement is raised during the treatment of an announcement? Does it loop?), they block the caller until all announcements are processed (not so decoupled, huh :P), right now without looking at the code it's not clear to me if they can handle the subscription of arbitrary instances instead of just Announcement (sub)classes (announcer on: arbitraryObject send: ...). Not even mentioning the weak/ephemeron mess :). > >> >> What should be rewritten are all the references to Transcript, and likely the creation of dedicated Signals. This can be done gradually, and should not delay anything else. > > I want to be concrete, please. I do not want the Kernel to rely on Beacon nor announcements to work. > > To me the idea of "all transcript usages should be rewritten" is way too abstract. > What is the list of packages that should be rewritten in Pharo? I want to be concrete. > Maybe some Transcript usages need to be removed, maybe some should be replaced. > The solution "We introduce it and eventually we replace users" will decay into "We introduce it and never replace the users" unless there is an active effort to do it, we have all seen it. > > So, who is willing to do that effort? > > Because, integrating Beacon into Pharo has also its drawbacks for users: People will not be able to freely install/upgrade the version of Beacon they want without putting the system in risk. On the other side, keeping it as an external library, people can have as a simple dependency whatever they need for their application. > > Also, didn't we agree that in Pharo7 we should try to remove something for each thing that is added? > >> >> And just to make it clear: while the stack logging is a useful too, its main purpose is to be an example to show how a dedicated Signal can enable advanced debugging and monitoring options. > > So when people say "We shoud integrate Beacon", what exactly are we talking about? > - What are the packages/features included? What loggers? What kind of events/logs? > - What are the good/bad practices of using Beacon? > >> >> Cheers, >> Doru >> >> >> > On May 26, 2018, at 9:00 AM, Guillermo Polito <guillermopolito(a)gmail.com> wrote: >> > >> > Hi all, >> > >> > Just to state my position about the integration of Beacon. My main concern now is that I do not want to it become Glamour. >> > I do not want to integrate a new tool that will be half integrated and not maintained because "the future is elsewhere". >> > >> > If people would like to introduce Beacon, I'd like to have at least the following questions answered: >> > - What tools/core libraries of Pharo should be rewritten using Beacon? >> > - And which ones do not? >> > - **Who** is going to take care for those integrations/rewritings? >> > - How will this delay (even more) the release of Pharo 7? >> > >> > Then, more on the technical side. Beacon uses *global* announcers as its log/event delivery technique. It gives people an easy way to log a full stack (with all objects it references). While all this is nice, it may generate memory leaks, people may need to have a "Logger manager" to see connected loggers and detect problems... >> > - What are the tools/techniques that we have to develop to manage connected loggers/appenders efficiently? >> > - What are the tools/techniques that we have to develop to detect and avoid memory leaks more efficiently? >> > - **Who** is going to implement them? >> > >> > If I put this side by side with the versionning questions above, this ones have more weight to me... >> > >> > Cheers, >> > Guille >> >> -- >> www.tudorgirba.com >> www.feenk.com >> >> "Some battles are better lost than fought." >> >> >> >> >> > > > > -- > > Guille Polito > Research Engineer > Centre de Recherche en Informatique, Signal et Automatique de Lille > CRIStAL - UMR 9189 > French National Center for Scientific Research - http://www.cnrs.fr > > Web: http://guillep.github.io > Phone: +33 06 52 70 66 13

May 26, 2018