- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Never ending BlockClosure>>benchFor:

by Ben Coman

Could you try the various DelayExperimentalXXXSchedulers? Under System > Settings > System > Delay Scheduler. I believe there's a good chance one of them may fix also the problem. I had planned to push one of these as the default in Pharo 5 but hadn't decided which, got distracted and stretched a bit thin IRL. I believe these experimental schedulers are better than the current mutex based scheduler, since passing Delays from your normal-priority process to the timing-priority delay scheduling process is a signalling paradigm more than a shared memory paradigm. I came to this conclusion reading [2] which says... "The correct use of a semaphore is for signalling from one task to another. A mutex is meant to be taken and released, always in that order, by [the same task using] the shared resource it protects. By contrast, tasks that use semaphores either signal or waitânot both. ... Another important distinction between a mutex and a semaphore is that the proper use of a mutex to protect a shared resource can have a dangerous unintended side effect ... any two tasks that operate at different priorities and coordinate via a mutex, create the opportunity for priority inversion ... Fortunately ... semaphores ... do not cause priority inversion when used for signalling." A mutex makes DelayXXXScheduler>>#schedule: susceptible to priority inversion since it is used by many processes at different priority levels. Using a signalling-semaphore might avoid the problem. The experimental delay schedulers were developed to address the problem reported in the "super fast delay" [2], but there may also have been a fix for that through another means. [1] http://www.barrgroup.com/Embedded-Systems/How-To/RTOS-Mutex-Semaphore [2] http://forum.world.st/Super-fast-delay-td4787257.html cheers -ben On Fri, Nov 20, 2015 at 8:03 PM, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > > 2015-11-20 12:46 GMT+01:00 Martin Dias <tinchodias(a)gmail.com>: >> >> Hi! >> >> With Guille and Pablo this morning we found what is the problem and >> apparently fixed it. The problem is that there is a process in priority 10 >> that takes the delay lock (accessProtect semaphore) and go to sleep. Then >> the #benchFor: launches another process in priority 79 that tries to take >> the delay lock but, since it's taken by the other process, it goes to sleep. >> But the UI process is in priority 40 so it will always have priority over >> the one that has the lock. Consequence: the process in priority 10 has the >> delay lock and will never be awaken until the UI process gets suspended. > > > I believe this is a text book example of priority inversion. > > Kudo's for finding it! > > Thierry > >> >> >> More in detail, the following method makes the assumption that once the >> delay process finishes the handling of the delay to schedule, it will return >> to same process. But that is not true if there is another process with more >> priority. >> >> schedule: aDelay >> | scheduled | >> scheduled := false. >> aDelay schedulerBeingWaitedOn ifTrue: [^self error: 'This Delay has >> already been scheduled.']. >> accessProtect critical: [ >> scheduledDelay := aDelay. >> timingSemaphore signal. "#handleTimerEvent: sets scheduledDelay:=nil" >> scheduled := (scheduledDelay == nil). >> ]. >> ^scheduled. >> >> Our solution is to run the critical section in a higher priority to ensure >> that always comes back to the same process to release the delay lock. >> >> schedule: aDelay | scheduled | scheduled := false. aDelay >> schedulerBeingWaitedOn ifTrue: [ ^ self error: 'This Delay has already been >> scheduled.' ]. [ accessProtect critical: [ scheduledDelay := aDelay. >> timingSemaphore signal. "#handleTimerEvent: sets scheduledDelay:=nil" >> scheduled := scheduledDelay == nil ]. ^ scheduled ] valueAt: Processor >> timingPriority - 1 >> >> Also, to not have interference with the benchmarking process we reduce the >> priority of the process used in benchFor: >> >> benchFor: duration >> "Run me for duration and return a BenchmarkResult" >> "[ 100 factorial ] benchFor: 2 seconds" >> | count run started | >> count := 0. >> run := true. >> [ duration wait. run := false ] forkAt: Processor timingPriority - 2. >> started := Time millisecondClockValue. >> [ run ] whileTrue: [ self value. count := count + 1 ]. >> ^ BenchmarkResult new >> iterations: count; >> elapsedTime: (Time millisecondsSince: started) milliSeconds; >> yourself >> >> >> We will create an entry in fogbugz and submit the change. >> >> Regards, >> Martin, Guille and Pablo >> >> >> >> On Wed, Nov 18, 2015 at 8:33 PM, Max Leske <maxleske(a)gmail.com> wrote: >>> >>> I went ahead and implemented a simple fix for the styler. Not sure how >>> well it performs but in my simple experiments it seems to work fine. Please >>> test it and let me know what you think. >>> >>> >>> https://pharo.fogbugz.com/f/cases/17050/SHTextStyler-styleInBackgroundProce… >>> >>> Cheers, >>> Max >>> >>> >>> > On 18 Nov 2015, at 19:02, Andreas Wacknitz <A.Wacknitz(a)gmx.de> wrote: >>> > >>> > >>> > >>> > Am 18.11.15 um 11:48 schrieb Andrei Chis: >>> >> Can you try the script below in the latest Pharo image and let me know >>> >> how much does it take to execute on your machine: >>> >> >>> >> |duration benchmarkResult| >>> >> 100 timesRepeat: [ >>> >> RubScrolledTextMorph new >>> >> model: (RubScrolledTextModel new setInitialText: '1+1'; yourself); >>> >> beForSmalltalkScripting; >>> >> yourself >>> >> ] . >>> >> >>> >> duration := 100 milliSeconds. >>> >> benchmarkResult := [ 100 factorial ] benchFor: duration. >>> >> >>> >> In my case it takes around 1 minute. The inner loop finishes >>> >> immediately and then most time is spend executing the benchmark. >>> >> >>> >> At the end I get the following result: >>> >> "a BenchmarkResult(2,060,386 iterations in 1 minute 7 seconds 498 >>> >> milliseconds. 30,525 per second)" >>> >> >>> >> So the benchmark is executed for over one minute before the delay >>> >> expires. >>> >> >>> >> >>> >> >>> > "a BenchmarkResult(1,263,573 iterations in 37 seconds 689 milliseconds. >>> > 33,526 per second)" >>> > (HP Z420 with Xeon E5-1620 under OpenIndiana). >>> > >>> > Regards >>> > Andreas >>> >>> >> >

Nov. 20, 2015

Re: [Pharo-dev] Kernel-Tests and SystemAnnouncer

by Andrei Chis

On Fri, Nov 20, 2015 at 12:44 PM, Peter UhnÃ¡k <i.uhnak(a)gmail.com> wrote: > And not just announcers, some of the tests there leave quite a footprint. > Indeed some tests are not designed to be silent. > I've recently discussed something similar with regards to performance > of subclassing in tests > > http://forum.world.st/fast-subclassing-class-creation-in-tests-td4857779.ht… > So Kernel-Tests might also benefit from it. > > Right now I've also encountered this thread > > http://forum.world.st/New-class-factory-for-tests-and-other-use-cases-td129… > , it's quite old (2009), but maybe it could be used instead of > directly manipulating the env. > > On the other hand, how often do you manually trigger Kernel-Tests? CI > doesn't care that much about it. > Our current issue is that we need to update something in an application based on system announcers. The fact that tests trigger events generates extra work which makes some builds longer. Changing the tests would take too much time, so we'll look for alternatives. Cheers, Andrei > > Peter > > On Fri, Nov 20, 2015 at 12:21 PM, Andrei Chis > <chisvasileandrei(a)gmail.com> wrote: > > Hi, > > > > Ideally shouldn't Kernel-Tests, or tests in general, be silent and not > > trigger a lot of system announcers? > > Right now Kernel-Tests trigger around 250 announcements through the > > SystemAnnouncer. > > > > Cheers, > > Andrei > >

Nov. 20, 2015

Issue with large file path on windows

by Blondeau Vincent

Hello, I would like to read from Pharo a file with a long name (so more than 255 chars) under Windows. A thread from last month (13 oct): "Re: [Pharo-dev] [Vm-dev] Re: issue with large file path on windows", talks about a patch. Does it has been integrated? If it has, which VM should I take to have the patch? My current one is http://files.pharo.org/vm/pharo/win/494.zip Thanks, Vincent Blondeau ________________________________ Ce message et les pi?ces jointes sont confidentiels et r?serv?s ? l'usage exclusif de ses destinataires. Il peut ?galement ?tre prot?g? par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir imm?diatement l'exp?diteur et de le d?truire. L'int?grit? du message ne pouvant ?tre assur?e sur Internet, la responsabilit? de Worldline ne pourra ?tre recherch?e quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'exp?diteur ne donne aucune garantie ? cet ?gard et sa responsabilit? ne saurait ?tre recherch?e pour tout dommage r?sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.

Nov. 20, 2015

Smalltalkhub is very instable this afternoon

by Blondeau Vincent

Hello, Can Smalltalkhub be restarted (or other) because it is impossible to work by getting continuously errors 500, 503, 403, 401.... while loading or committing? TIA Cheers, Vincent BLONDEAU ________________________________ Ce message et les pi?ces jointes sont confidentiels et r?serv?s ? l'usage exclusif de ses destinataires. Il peut ?galement ?tre prot?g? par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir imm?diatement l'exp?diteur et de le d?truire. L'int?grit? du message ne pouvant ?tre assur?e sur Internet, la responsabilit? de Worldline ne pourra ?tre recherch?e quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'exp?diteur ne donne aucune garantie ? cet ?gard et sa responsabilit? ne saurait ?tre recherch?e pour tout dommage r?sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.

Nov. 20, 2015

Re: [Pharo-dev] Never ending BlockClosure>>benchFor:

by Martin Dias

On Fri, Nov 20, 2015 at 2:27 PM, Andrei Chis <chisvasileandrei(a)gmail.com> wrote: > Great catch! > > Can something similar be also the > reason SHTextStyler>>styleInBackgroundProcess: spawns zombie processes? > Even with this fix script [1] blocks in testBenchFor. If I apply the fix > proposed by ben in this thread or the slice for issue 17050 > <https://pharo.fogbugz.com/f/cases/17050/SHTextStyler-styleInBackgroundProce…> then > [1] does not block in testBenchFor. > > [1] ./pharo Pharo.image eval "CommandLineTestRunner runClasses: > {GTInspectorExamplesTest} named: 'x'. CommandLineTestRunner runClasses: > {BlockClosureTest} named: 'y'" > > I confirm! Andrei's script hangs when loading 17066's slice without 17050's slice (and post-load do it). So I set 17050 as a subcase of 17066. Martin

Nov. 20, 2015

Refactoring and traitsâ¦

by Yuriy Tymchuk

Hi guys, is it a known issue that if you are refactoring something (eg. doing a class rename) and some changes should happen in a method which comes from trait, then new methods are compiled in concrete classes instead of changing the trait method? Cheers. Uko

Nov. 20, 2015

Re: [Pharo-dev] Never ending BlockClosure>>benchFor:

by Andrei Chis

Great catch! Can something similar be also the reason SHTextStyler>>styleInBackgroundProcess: spawns zombie processes? Even with this fix script [1] blocks in testBenchFor. If I apply the fix proposed by ben in this thread or the slice for issue 17050 <https://pharo.fogbugz.com/f/cases/17050/SHTextStyler-styleInBackgroundProce…> then [1] does not block in testBenchFor. [1] ./pharo Pharo.image eval "CommandLineTestRunner runClasses: {GTInspectorExamplesTest} named: 'x'. CommandLineTestRunner runClasses: {BlockClosureTest} named: 'y'" On Fri, Nov 20, 2015 at 12:46 PM, Martin Dias <tinchodias(a)gmail.com> wrote: > Hi! > > With Guille and Pablo this morning we found what is the problem and > apparently fixed it. The problem is that there is a process in priority 10 > that takes the delay lock (accessProtect semaphore) and go to sleep. Then > the #benchFor: launches another process in priority 79 that tries to take > the delay lock but, since it's taken by the other process, it goes to > sleep. But the UI process is in priority 40 so it will always have priority > over the one that has the lock. Consequence: the process in priority 10 has > the delay lock and will never be awaken until the UI process gets > suspended. > > More in detail, the following method makes the assumption that once the > delay process finishes the handling of the delay to schedule, it will > return to same process. But that is not true if there is another process > with more priority. > > schedule: aDelay > | scheduled | > scheduled := false. > aDelay schedulerBeingWaitedOn ifTrue: [^self error: 'This Delay has > already been scheduled.']. > accessProtect critical: [ > scheduledDelay := aDelay. > timingSemaphore signal. "#handleTimerEvent: sets scheduledDelay:=nil" > scheduled := (scheduledDelay == nil). > ]. > ^scheduled. > > Our solution is to run the critical section in a higher priority to ensure > that always comes back to the same process to release the delay lock. > > schedule: aDelay | scheduled | scheduled := false. aDelay > schedulerBeingWaitedOn ifTrue: [ ^ self error: 'This Delay has already been > scheduled.' ]. [ accessProtect critical: [ scheduledDelay := aDelay. > timingSemaphore signal. "#handleTimerEvent: sets scheduledDelay:=nil" > scheduled := scheduledDelay == nil ]. ^ scheduled ] valueAt: Processor > timingPriority - 1 > > Also, to not have interference with the benchmarking process we reduce the > priority of the process used in benchFor: > > benchFor: duration > "Run me for duration and return a BenchmarkResult" > "[ 100 factorial ] benchFor: 2 seconds" > | count run started | > count := 0. > run := true. > [ duration wait. run := false ] forkAt: Processor timingPriority *- 2*. > started := Time millisecondClockValue. > [ run ] whileTrue: [ self value. count := count + 1 ]. > ^ BenchmarkResult new > iterations: count; > elapsedTime: (Time millisecondsSince: started) milliSeconds; > yourself > > > We will create an entry in fogbugz and submit the change. > > Regards, > Martin, Guille and Pablo > > > > On Wed, Nov 18, 2015 at 8:33 PM, Max Leske <maxleske(a)gmail.com> wrote: > >> I went ahead and implemented a simple fix for the styler. Not sure how >> well it performs but in my simple experiments it seems to work fine. Please >> test it and let me know what you think. >> >> >> https://pharo.fogbugz.com/f/cases/17050/SHTextStyler-styleInBackgroundProce… >> >> Cheers, >> Max >> >> >> > On 18 Nov 2015, at 19:02, Andreas Wacknitz <A.Wacknitz(a)gmx.de> wrote: >> > >> > >> > >> > Am 18.11.15 um 11:48 schrieb Andrei Chis: >> >> Can you try the script below in the latest Pharo image and let me know >> how much does it take to execute on your machine: >> >> >> >> |duration benchmarkResult| >> >> 100 timesRepeat: [ >> >> RubScrolledTextMorph new >> >> model: (RubScrolledTextModel new setInitialText: '1+1'; yourself); >> >> beForSmalltalkScripting; >> >> yourself >> >> ] . >> >> >> >> duration := 100 milliSeconds. >> >> benchmarkResult := [ 100 factorial ] benchFor: duration. >> >> >> >> In my case it takes around 1 minute. The inner loop finishes >> immediately and then most time is spend executing the benchmark. >> >> >> >> At the end I get the following result: >> >> "a BenchmarkResult(2,060,386 iterations in 1 minute 7 seconds 498 >> milliseconds. 30,525 per second)" >> >> >> >> So the benchmark is executed for over one minute before the delay >> expires. >> >> >> >> >> >> >> > "a BenchmarkResult(1,263,573 iterations in 37 seconds 689 milliseconds. >> 33,526 per second)" >> > (HP Z420 with Xeon E5-1620 under OpenIndiana). >> > >> > Regards >> > Andreas >> >> >> >

Nov. 20, 2015

Re: [Pharo-dev] Does Pharo stores original author of methods?

by Marcus Denker

> On 20 Nov 2015, at 10:03, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > > Maybe we should include this information in new changes system (Epicea?) epicea data is not stored forever. It is seasion based and the shipped image is clean. This information needs to be in the repository. That is why one has a repository of code. > > 2015-11-20 13:50 GMT+01:00 Marcus Denker <marcus.denker(a)inria.fr <mailto:marcus.denker@inria.fr>>: > > > On 20 Nov 2015, at 09:46, Denis Kudriashov <dionisiydk(a)gmail.com <mailto:dionisiydk@gmail.com>> wrote: > > > > Hi. > > > > Is information about original authors of methods exists in sources of image? > No. > > > I found only authors of last methods modification. Nautilus combobox source/timestamp. > > > > It will be useful to know who introduce some particular method to ask him questions directly. Now when we doing refactoring many methods become touched and this information become lost (searching repositories is not very easy now). > > Yes, repository search needs to be better. But we know that since 10 years. > > Marcus >

Nov. 20, 2015

Re: [Pharo-dev] Does Pharo stores original author of methods?

by Dimitris Chloupis

or just use git *run and hides* On Fri, Nov 20, 2015 at 3:04 PM Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > Maybe we should include this information in new changes system (Epicea?) > > 2015-11-20 13:50 GMT+01:00 Marcus Denker <marcus.denker(a)inria.fr>: > >> >> > On 20 Nov 2015, at 09:46, Denis Kudriashov <dionisiydk(a)gmail.com> >> wrote: >> > >> > Hi. >> > >> > Is information about original authors of methods exists in sources of >> image? >> No. >> >> > I found only authors of last methods modification. Nautilus combobox >> source/timestamp. >> > >> > It will be useful to know who introduce some particular method to ask >> him questions directly. Now when we doing refactoring many methods become >> touched and this information become lost (searching repositories is not >> very easy now). >> >> Yes, repository search needs to be better. But we know that since 10 >> years. >> >> Marcus >> > >

Nov. 20, 2015

Re: [Pharo-dev] Never ending BlockClosure>>benchFor:

by Martin Dias

On Fri, Nov 20, 2015 at 2:00 PM, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > > 2015-11-20 13:44 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > >> >> > On 20 Nov 2015, at 13:03, Thierry Goubier <thierry.goubier(a)gmail.com> >> wrote: >> > >> > >> > >> > 2015-11-20 12:46 GMT+01:00 Martin Dias <tinchodias(a)gmail.com>: >> > Hi! >> > >> > With Guille and Pablo this morning we found what is the problem and >> apparently fixed it. The problem is that there is a process in priority 10 >> that takes the delay lock (accessProtect semaphore) and go to sleep. Then >> the #benchFor: launches another process in priority 79 that tries to take >> the delay lock but, since it's taken by the other process, it goes to >> sleep. But the UI process is in priority 40 so it will always have priority >> over the one that has the lock. Consequence: the process in priority 10 has >> the delay lock and will never be awaken until the UI process gets suspended. >> > >> > I believe this is a text book example of priority inversion. >> > >> > Kudo's for finding it! >> >> Great work, great team ! >> >> Also a bit scary. >> > > No. And yes. > > No, because its inherent to the Pharo scheduler. > > Yes, because anywhere you acquire a non-local lock in Pharo in a process > with less that max priority, it may happen. And that the cooperative nature > of the Pharo scheduler increase the risks. > Ah! DelayMillisecondScheduler should be fixed too.

Nov. 20, 2015