July 2019 - Pharo-dev - lists.pharo.org

[Pharo 8.0] Build #589: First pass on compiler tests for full blocks

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #589 was: FAILURE. The Pull Request #4151 was integrated: "First pass on compiler tests for full blocks" Pull request url: https://github.com/pharo-project/pharo/pull/4151 Issue Url: https://github.com/pharo-project/pharo/issues/enhncement/compiler Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

July 31, 2019

[Pharo 8.0] Build #588: 4152-Cleanups-in-UIManager

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #588 was: FAILURE. The Pull Request #4153 was integrated: "4152-Cleanups-in-UIManager" Pull request url: https://github.com/pharo-project/pharo/pull/4153 Issue Url: https://github.com/pharo-project/pharo/issues/4152 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

July 31, 2019

[Pharo 8.0] Build #587: 4154-Hiedra-Merge-new-version-with-branch-coloring

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #587 was: FAILURE. The Pull Request #4155 was integrated: "4154-Hiedra-Merge-new-version-with-branch-coloring" Pull request url: https://github.com/pharo-project/pharo/pull/4155 Issue Url: https://github.com/pharo-project/pharo/issues/4154 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

July 31, 2019

Re: [Pharo-dev] Pharo process scheduling

by Ben Coman

On Mon, 29 Jul 2019 at 17:40, Alistair Grant <akgrant0710(a)gmail.com> wrote: > Hi Everyone, > > I'm trying to get a better grip on process scheduling in Pharo but > don't understand the behaviour I'm seeing. > > My understanding is that the VM (which does the real work of > scheduling the processes) is a pre-emptive co-operative model, > meaning: > > - Higher priority processes are always scheduled ahead of lower > priority processes. > - Processes at the same priority get the CPU until something is done > to explicitly stop processing, e.g.: > -- terminate the process, wait on semaphore (I/O, timer, mutex, etc.), > #yield the processor > > But given the following: > > | p x a index | > > p := 35. > a := Array new: 20. > index := 1. > [ 1 to: 10 do: [ :i | > [ a at: index put: i. > index := index + 1. > x := 0. > 300000000 timesRepeat: [ x := x + 1 ]. > a at: index put: i. > index := index + 1. ] forkAt: p named: 'Proc', i asString ]. > ] forkAt: p+1. > a. > > > The code above was written to be as simple as possible (it isn't going > to be creating streams, have semaphores etc. called somewhere deep > within). The 300000000 number was manually determined as something > that will run for a few seconds on my laptop. > > Since the forking process is higher priority than the 10 worker > processes, I expect that all 10 processes will be created before the > first gets any CPU, and they will then be executed in order (since > there is nothing obvious to cause a context switch), and the results > to be: > > #(1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10) > Wow, I find I've been working under a mis-understanding for years. I thought this was how Pharo operated. I just checked Squeak. It gives this result since it has... Smalltalk vm processPreemptionYields "==> false" Pharo has... Smalltalk vm processPreemptionYields "==> true" since at least Pharo 2.0. Some pertitant background... [1] http://forum.world.st/User-interrupt-does-not-always-break-UI-process-tp361… [2] http://forum.world.st/Stopping-a-Smalltalk-process-from-lldb-gdb-tp4917963p… So would Pharo having "processPreemptionYields: false" be beneficial to match Squeak & VisualWorks? btw, what does "safer" [1] mean? Or... even though we are a long way from running multi-CPU, would a system working with "processPreemptionYields=>false" be more ready to run threaded on multi-CPU? Or is that too far away to be concerned with? How would assumptions around "processPreemptionYields=>false" be affected by FFI callbacks ? cheers -ben

July 31, 2019

Re: [Pharo-dev] Pharo process scheduling

by ducasse

Thank you guy for the discussion. This is still on my todo to have version in english of the chapter we wrote for the squeak book. Stef > On 30 Jul 2019, at 20:14, Alistair Grant <akgrant0710(a)gmail.com> wrote: > > Hi Henrik, > > I have to admit that I'm having a bit of trouble following what you're > saying, so I'll make a few comments that I hope are helpful... > > Just to start off, my understanding of the term "pre-emptive, > cooperative" processing is: > > - a higher priority process will interrupt a lower priority process and > run until it voluntarily waits (or is suspended or terminated). > - "cooperative" should be read as "unless the processes cooperate, one > bad actor can stop anyone else from getting the CPU". > > > On Tue, Jul 30, 2019 at 05:32:13AM -0500, Henrik Sperre Johansen wrote: >> Alistair Grant wrote >>> The following produces what I was expecting: >>> >>> | p x a index first | >>> >>> Smalltalk vm processPreemptionYields: false. >>> >>> >>> Thanks again! >>> Alistair >> >> Well... what your example shows, is that with processes >> p50 (pri50) periodic, becoming runnable in time slot 2) >> p40.1 (taking 3 time slots to complete) >> p40.2 (taking 3 time slots to complete) >> >> When preemption yields, you get processor time allocations like this: >> [p40.1][p50][p40.2][p40.1][p40.2][p40.1][p40.2] > > I think this might be demonstrating an incorrect expectation on your > part. > > Your use of the term "time slot" seems to imply that, within a given > process priority, each process can expect roughly equal CPU time. > However that isn't what is implied. With processPreemptionYields: true > *any* higher priority process becoming runnable will cause the currently > executing process to be moved to the back of the queue. In practice, > where processes are running at a priority of less than 60, this means > that with every mouse move, keyboard input, timer event, etc., the > active process will be moved to the back of the queue. The amount of > time allocated to each process is effectively random. > > So in the above example, assuming that the p50 process only becomes > runnable once, and no other high priority process becomes runnable, we > can expect: > > [p40.1][p50][p40.2][p40.1] > > >> With preemption NOT yielding, you get processor time allocations like this: >> [p40.1][p50][p40.1][p40.1][p40.2][p40.2][p40.2] > > I'd expect: > > [p40.1][p50][p40.1][p40.2] > > (running the example code used earlier in VW also supports this) > > >> Which clearly contradicts the statements in the linked post >> "Nothing changes within priorities. > > I haven't looked up the original post, but it seems out of > context. > > >> What the setting controls is what >> happens when a process is preempted by a higher-priority process. The old >> code did an implicit yield of the preempted process, destroying the >> guarantee of cooperative scheduling within a priority." > > This matches my understanding (remember that "cooperative" means that a > process has to voluntarily relinquish the CPU). > > >> Which (to me) would imply slot allocations when not yielding would be like >> this: >> [p40.1][p50][p40.1][p40.2][p40.1][p40.2][p40.2] >> >> Maybe it's what is intended and the statements are inaccurate, or Pharo is >> lacking some related changes, but it doesn't make for a good experience when >> you have multiple processes requiring "real-time" execution running at the >> same priority if you have to add explicit Processor yield's everywhere. > > Actually, this matches exactly my understanding of real-time > processing. E.g. in VAX/VMS there were two tiers of process priority: > > - the low priority tier, e.g. 1 - 40 (I can't remember the actual > numbers) were time sliced. So if multiple processes were at the same > priority, they could expect roughly equal amounts of CPU time. > - the high priority tier, e.g. 41 - 80, were cooperative. So if > multiple processes were at the same priority, the second one had to > wait for the first to voluntarily give up the CPU. > The expectation in this case was that any time a real-time process had > work to do, the time required would be reasonably short, and that it > was important to let it have the CPU until it had completed that work. > > > > On Tue, Jul 30, 2019 at 05:53:52AM -0500, Henrik Sperre Johansen wrote: >> Henrik Sperre Johansen wrote >>> Which clearly contradicts the statements in the linked post >>> "Nothing changes within priorities. What the setting controls is what >>> happens when a process is preempted by a higher-priority process. The old >>> code did an implicit yield of the preempted process, destroying the >>> guarantee of cooperative scheduling within a priority." >> >> Or it could mean the understanding was that processes at the same priority >> would have to explicitly yield for other processes of the same priority to >> run also with preemptionyields = true. >> ... which, seems to be true. >> >> It's just that we never see that in practice, even running at pri 80, as the >> DelaySemaphoreScheduler (which runs at 80) is at the head of the list, and >> gets woken every now and then. >> Reducing it's pri to 70, and then doing the same test with threads at pri80 >> (but containing plenty of suspension points), we indeed get >> 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 > > You get this result even if you don't lower the priority of the timer > process. If the sample code is run at priority 80 it will continue > until it has finished, and the DelaySemaphoreScheduler has to wait for > it to finish (which has other bad side effects, but we don't care about > those here). > > >> TIL, I guess :) >> >> Cheers, >> Henry > > > Hope this helps, > Alistair

July 30, 2019

Re: [Pharo-dev] Pharo process scheduling

by Alistair Grant

Hi Henrik, I have to admit that I'm having a bit of trouble following what you're saying, so I'll make a few comments that I hope are helpful... Just to start off, my understanding of the term "pre-emptive, cooperative" processing is: - a higher priority process will interrupt a lower priority process and run until it voluntarily waits (or is suspended or terminated). - "cooperative" should be read as "unless the processes cooperate, one bad actor can stop anyone else from getting the CPU". On Tue, Jul 30, 2019 at 05:32:13AM -0500, Henrik Sperre Johansen wrote: > Alistair Grant wrote > > The following produces what I was expecting: > > > > | p x a index first | > > > > Smalltalk vm processPreemptionYields: false. > > > > > > Thanks again! > > Alistair > > Well... what your example shows, is that with processes > p50 (pri50) periodic, becoming runnable in time slot 2) > p40.1 (taking 3 time slots to complete) > p40.2 (taking 3 time slots to complete) > > When preemption yields, you get processor time allocations like this: > [p40.1][p50][p40.2][p40.1][p40.2][p40.1][p40.2] I think this might be demonstrating an incorrect expectation on your part. Your use of the term "time slot" seems to imply that, within a given process priority, each process can expect roughly equal CPU time. However that isn't what is implied. With processPreemptionYields: true *any* higher priority process becoming runnable will cause the currently executing process to be moved to the back of the queue. In practice, where processes are running at a priority of less than 60, this means that with every mouse move, keyboard input, timer event, etc., the active process will be moved to the back of the queue. The amount of time allocated to each process is effectively random. So in the above example, assuming that the p50 process only becomes runnable once, and no other high priority process becomes runnable, we can expect: [p40.1][p50][p40.2][p40.1] > With preemption NOT yielding, you get processor time allocations like this: > [p40.1][p50][p40.1][p40.1][p40.2][p40.2][p40.2] I'd expect: [p40.1][p50][p40.1][p40.2] (running the example code used earlier in VW also supports this) > Which clearly contradicts the statements in the linked post > "Nothing changes within priorities. I haven't looked up the original post, but it seems out of context. > What the setting controls is what > happens when a process is preempted by a higher-priority process. The old > code did an implicit yield of the preempted process, destroying the > guarantee of cooperative scheduling within a priority." This matches my understanding (remember that "cooperative" means that a process has to voluntarily relinquish the CPU). > Which (to me) would imply slot allocations when not yielding would be like > this: > [p40.1][p50][p40.1][p40.2][p40.1][p40.2][p40.2] > > Maybe it's what is intended and the statements are inaccurate, or Pharo is > lacking some related changes, but it doesn't make for a good experience when > you have multiple processes requiring "real-time" execution running at the > same priority if you have to add explicit Processor yield's everywhere. Actually, this matches exactly my understanding of real-time processing. E.g. in VAX/VMS there were two tiers of process priority: - the low priority tier, e.g. 1 - 40 (I can't remember the actual numbers) were time sliced. So if multiple processes were at the same priority, they could expect roughly equal amounts of CPU time. - the high priority tier, e.g. 41 - 80, were cooperative. So if multiple processes were at the same priority, the second one had to wait for the first to voluntarily give up the CPU. The expectation in this case was that any time a real-time process had work to do, the time required would be reasonably short, and that it was important to let it have the CPU until it had completed that work. On Tue, Jul 30, 2019 at 05:53:52AM -0500, Henrik Sperre Johansen wrote: > Henrik Sperre Johansen wrote > > Which clearly contradicts the statements in the linked post > > "Nothing changes within priorities. What the setting controls is what > > happens when a process is preempted by a higher-priority process. The old > > code did an implicit yield of the preempted process, destroying the > > guarantee of cooperative scheduling within a priority." > > Or it could mean the understanding was that processes at the same priority > would have to explicitly yield for other processes of the same priority to > run also with preemptionyields = true. > ... which, seems to be true. > > It's just that we never see that in practice, even running at pri 80, as the > DelaySemaphoreScheduler (which runs at 80) is at the head of the list, and > gets woken every now and then. > Reducing it's pri to 70, and then doing the same test with threads at pri80 > (but containing plenty of suspension points), we indeed get > 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 You get this result even if you don't lower the priority of the timer process. If the sample code is run at priority 80 it will continue until it has finished, and the DelaySemaphoreScheduler has to wait for it to finish (which has other bad side effects, but we don't care about those here). > TIL, I guess :) > > Cheers, > Henry Hope this helps, Alistair

July 30, 2019

Re: [Pharo-dev] [Pharo-users] [ANN] OSWinSubprocess a library to spawn Windows System processes

by Julien

Done. https://github.com/pharo-open-documentation/awesome-pharo/pull/89 <https://github.com/pharo-open-documentation/awesome-pharo/pull/89> Julien --- Julien Delplanque Doctorant Ã lâUniversitÃ© de Lille http://juliendelplanque.be/phd.html Equipe Rmod, Inria BÃ¢timent B 40, Avenue Halley 59650 Villeneuve d'Ascq NumÃ©ro de tÃ©lÃ©phone: +333 59 35 86 40 > Le 9 juil. 2019 Ã 08:43, ducasse <stepharo(a)netcourrier.com> a Ã©crit : > > It would be good to add this in the awesome documentation. > > Stef > >> On 8 Jul 2019, at 20:44, Torsten Bergmann <astares(a)gmx.de> wrote: >> >> Without having a deeper look I guess what was done for Windows now is wrapping the usual STARTUPINFO, CreateProcess() >> functions and friends yet again ... >> >> >> Such wrappers and many other native friends were created from my side already years ago with my "OS-XXX" series >> of projects for different platforms: >> >> - http://smalltalkhub.com/#!/~OS/OS-Windows >> - http://smalltalkhub.com/#!/~OS/OS-OSX >> - http://smalltalkhub.com/#!/~OS/OS-Unix >> - http://smalltalkhub.com/#!/~OS/OS-Linux-CentOS >> - http://smalltalkhub.com/#!/~OS/OS-Linux-Ubuntu >> - http://smalltalkhub.com/#!/~OS/OS-Raspbian >> >> So for instance on Windows you could easily use >> >> WinProcess createProcess: 'explorer.exe' >> >> or >> >> WinProcess createAndWaitForProcess: 'cmd.exe'. >> >> The projects are available on Github now since 2018 and kept up to date for recent Pharo versions. >> >> - https://github.com/astares/Pharo-OS-Windows >> - https://github.com/astares/Pharo-OS-OSX >> - https://github.com/astares/Pharo-OS-Unix >> - https://github.com/astares/Pharo-OS-Linux-Ubuntu >> - https://github.com/astares/Pharo-OS-Raspbian >> >> All of them are available in Catalog. >> >> I have no problem with more and more implementations and the double efforts. But for Pharo users I have the feeling >> that it gets more and more confusing in the area of OSProcess, OSSubProcess, OSWinSubprocess, .... and more wrappers >> for basically the same thing ... >> >> Bye >> Torsten (aka astares) >> >> > > >

July 30, 2019

Re: [Pharo-dev] Pharo process scheduling

by Henrik Sperre Johansen

Henrik Sperre Johansen wrote > Which clearly contradicts the statements in the linked post > "Nothing changes within priorities. What the setting controls is what > happens when a process is preempted by a higher-priority process. The old > code did an implicit yield of the preempted process, destroying the > guarantee of cooperative scheduling within a priority." Or it could mean the understanding was that processes at the same priority would have to explicitly yield for other processes of the same priority to run also with preemptionyields = true. ... which, seems to be true. It's just that we never see that in practice, even running at pri 80, as the DelaySemaphoreScheduler (which runs at 80) is at the head of the list, and gets woken every now and then. Reducing it's pri to 70, and then doing the same test with threads at pri80 (but containing plenty of suspension points), we indeed get 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 TIL, I guess :) Cheers, Henry -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

July 30, 2019

Re: [Pharo-dev] Pharo process scheduling

by Henrik Sperre Johansen

Alistair Grant wrote > The following produces what I was expecting: > > | p x a index first | > > Smalltalk vm processPreemptionYields: false. > > > Thanks again! > Alistair Well... what your example shows, is that with processes p50 (pri50) periodic, becoming runnable in time slot 2) p40.1 (taking 3 time slots to complete) p40.2 (taking 3 time slots to complete) When preemption yields, you get processor time allocations like this: [p40.1][p50][p40.2][p40.1][p40.2][p40.1][p40.2] With preemption NOT yielding, you get processor time allocations like this: [p40.1][p50][p40.1][p40.1][p40.2][p40.2][p40.2] Which clearly contradicts the statements in the linked post "Nothing changes within priorities. What the setting controls is what happens when a process is preempted by a higher-priority process. The old code did an implicit yield of the preempted process, destroying the guarantee of cooperative scheduling within a priority." Which (to me) would imply slot allocations when not yielding would be like this: [p40.1][p50][p40.1][p40.2][p40.1][p40.2][p40.2] Maybe it's what is intended and the statements are inaccurate, or Pharo is lacking some related changes, but it doesn't make for a good experience when you have multiple processes requiring "real-time" execution running at the same priority if you have to add explicit Processor yield's everywhere. Cheers, Henry -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

July 30, 2019

[Pharo 8.0] Build #586: Add-some-icons-to-commands

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #586 was: FAILURE. The Pull Request #4131 was integrated: "Add-some-icons-to-commands" Pull request url: https://github.com/pharo-project/pharo/pull/4131 Issue Url: https://github.com/pharo-project/pharo/issues/AddIconsToCommands Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

July 30, 2019