Hi I wanted to explain | semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork. p2 := [semaphore signal. 'p2' crTrace ] fork. displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal. - ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists. Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not? signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive." <primitive: 85> self primitiveFailed "self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]" I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor. I will look in the VM code. S S.
here is what Iâm writing in the book. Let us image that we have the following two processes and one semaphore. [[[ | semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork. p2 := [ semaphore signal. 'p2' crTrace ] fork. ]]] - The first process is waiting on the semaphore. As soon as the semaphore will be signalled, ==p1== is signal also the semaphore. - The second process is just signalling the semaphore and printing. Now the question is to understand what is the generated trace which is ==p2== then ==p1==. Let us explain why. - ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedules it by adding it to the end of the suspended lists. ==p2== continues its execution. Let us slightly change the example to show how priorities are involved during the signalling process. [[[ | semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] forkAt: 30. p2 := [semaphore signal. 'p2' crTrace ] forkAt: 20. ]] As a conclusion when the semaphore is signalled, the first waiting process of the semaphore is removed from the semaphore list, and resumed following the preemption rules of the scheduler.
On 9 Jan 2020, at 13:01, ducasse <stepharo@netcourrier.com> wrote:
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote:
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run. So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this: "A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one." "When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore." "When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented. Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
-- _,,,^..^,,,_ best, Eliot
Maybe to add this into the class comment, this is the most concise and clear description of how it works i've ever seen пÑ, 10 Ñнв. 2020 г., 8:13 Eliot Miranda <eliot.miranda@gmail.com>:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote:
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
-- _,,,^..^,,,_ best, Eliot
Actually, it is just a, albeit concise, description of how Semaphores are implemented. It does not help much in understanding them, in learning how they can/should be used, for what purposes and how code behaves. Understanding of Process, priorities and Scheduling are also needed for a more complete understanding. This is not a simple subject. Read https://en.wikipedia.org/wiki/Semaphore_(programming) and see how well you understand the subject. In short, it does not answer Stef's concrete question(s).
On 10 Jan 2020, at 06:30, Danil Osipchuk <danil.osipchuk@gmail.com> wrote:
Maybe to add this into the class comment, this is the most concise and clear description of how it works i've ever seen
пÑ, 10 Ñнв. 2020 г., 8:13 Eliot Miranda <eliot.miranda@gmail.com>:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
-- _,,,^..^,,,_ best, Eliot
I didn't claim expertise on the subject (although I use semaphores extensively), nor its simplicity, nor that the implementation description should be the only guide on its usage (hence 'to add..., how it works' wording) Said that, to me it is the case, when a clear description of what is going on aids a lot. Instead of trying to define some rules and scenarios abstractly - to help a user to reason about the system behavior (isn't Stef was willing to look into VM code for this reason?). To me both scenarios of Stef could be explained that in first case the 'signal' process is not getting preempted by the 'wait' process of the same priority, while in second the preemption happens upon return from primitive (hopefully my memory serves me well and my understanding is correct). A tangent note on comments in general -- I've noticed more than once, that people tend to produce far clearer descriptions in exchanges like this -- when discussing matter with others. When a person is in documentation/comment writing mode he/she is sort of tenses up in a formal state and often produces something not very helpful. Current class comment of Semaphore is a perfect example, if I were not familiar with the concept from other sources, I would not be able to make any sense of it. So, I would suggest to use opportunities like this to improve comments/docs when a bit of knowledge shows up in a discussion. regards, Danil пÑ, 10 Ñнв. 2020 г. в 13:09, Sven Van Caekenberghe <sven@stfx.eu>:
Actually, it is just a, albeit concise, description of how Semaphores are implemented.
It does not help much in understanding them, in learning how they can/should be used, for what purposes and how code behaves.
Understanding of Process, priorities and Scheduling are also needed for a more complete understanding.
This is not a simple subject.
Read https://en.wikipedia.org/wiki/Semaphore_(programming) and see how well you understand the subject.
In short, it does not answer Stef's concrete question(s).
On 10 Jan 2020, at 06:30, Danil Osipchuk <danil.osipchuk@gmail.com> wrote:
Maybe to add this into the class comment, this is the most concise and clear description of how it works i've ever seen
пÑ, 10 Ñнв. 2020 г., 8:13 Eliot Miranda <eliot.miranda@gmail.com>:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
-- _,,,^..^,,,_ best, Eliot
For example, whether a Semaphore would queue waiting process by order of registration (thru a linked list for example) or by order of priority (thru a Heap for example), would completely change its behavior. So isn't that kind of implementation detail SUPER important, especially when hidden in VM? Also, describing HOW it works is very often used as a mean to explain (and make understand) a higher level feature. IMO, understanding a feature from one implementation is as useful as understanding a feature by examples of usage. Even when implementation is plain Smalltalk, it's already an added value to give main guidelines to help reading code (see this particular class or message for understanding the core...), so when it's in VM... Le ven. 10 janv. 2020 à 12:59, Danil Osipchuk <danil.osipchuk@gmail.com> a écrit :
I didn't claim expertise on the subject (although I use semaphores extensively), nor its simplicity, nor that the implementation description should be the only guide on its usage (hence 'to add..., how it works' wording) Said that, to me it is the case, when a clear description of what is going on aids a lot. Instead of trying to define some rules and scenarios abstractly - to help a user to reason about the system behavior (isn't Stef was willing to look into VM code for this reason?).
To me both scenarios of Stef could be explained that in first case the 'signal' process is not getting preempted by the 'wait' process of the same priority, while in second the preemption happens upon return from primitive (hopefully my memory serves me well and my understanding is correct).
A tangent note on comments in general -- I've noticed more than once, that people tend to produce far clearer descriptions in exchanges like this -- when discussing matter with others. When a person is in documentation/comment writing mode he/she is sort of tenses up in a formal state and often produces something not very helpful. Current class comment of Semaphore is a perfect example, if I were not familiar with the concept from other sources, I would not be able to make any sense of it. So, I would suggest to use opportunities like this to improve comments/docs when a bit of knowledge shows up in a discussion.
regards, Danil
пÑ, 10 Ñнв. 2020 г. в 13:09, Sven Van Caekenberghe <sven@stfx.eu>:
Actually, it is just a, albeit concise, description of how Semaphores are implemented.
It does not help much in understanding them, in learning how they can/should be used, for what purposes and how code behaves.
Understanding of Process, priorities and Scheduling are also needed for a more complete understanding.
This is not a simple subject.
Read https://en.wikipedia.org/wiki/Semaphore_(programming) and see how well you understand the subject.
In short, it does not answer Stef's concrete question(s).
On 10 Jan 2020, at 06:30, Danil Osipchuk <danil.osipchuk@gmail.com> wrote:
Maybe to add this into the class comment, this is the most concise and clear description of how it works i've ever seen
пÑ, 10 Ñнв. 2020 г., 8:13 Eliot Miranda <eliot.miranda@gmail.com>:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
-- _,,,^..^,,,_ best, Eliot
Yes this is why in my chapter on Exception I show the VM code while some people told me that it was not interesting. And this is why in the current chapter on semaphore I have a section on the implementation. Now it does not mean that the we cannot have a higher view too :).
On 10 Jan 2020, at 18:29, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
For example, whether a Semaphore would queue waiting process by order of registration (thru a linked list for example) or by order of priority (thru a Heap for example), would completely change its behavior. So isn't that kind of implementation detail SUPER important, especially when hidden in VM?
Also, describing HOW it works is very often used as a mean to explain (and make understand) a higher level feature. IMO, understanding a feature from one implementation is as useful as understanding a feature by examples of usage. Even when implementation is plain Smalltalk, it's already an added value to give main guidelines to help reading code (see this particular class or message for understanding the core...), so when it's in VM...
Le ven. 10 janv. 2020 à 12:59, Danil Osipchuk <danil.osipchuk@gmail.com <mailto:danil.osipchuk@gmail.com>> a écrit : I didn't claim expertise on the subject (although I use semaphores extensively), nor its simplicity, nor that the implementation description should be the only guide on its usage (hence 'to add..., how it works' wording) Said that, to me it is the case, when a clear description of what is going on aids a lot. Instead of trying to define some rules and scenarios abstractly - to help a user to reason about the system behavior (isn't Stef was willing to look into VM code for this reason?).
To me both scenarios of Stef could be explained that in first case the 'signal' process is not getting preempted by the 'wait' process of the same priority, while in second the preemption happens upon return from primitive (hopefully my memory serves me well and my understanding is correct).
A tangent note on comments in general -- I've noticed more than once, that people tend to produce far clearer descriptions in exchanges like this -- when discussing matter with others. When a person is in documentation/comment writing mode he/she is sort of tenses up in a formal state and often produces something not very helpful. Current class comment of Semaphore is a perfect example, if I were not familiar with the concept from other sources, I would not be able to make any sense of it. So, I would suggest to use opportunities like this to improve comments/docs when a bit of knowledge shows up in a discussion.
regards, Danil
пÑ, 10 Ñнв. 2020 г. в 13:09, Sven Van Caekenberghe <sven@stfx.eu <mailto:sven@stfx.eu>>: Actually, it is just a, albeit concise, description of how Semaphores are implemented.
It does not help much in understanding them, in learning how they can/should be used, for what purposes and how code behaves.
Understanding of Process, priorities and Scheduling are also needed for a more complete understanding.
This is not a simple subject.
Read https://en.wikipedia.org/wiki/Semaphore_(programming) <https://en.wikipedia.org/wiki/Semaphore_(programming)> and see how well you understand the subject.
In short, it does not answer Stef's concrete question(s).
On 10 Jan 2020, at 06:30, Danil Osipchuk <danil.osipchuk@gmail.com <mailto:danil.osipchuk@gmail.com>> wrote:
Maybe to add this into the class comment, this is the most concise and clear description of how it works i've ever seen
пÑ, 10 Ñнв. 2020 г., 8:13 Eliot Miranda <eliot.miranda@gmail.com <mailto:eliot.miranda@gmail.com>>:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com <mailto:stepharo@netcourrier.com>> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
-- _,,,^..^,,,_ best, Eliot
Hi Steph, On Jan 10, 2020, at 12:42 PM, ducasse <stepharo@netcourrier.com> wrote: Yes this is why in my chapter on Exception I show the VM code while some people told me that it was not interesting. And this is why in the current chapter on semaphore I have a section on the implementation. Now it does not mean that the we cannot have a higher view too :). Indeed. Note that now we have two improvements supported by the VM over the blue book scheduler & synchronization primitives. First we have a native critical section which is not a queuing semaphore, but a queueing lock, which is in one of three states. It is a replacement for the old Mutex class. It is not in Pharo yet but you can easily adapt the Squeak implementation (see below). Let me give a similar definition. A native critical section is a queue that can have an owner. A new native critical section is empty and unowned. A process attempts to enter the native critical section via primitiveEnterCriticalSection. This occurs in one of three ways. - If the native critical section is unowned (which implies it is empty) then the process becomes the owner of the native critical section, the primitive answers false (meaning that it was previously unowned or owned by some other process), and the process proceeds. - if the native critical section is already owned by the process then the process remains the owner, the primitive answers true (meaning that it is already owned) and the process proceeds. - if the native critical section is already owned by some other process then the process is suspended and added to the end of the native critical sectionâs queue, where it will wait until a primitiveExitCriticalSection resumes it and makes it owner. A process leaves a native critical section via primitiveExitCriticalSection. It is the process's responsibility to use primitiveExitCriticalSection only when it entered via a primitiveEnterCriticalSection that answered false. If the native critical section is empty then on executing primitiveExitCriticalSection it becomes unowned. If the native critical section is not empty then on executing primitiveExitCriticalSection it becomes owned by the first process in its queue, the process is removed from the queue and is scheduled, proceeding from the primitiveEnterCriticalSection which caused it to block with primitiveEnterCriticalSection answering false. In addition a process may test and set its ownership of a native critical section without danger of blocking. A process tests and sets its ownership of a native critical section via primitiveTestAndSetOwnershipOfCriticalSection. On executing primitiveTestAndSetOwnershipOfCriticalSection, if the native critical section is unowned then the process becomes its owner and primitiveTestAndSetOwnershipOfCriticalSection answers false. If the native critical section is owned by the process primitiveTestAndSetOwnershipOfCriticalSection answers true. If the native critical section is owned by some other process then primitiveTestAndSetOwnershipOfCriticalSection answers nil. Using primitiveExitCriticalSection and primitiveExitCriticalSection allows for efficient implemntation of rentrant critical sections: critical: aBlock "Evaluate aBlock protected by the receiver." <criticalSection> ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]] Adding primitiveTestAndSetOwnershipOfCriticalSection makes it easy to implement and understand the following: critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it is already in a critical section on behalf of some other process answer the evaluation of lockedBlock." <criticalSection> ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil: [:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]] Once a process owns a critical section it can enter the criutical section as many times as it wants. With the Semaphore it can only enter once per signal. Of course we have constructed the class Mutex to operate similarly to a native critical section, but it is inefficient and not entirely safe (because we rely on implementation-defined behaviour to be able to assign the Mutex's owner without being preempted. In Squeak we already replaced the old Mutex with a Mutex built using the native crittical section reoresentation and primitives. The file-in is attached. An implementation note is that semaphores and native crtitical sections look very similar; they are both queues so their first and second and instance variables are firstLink & lastLink, inherited from LinkedList. A Semaphore's third inst var is excessSignals, its excess signals count. A Mutex's third inst var is is owner. HTH P.S. This is an interesting exercise. What we have done in specifying behavior here is focus on processes. The documentation on the primitive methods in the system focus on the semaphore or native critical section. What (I think) programmers want is to understand how the process behaves, not understand how the semaphore or native critical section works. So documenting things from a process perspective is more useful. P.P.S. If you compare the performance of the constructed Mutex against the native Mitex please report the results. P.P.P.S. We had tio step carefully to replace the old Mutex with the new one. I can't remember her the details, but we handled it with Monticello load scripts and we can find the details if you need them On 10 Jan 2020, at 18:29, Nicolas Cellier < nicolas.cellier.aka.nice@gmail.com> wrote: For example, whether a Semaphore would queue waiting process by order of registration (thru a linked list for example) or by order of priority (thru a Heap for example), would completely change its behavior. So isn't that kind of implementation detail SUPER important, especially when hidden in VM? Also, describing HOW it works is very often used as a mean to explain (and make understand) a higher level feature. IMO, understanding a feature from one implementation is as useful as understanding a feature by examples of usage. Even when implementation is plain Smalltalk, it's already an added value to give main guidelines to help reading code (see this particular class or message for understanding the core...), so when it's in VM... Le ven. 10 janv. 2020 à 12:59, Danil Osipchuk <danil.osipchuk@gmail.com> a écrit :
I didn't claim expertise on the subject (although I use semaphores extensively), nor its simplicity, nor that the implementation description should be the only guide on its usage (hence 'to add..., how it works' wording) Said that, to me it is the case, when a clear description of what is going on aids a lot. Instead of trying to define some rules and scenarios abstractly - to help a user to reason about the system behavior (isn't Stef was willing to look into VM code for this reason?).
To me both scenarios of Stef could be explained that in first case the 'signal' process is not getting preempted by the 'wait' process of the same priority, while in second the preemption happens upon return from primitive (hopefully my memory serves me well and my understanding is correct).
A tangent note on comments in general -- I've noticed more than once, that people tend to produce far clearer descriptions in exchanges like this -- when discussing matter with others. When a person is in documentation/comment writing mode he/she is sort of tenses up in a formal state and often produces something not very helpful. Current class comment of Semaphore is a perfect example, if I were not familiar with the concept from other sources, I would not be able to make any sense of it. So, I would suggest to use opportunities like this to improve comments/docs when a bit of knowledge shows up in a discussion.
regards, Danil
пÑ, 10 Ñнв. 2020 г. в 13:09, Sven Van Caekenberghe <sven@stfx.eu>:
Actually, it is just a, albeit concise, description of how Semaphores are implemented.
It does not help much in understanding them, in learning how they can/should be used, for what purposes and how code behaves.
Understanding of Process, priorities and Scheduling are also needed for a more complete understanding.
This is not a simple subject.
Read https://en.wikipedia.org/wiki/Semaphore_(programming) and see how well you understand the subject.
In short, it does not answer Stef's concrete question(s).
On 10 Jan 2020, at 06:30, Danil Osipchuk <danil.osipchuk@gmail.com> wrote:
Maybe to add this into the class comment, this is the most concise and clear description of how it works i've ever seen
пÑ, 10 Ñнв. 2020 г., 8:13 Eliot Miranda <eliot.miranda@gmail.com>:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
On Fri, Jan 10, 2020 at 2:01 PM Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Steph,
On Jan 10, 2020, at 12:42 PM, ducasse <stepharo@netcourrier.com> wrote:
Yes this is why in my chapter on Exception I show the VM code while some people told me that it was not interesting. And this is why in the current chapter on semaphore I have a section on the implementation. Now it does not mean that the we cannot have a higher view too :).
Indeed. Note that now we have two improvements supported by the VM over the blue book scheduler & synchronization primitives.
Oops! I forgot to mention the other improvement. That is the ability of the scheduler to add a process to the front of a particular run queue when a process is preempted, not to the back of its run queue as is specified (erroneously) in the original Smalltalk-80 specification. Why is this erroneous? Smalltalk has a real-time preemptive-across-priorities, cooperative-within-priorities scheduling model. No process at the same priority as the active process can preempt the active process. Instead it must wait until the active process yields (which moves a process to the back of its run queue, allowing all other runnable processes at its priority a chance to run until it will run again), is suspended (on a semaphore or mutex), or explicitly suspends (via the suspend primitive). So when the original scheduler puts a process at the end of its run queue when a higher priority process preempts it that introduces an implicit yield, which violates the contract, a contract that can be used to implement cheapjack-free mutual exclusion between processes of the same priority. So the improvement, selected by a vm flag, is to cause preemption to add a process to the front of its run queue, maintaining the order and preserving the contract. _,,,^..^,,,_ best, Eliot
Hi Nicolas, Sure the implementation description is super important, and did not say that. All I said is that it is an implementation description and not a high level one. I just checked the descriptions starting page 257 of the Smalltalk-80 The Language book, as well as VisualWorks's 1.0 Cookbook page 1029 - these are of a *totally* different, much higher level. That is what we need the most. Sven
On 10 Jan 2020, at 18:29, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
For example, whether a Semaphore would queue waiting process by order of registration (thru a linked list for example) or by order of priority (thru a Heap for example), would completely change its behavior. So isn't that kind of implementation detail SUPER important, especially when hidden in VM?
Also, describing HOW it works is very often used as a mean to explain (and make understand) a higher level feature. IMO, understanding a feature from one implementation is as useful as understanding a feature by examples of usage. Even when implementation is plain Smalltalk, it's already an added value to give main guidelines to help reading code (see this particular class or message for understanding the core...), so when it's in VM...
Le ven. 10 janv. 2020 à 12:59, Danil Osipchuk <danil.osipchuk@gmail.com> a écrit : I didn't claim expertise on the subject (although I use semaphores extensively), nor its simplicity, nor that the implementation description should be the only guide on its usage (hence 'to add..., how it works' wording) Said that, to me it is the case, when a clear description of what is going on aids a lot. Instead of trying to define some rules and scenarios abstractly - to help a user to reason about the system behavior (isn't Stef was willing to look into VM code for this reason?).
To me both scenarios of Stef could be explained that in first case the 'signal' process is not getting preempted by the 'wait' process of the same priority, while in second the preemption happens upon return from primitive (hopefully my memory serves me well and my understanding is correct).
A tangent note on comments in general -- I've noticed more than once, that people tend to produce far clearer descriptions in exchanges like this -- when discussing matter with others. When a person is in documentation/comment writing mode he/she is sort of tenses up in a formal state and often produces something not very helpful. Current class comment of Semaphore is a perfect example, if I were not familiar with the concept from other sources, I would not be able to make any sense of it. So, I would suggest to use opportunities like this to improve comments/docs when a bit of knowledge shows up in a discussion.
regards, Danil
пÑ, 10 Ñнв. 2020 г. в 13:09, Sven Van Caekenberghe <sven@stfx.eu>: Actually, it is just a, albeit concise, description of how Semaphores are implemented.
It does not help much in understanding them, in learning how they can/should be used, for what purposes and how code behaves.
Understanding of Process, priorities and Scheduling are also needed for a more complete understanding.
This is not a simple subject.
Read https://en.wikipedia.org/wiki/Semaphore_(programming) and see how well you understand the subject.
In short, it does not answer Stef's concrete question(s).
On 10 Jan 2020, at 06:30, Danil Osipchuk <danil.osipchuk@gmail.com> wrote:
Maybe to add this into the class comment, this is the most concise and clear description of how it works i've ever seen
пÑ, 10 Ñнв. 2020 г., 8:13 Eliot Miranda <eliot.miranda@gmail.com>:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
-- _,,,^..^,,,_ best, Eliot
Done. https://github.com/pharo-project/pharo/issues/5466 It was so simple that I preferred to do it myself so like that I will look like a great Pharo contributor.
On 10 Jan 2020, at 06:30, Danil Osipchuk <danil.osipchuk@gmail.com> wrote:
Maybe to add this into the class comment, this is the most concise and clear description of how it works i've ever seen
пÑ, 10 Ñнв. 2020 г., 8:13 Eliot Miranda <eliot.miranda@gmail.com <mailto:eliot.miranda@gmail.com>>:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com <mailto:stepharo@netcourrier.com>> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
S
S.
-- _,,,^..^,,,_ best, Eliot
ducasse wrote
It was so simple that I preferred to do it myself so like that I will look like a great Pharo contributor.
:) ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
ducasse wrote
It was so simple that I preferred to do it myself so like that I will look like a great Pharo contributor.
:) :)
Sometimes I do not know if my humour is strange or not :) But strange humour is better than no humour :) Stef PS: I read too many gary larson comics to be safe :)
Am 10.01.2020 um 22:32 schrieb ducasse <stepharo@netcourrier.com>:

ducasse wrote
It was so simple that I preferred to do it myself so like that I will look like a great Pharo contributor.
:) :)
Sometimes I do not know if my humour is strange or not :)
It is! And you are the only one having that kind of humor ;) Norbert
But strange humour is better than no humour :)
Stef
PS: I read too many gary larson comics to be safe :)
:)
:)
Sometimes I do not know if my humour is strange or not :)
It is! And you are the only one having that kind of humor ;)
Not sure that we are safe :)))) https://www.thefarside.com Just arrived last month and it is super great
For greater visibility and comprehension, it might be useful to early in the manual define a utility method... Object>>crTracePriority self crTrace: '[', Processor activePriority printString, ']', self printString On Fri, 10 Jan 2020 at 13:13, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote:
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
This is a good point. It may be useful for the example to be expanded to... | semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'Process 1' crTracePriority ] fork. p2 := [semaphore signal. 'Process 2' crTracePriority ] fork. 'Original process pre-yield' crTracePriority . 1 milliSeconds wait. 'Original process post-yield' crTracePriority . which would produce==> [40]'Original process pre-yield' [40]'Process 2' [40]'Process 1' [40]'Original process post-yield' with other examples producing... [40]'Original process pre-yield' [30]'Process 1' [20]'Process 2' [40]'Original process post-yield' [60]'Process 2' [50]'Process 1' [40]'Original process pre-yield' [40]'Original process post-yield'
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
For quick reference, here is some relevant VM code (the StackInterpreter code is a little simpler...) https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... StackInterpreter class >> initializePrimitiveTable [ ... "Control Primitives (80-89)" (85 primitiveSignal) (86 primitiveWait) ... ] https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... InterpreterPrimitives >> primitiveWait [ | sema excessSignals activeProc | sema := self stackTop. "rcvr" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: [activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority] ] https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... InterpreterPrimitives >> primitiveSignal [ "Synchronously signal the semaphore. This may change the active process as a result." self synchronousSignal: self stackTop ] https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... StackInterpreter >> synchronousSignal: aSemaphore [ "Signal the given semaphore from within the interpreter. Answer if the current process was preempted." | excessSignals | <inline: false> (self isEmptyList: aSemaphore) ifTrue: ["no process is waiting on this semaphore" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: aSemaphore. self storeInteger: ExcessSignalsIndex ofObject: aSemaphore withValue: excessSignals + 1. ^false]. objectMemory ensureSemaphoreUnforwardedThroughContext: aSemaphore. ^self resume: (self removeFirstLinkOfList: aSemaphore) preemptedYieldingIf: preemptionYields ] cheers -ben
Thanks ben, this is a nice idea to illustrate what is happening I will use it. Thanks for the VM code :). I like it too. I will add a note also mentioning that indeed there is a process to execute the snippets that the reader will write and execute :). Stef PS: what makes me laugh a bit is that people talk about documentation when there are nearly none on the topics on which I write. I write to be able to fully forget everything and free my brain. I remember writing on bloc, exceptions, when there was only obscure texts or none as if mail discussions would make a book. At least this is not the level I want. To me I would like to have a book like Smalltalk and its implementation but about Pharo. This was a book! So I will write it piece by piece.
For greater visibility and comprehension, it might be useful to early in the manual define a utility method... Object>>crTracePriority self crTrace: '[', Processor activePriority printString, ']', self printString
On Fri, 10 Jan 2020 at 13:13, Eliot Miranda <eliot.miranda@gmail.com <mailto:eliot.miranda@gmail.com>> wrote:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com <mailto:stepharo@netcourrier.com>> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
This is a good point. It may be useful for the example to be expanded to...
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'Process 1' crTracePriority ] fork.
p2 := [semaphore signal. 'Process 2' crTracePriority ] fork.
'Original process pre-yield' crTracePriority . 1 milliSeconds wait. 'Original process post-yield' crTracePriority .
which would produce==>
[40]'Original process pre-yield' [40]'Process 2' [40]'Process 1' [40]'Original process post-yield'
with other examples producing...
[40]'Original process pre-yield' [30]'Process 1' [20]'Process 2' [40]'Original process post-yield'
[60]'Process 2' [50]'Process 1' [40]'Original process pre-yield' [40]'Original process post-yield'
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
For quick reference, here is some relevant VM code (the StackInterpreter code is a little simpler...)
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt...> StackInterpreter class >> initializePrimitiveTable [ ... "Control Primitives (80-89)" (85 primitiveSignal) (86 primitiveWait) ... ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt...> InterpreterPrimitives >> primitiveWait [ | sema excessSignals activeProc | sema := self stackTop. "rcvr" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: [activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority] ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt...> InterpreterPrimitives >> primitiveSignal [ "Synchronously signal the semaphore. This may change the active process as a result." self synchronousSignal: self stackTop ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt...> StackInterpreter >> synchronousSignal: aSemaphore [ "Signal the given semaphore from within the interpreter. Answer if the current process was preempted." | excessSignals | <inline: false> (self isEmptyList: aSemaphore) ifTrue: ["no process is waiting on this semaphore" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: aSemaphore. self storeInteger: ExcessSignalsIndex ofObject: aSemaphore withValue: excessSignals + 1. ^false].
objectMemory ensureSemaphoreUnforwardedThroughContext: aSemaphore.
^self resume: (self removeFirstLinkOfList: aSemaphore) preemptedYieldingIf: preemptionYields ]
cheers -ben
Hi Ben, Great approach, though I would make one change to make your example completely copy/paste runnable. Stef's original example: | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ]. p1 := [ semaphore wait. trace value: 'Process 1' ] fork. p2 := [ semaphore signal. trace value: 'Process 2' ] fork. trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'. Gives: '[40] Original process pre-yield' '[40] Process 2' '[40] Original process post-yield' '[40] Process 1' But not running the yield section gives: '[40] Process 2' '[40] Process 1'
From this it would seem that the code in p2 continues after signal and only later does p1 get past its wait.
Playing with the priorities we can change that order (apparently); | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ]. p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 30. p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 20. Gives: '[30] Process 1' '[20] Process 2' Again, the yield section makes no difference. So something else happened. The other way around: | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ]. p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 50. p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 60. Gives: '[60] Process 2' '[50] Process 1' Obviously the details about scheduling, order and priorities are really important to understanding this behaviour, and we should be able to explain this is simple terms, so that normal people can use this correctly. Sven
On 10 Jan 2020, at 19:02, Ben Coman <btc@openinworld.com> wrote:
For greater visibility and comprehension, it might be useful to early in the manual define a utility method... Object>>crTracePriority self crTrace: '[', Processor activePriority printString, ']', self printString
On Fri, 10 Jan 2020 at 13:13, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
This is a good point. It may be useful for the example to be expanded to...
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'Process 1' crTracePriority ] fork.
p2 := [semaphore signal. 'Process 2' crTracePriority ] fork.
'Original process pre-yield' crTracePriority . 1 milliSeconds wait. 'Original process post-yield' crTracePriority .
which would produce==>
[40]'Original process pre-yield' [40]'Process 2' [40]'Process 1' [40]'Original process post-yield'
with other examples producing...
[40]'Original process pre-yield' [30]'Process 1' [20]'Process 2' [40]'Original process post-yield'
[60]'Process 2' [50]'Process 1' [40]'Original process pre-yield' [40]'Original process post-yield'
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
For quick reference, here is some relevant VM code (the StackInterpreter code is a little simpler...)
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... StackInterpreter class >> initializePrimitiveTable [ ... "Control Primitives (80-89)" (85 primitiveSignal) (86 primitiveWait) ... ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... InterpreterPrimitives >> primitiveWait [ | sema excessSignals activeProc | sema := self stackTop. "rcvr" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: [activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority] ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... InterpreterPrimitives >> primitiveSignal [ "Synchronously signal the semaphore. This may change the active process as a result." self synchronousSignal: self stackTop ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... StackInterpreter >> synchronousSignal: aSemaphore [ "Signal the given semaphore from within the interpreter. Answer if the current process was preempted." | excessSignals | <inline: false> (self isEmptyList: aSemaphore) ifTrue: ["no process is waiting on this semaphore" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: aSemaphore. self storeInteger: ExcessSignalsIndex ofObject: aSemaphore withValue: excessSignals + 1. ^false].
objectMemory ensureSemaphoreUnforwardedThroughContext: aSemaphore.
^self resume: (self removeFirstLinkOfList: aSemaphore) preemptedYieldingIf: preemptionYields ]
cheers -ben
Thanks Sven I started to use the suggestions of Ben. And Iâm thinking that I could show my version then step back and explain that indeed there is another process (the UI one). I will digest your example and produce a new version of the booklet and people can give feedback. S.
On 10 Jan 2020, at 23:30, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi Ben,
Great approach, though I would make one change to make your example completely copy/paste runnable.
Stef's original example:
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] fork.
p2 := [ semaphore signal. trace value: 'Process 2' ] fork.
trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
Gives:
'[40] Original process pre-yield' '[40] Process 2' '[40] Original process post-yield' '[40] Process 1'
But not running the yield section gives:
'[40] Process 2' '[40] Process 1'
From this it would seem that the code in p2 continues after signal and only later does p1 get past its wait.
Playing with the priorities we can change that order (apparently);
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 30.
p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 20.
Gives:
'[30] Process 1' '[20] Process 2'
Again, the yield section makes no difference. So something else happened.
The other way around:
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 50.
p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 60.
Gives:
'[60] Process 2' '[50] Process 1'
Obviously the details about scheduling, order and priorities are really important to understanding this behaviour, and we should be able to explain this is simple terms, so that normal people can use this correctly.
Sven
On 10 Jan 2020, at 19:02, Ben Coman <btc@openinworld.com> wrote:
For greater visibility and comprehension, it might be useful to early in the manual define a utility method... Object>>crTracePriority self crTrace: '[', Processor activePriority printString, ']', self printString
On Fri, 10 Jan 2020 at 13:13, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Thu, Jan 9, 2020 at 5:03 AM ducasse <stepharo@netcourrier.com> wrote: Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
- ==p1== is scheduled and its execution starts to wait on the semaphore, so it is removed from the run queue of the scheduler and added to the waiting list of the semaphore. - ==p2== is scheduled and it signals the semaphore. The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.
Since Smalltalk does not have a preemptive scheduler, neither p1 nor p2 will start to run until something else happens after the execution of p1 := [...] fork. p2 := [...] fork. So for example, if there is Processor yield then p1 can start to run.
So you need to add code to your example to be able to determine what will happen. The easiest thing would be to delay long enough that both can run. 1 millisecond is more than enough.
This is a good point. It may be useful for the example to be expanded to...
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'Process 1' crTracePriority ] fork.
p2 := [semaphore signal. 'Process 2' crTracePriority ] fork.
'Original process pre-yield' crTracePriority . 1 milliSeconds wait. 'Original process post-yield' crTracePriority .
which would produce==>
[40]'Original process pre-yield' [40]'Process 2' [40]'Process 1' [40]'Original process post-yield'
with other examples producing...
[40]'Original process pre-yield' [30]'Process 1' [20]'Process 2' [40]'Original process post-yield'
[60]'Process 2' [50]'Process 1' [40]'Original process pre-yield' [40]'Original process post-yield'
Now this sentence "The semaphore takes the first waiting process (==p1==) and reschedule it by adding it to the end of the suspended lists.â is super naive. Is the semaphore signalling scheduled? or not?
I would say these three things, something like this:
"A semaphore is a queue (implemented as a linked list) and an excess signals count, which is a non-negative integer. On instance creation a new semaphore is empty and has a zero excess signals count. A semaphore created for mutual exclusion is empty and has an excess signals count of one."
"When a process waits on a semaphore, if the semaphore's excess signals count is non-zero, then the excess signal count is decremented, and the process proceeds. But if the semaphore has a zero excess signals count then the process is unscheduled and added to the end of the semaphore, after any other processes that are queued on the semaphore."
"When a semaphore is signaled, if it is not empty, the first process is removed from it and added to the runnable processes in the scheduler. If the semaphore is empty its excess signals count is incremented.
Given these three statements it is easy to see how they work, how to use them for mutual exclusion, etc.
signal "Primitive. Send a signal through the receiver. If one or more processes have been suspended trying to receive a signal, allow the first one to proceed. If no process is waiting, remember the excess signal. Essential. See Object documentation whatIsAPrimitive."
<primitive: 85> self primitiveFailed
"self isEmpty ifTrue: [excessSignals := excessSignals+1] ifFalse: [Processor resume: self removeFirstLink]"
I wanted to know what is really happening when a semaphore is signalled. Now resume: does not exist on Processor.
I will look in the VM code.
For quick reference, here is some relevant VM code (the StackInterpreter code is a little simpler...)
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... StackInterpreter class >> initializePrimitiveTable [ ... "Control Primitives (80-89)" (85 primitiveSignal) (86 primitiveWait) ... ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... InterpreterPrimitives >> primitiveWait [ | sema excessSignals activeProc | sema := self stackTop. "rcvr" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: [activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority] ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... InterpreterPrimitives >> primitiveSignal [ "Synchronously signal the semaphore. This may change the active process as a result." self synchronousSignal: self stackTop ]
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/pharo/headless/smallt... StackInterpreter >> synchronousSignal: aSemaphore [ "Signal the given semaphore from within the interpreter. Answer if the current process was preempted." | excessSignals | <inline: false> (self isEmptyList: aSemaphore) ifTrue: ["no process is waiting on this semaphore" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: aSemaphore. self storeInteger: ExcessSignalsIndex ofObject: aSemaphore withValue: excessSignals + 1. ^false].
objectMemory ensureSemaphoreUnforwardedThroughContext: aSemaphore.
^self resume: (self removeFirstLinkOfList: aSemaphore) preemptedYieldingIf: preemptionYields ]
cheers -ben
On Sat, 11 Jan 2020 at 06:31, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi Ben,
Great approach, though I would make one change to make your example completely copy/paste runnable.
Stef's original example:
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] fork.
p2 := [ semaphore signal. trace value: 'Process 2' ] fork.
trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
Gives:
'[40] Original process pre-yield' '[40] Process 2' '[40] Original process post-yield' '[40] Process 1'
But not running the yield section gives:
'[40] Process 2' '[40] Process 1'
which is an identical result if the 'Original process' traces are filtered out.
From this it would seem that the code in p2 continues after signal and only later does p1 get past its wait.
Yes, a #signal does not transfer execution unless the waiting-process that received the signal is a higher priority. Within the same priority, it just makes waiting-process runnable, and the highest-priority-runnable-process is the one that is run. Playing with the priorities we can change that order (apparently);
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 30.
p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 20.
Gives:
'[30] Process 1' '[20] Process 2'
Again, the yield section makes no difference. So something else happened.
The yield made no difference because it only facilitates other processes at-the-SAME-priority getting a chance to run. Yield doesn't put the current-process to sleep, it just moves the process to the back of its-priority-runQueue. It gets to run again before any lower priority process gets a chance to run. Yielding will never allow a lower-priority-process to run. For a lower-priority process to run, the current-process needs to sleep rather than yield. Compare... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'. ==> '@40 Original process pre-yield' '@40 Original process post-yield' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues' with... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-delay'. 1 milliSecond wait. trace value: 'Original process post-delay'. ==> '@40 Original process pre-delay' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues' '@40 Original process post-delay' Stef, on further consideration I think your first examples should not-have p1 and p2 the same priority. Scheduling of same-priority processes and how they interact with the UI thread is an extra level of complexity that may be better done shortly after. Not needing to trace "Original process" in the first example gives less for the reader to digest So your first example might compare... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 20. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 30. ==> '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues' with the priority order swapped... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. ==> '@30 Process 2a signals semaphore' '@30 Process 2b continues' '@20 Process 1a waits for signal on semaphore' '@20 Process 1b received signal' cheers -ben
On 11 Jan 2020, at 10:50, Ben Coman <btc@openinworld.com> wrote:
On Sat, 11 Jan 2020 at 06:31, Sven Van Caekenberghe <sven@stfx.eu <mailto:sven@stfx.eu>> wrote: Hi Ben,
Great approach, though I would make one change to make your example completely copy/paste runnable.
Stef's original example:
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] fork.
p2 := [ semaphore signal. trace value: 'Process 2' ] fork.
trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
Gives:
'[40] Original process pre-yield' '[40] Process 2' '[40] Original process post-yield' '[40] Process 1'
But not running the yield section gives:
'[40] Process 2' '[40] Process 1'
which is an identical result if the 'Original process' traces are filtered out.
From this it would seem that the code in p2 continues after signal and only later does p1 get past its wait.
Yes, a #signal does not transfer execution unless the waiting-process that received the signal is a higher priority. Within the same priority, it just makes waiting-process runnable, and the highest-priority-runnable-process is the one that is run.
Playing with the priorities we can change that order (apparently);
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 30.
p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 20.
Gives:
'[30] Process 1' '[20] Process 2'
Again, the yield section makes no difference. So something else happened.
The yield made no difference because it only facilitates other processes at-the-SAME-priority getting a chance to run. Yield doesn't put the current-process to sleep, it just moves the process to the back of its-priority-runQueue. It gets to run again before any lower priority process gets a chance to run.
Yielding will never allow a lower-priority-process to run. For a lower-priority process to run, the current-process needs to sleep rather than yield.
Indeed. I will add this note to my chapter.
Compare... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
==> '@40 Original process pre-yield' '@40 Original process post-yield' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues'
with... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-delay'. 1 milliSecond wait. trace value: 'Original process post-delay'.
==> '@40 Original process pre-delay' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues' '@40 Original process post-delay'
Stef, on further consideration I think your first examples should not-have p1 and p2 the same priority. Scheduling of same-priority processes and how they interact with the UI thread is an extra level of complexity that may be better done shortly after.
Yes I realize it.
Not needing to trace "Original process" in the first example gives less for the reader to digest
Yes I thought the same. I thought that I could have the following strategy. Give a first simple version, them revisiting it after. With the idea that even if the reader meta model is a bit slanted after the first example but they get the result right then after the full explanation they should get it right instead of overhelming them with the full details at first. So I need to concentrate to have the full outline clear. Any way thanks for the discussion. Pedagogy is sometimes following not straight paths.
So your first example might compare... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 20. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 30.
==> '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues'
with the priority order swapped... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20.
==> '@30 Process 2a signals semaphore' '@30 Process 2b continues' '@20 Process 1a waits for signal on semaphore' '@20 Process 1b received signal'
cheers -ben
On Sat, 11 Jan 2020 at 18:01, ducasse <stepharo@netcourrier.com> wrote:
On 11 Jan 2020, at 10:50, Ben Coman <btc@openinworld.com> wrote:
On Sat, 11 Jan 2020 at 06:31, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi Ben,
Great approach, though I would make one change to make your example completely copy/paste runnable.
Stef's original example:
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] fork.
p2 := [ semaphore signal. trace value: 'Process 2' ] fork.
trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
Gives:
'[40] Original process pre-yield' '[40] Process 2' '[40] Original process post-yield' '[40] Process 1'
But not running the yield section gives:
'[40] Process 2' '[40] Process 1'
which is an identical result if the 'Original process' traces are filtered out.
From this it would seem that the code in p2 continues after signal and only later does p1 get past its wait.
Yes, a #signal does not transfer execution unless the waiting-process that received the signal is a higher priority. Within the same priority, it just makes waiting-process runnable, and the highest-priority-runnable-process is the one that is run.
Playing with the priorities we can change that order (apparently);
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 30.
p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 20.
Gives:
'[30] Process 1' '[20] Process 2'
Again, the yield section makes no difference. So something else happened.
The yield made no difference because it only facilitates other processes at-the-SAME-priority getting a chance to run. Yield doesn't put the current-process to sleep, it just moves the process to the back of its-priority-runQueue. It gets to run again before any lower priority process gets a chance to run.
Yielding will never allow a lower-priority-process to run. For a lower-priority process to run, the current-process needs to sleep rather than yield.
Indeed. I will add this note to my chapter.
Compare... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
==> '@40 Original process pre-yield' '@40 Original process post-yield' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues'
with... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-delay'. 1 milliSecond wait. trace value: 'Original process post-delay'.
==> '@40 Original process pre-delay' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues' '@40 Original process post-delay'
Stef, on further consideration I think your first examples should not-have p1 and p2 the same priority. Scheduling of same-priority processes and how they interact with the UI thread is an extra level of complexity that may be better done shortly after.
Yes I realize it.
Not needing to trace "Original process" in the first example gives less for the reader to digest
Yes I thought the same. I thought that I could have the following strategy. Give a first simple version, them revisiting it after.
At first glance I thought using #fork would a simpler example than using #forkAt:, but the former interacts with the implicit priority of existing UI process, while the latter is explicit and so actually makes a simpler example. cheers -ben
Yes this was really fun for me to discover that in newProcess newProcess "Answer a Process running the code in the receiver. The process is not scheduled. IMPORTANT! Debug stepping this deep infrastructure may lock your Image If you are not sure what you are doing, close the debugger now." <primitive: 19> "Simulation guard" ^Process forContext: [self value. Processor terminateActive] asContext priority: Processor activePriority
On 11 Jan 2020, at 11:06, Ben Coman <btc@openinworld.com> wrote:
On Sat, 11 Jan 2020 at 18:01, ducasse <stepharo@netcourrier.com <mailto:stepharo@netcourrier.com>> wrote:
On 11 Jan 2020, at 10:50, Ben Coman <btc@openinworld.com <mailto:btc@openinworld.com>> wrote:
On Sat, 11 Jan 2020 at 06:31, Sven Van Caekenberghe <sven@stfx.eu <mailto:sven@stfx.eu>> wrote: Hi Ben,
Great approach, though I would make one change to make your example completely copy/paste runnable.
Stef's original example:
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] fork.
p2 := [ semaphore signal. trace value: 'Process 2' ] fork.
trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
Gives:
'[40] Original process pre-yield' '[40] Process 2' '[40] Original process post-yield' '[40] Process 1'
But not running the yield section gives:
'[40] Process 2' '[40] Process 1'
which is an identical result if the 'Original process' traces are filtered out.
From this it would seem that the code in p2 continues after signal and only later does p1 get past its wait.
Yes, a #signal does not transfer execution unless the waiting-process that received the signal is a higher priority. Within the same priority, it just makes waiting-process runnable, and the highest-priority-runnable-process is the one that is run.
Playing with the priorities we can change that order (apparently);
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 30.
p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 20.
Gives:
'[30] Process 1' '[20] Process 2'
Again, the yield section makes no difference. So something else happened.
The yield made no difference because it only facilitates other processes at-the-SAME-priority getting a chance to run. Yield doesn't put the current-process to sleep, it just moves the process to the back of its-priority-runQueue. It gets to run again before any lower priority process gets a chance to run.
Yielding will never allow a lower-priority-process to run. For a lower-priority process to run, the current-process needs to sleep rather than yield.
Indeed. I will add this note to my chapter.
Compare... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
==> '@40 Original process pre-yield' '@40 Original process post-yield' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues'
with... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-delay'. 1 milliSecond wait. trace value: 'Original process post-delay'.
==> '@40 Original process pre-delay' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues' '@40 Original process post-delay'
Stef, on further consideration I think your first examples should not-have p1 and p2 the same priority. Scheduling of same-priority processes and how they interact with the UI thread is an extra level of complexity that may be better done shortly after.
Yes I realize it.
Not needing to trace "Original process" in the first example gives less for the reader to digest
Yes I thought the same. I thought that I could have the following strategy. Give a first simple version, them revisiting it after.
At first glance I thought using #fork would a simpler example than using #forkAt:, but the former interacts with the implicit priority of existing UI process, while the latter is explicit and so actually makes a simpler example.
cheers -ben
Hi Ben,
On 11 Jan 2020, at 10:50, Ben Coman <btc@openinworld.com> wrote:
On Sat, 11 Jan 2020 at 06:31, Sven Van Caekenberghe <sven@stfx.eu> wrote: Hi Ben,
Great approach, though I would make one change to make your example completely copy/paste runnable.
Stef's original example:
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] fork.
p2 := [ semaphore signal. trace value: 'Process 2' ] fork.
trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
Gives:
'[40] Original process pre-yield' '[40] Process 2' '[40] Original process post-yield' '[40] Process 1'
But not running the yield section gives:
'[40] Process 2' '[40] Process 1'
which is an identical result if the 'Original process' traces are filtered out.
From this it would seem that the code in p2 continues after signal and only later does p1 get past its wait.
Yes, a #signal does not transfer execution unless the waiting-process that received the signal is a higher priority. Within the same priority, it just makes waiting-process runnable, and the highest-priority-runnable-process is the one that is run.
OK, I can understand that, the question remains what happens when the processes have equal priorities.
Playing with the priorities we can change that order (apparently);
| trace semaphore p1 p2 |
semaphore := Semaphore new.
trace := [ :message | ('[{1}] {2}' format: { Processor activeProcess priority. message }) crLog ].
p1 := [ semaphore wait. trace value: 'Process 1' ] forkAt: 30.
p2 := [ semaphore signal. trace value: 'Process 2' ] forkAt: 20.
Gives:
'[30] Process 1' '[20] Process 2'
Again, the yield section makes no difference. So something else happened.
The yield made no difference because it only facilitates other processes at-the-SAME-priority getting a chance to run. Yield doesn't put the current-process to sleep, it just moves the process to the back of its-priority-runQueue. It gets to run again before any lower priority process gets a chance to run.
Yielding will never allow a lower-priority-process to run. For a lower-priority process to run, the current-process needs to sleep rather than yield.
These are clear statements.
Compare... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-yield'. Processor yield. trace value: 'Original process post-yield'.
==> '@40 Original process pre-yield' '@40 Original process post-yield' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues'
with... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20. trace value: 'Original process pre-delay'. 1 milliSecond wait. trace value: 'Original process post-delay'.
==> '@40 Original process pre-delay' '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues' '@40 Original process post-delay'
OK, good example: I think/hope I understand. Now, these further examples only strengthen my believe that it is simply impossible to talk about semaphores without talking about (the complexities) of process scheduling. Semaphores exist as a means to coordinate processes, hence when using them you have to understand what (will) happen, and apparently that is quite complex. In any case, thanks again for the explanations, Sven
Stef, on further consideration I think your first examples should not-have p1 and p2 the same priority. Scheduling of same-priority processes and how they interact with the UI thread is an extra level of complexity that may be better done shortly after. Not needing to trace "Original process" in the first example gives less for the reader to digest
So your first example might compare... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 20. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 30.
==> '@30 Process 1a waits for signal on semaphore' '@20 Process 2a signals semaphore' '@30 Process 1b received signal' '@20 Process 2b continues'
with the priority order swapped... | trace semaphore p1 p2 | semaphore := Semaphore new. trace := [ :message | ('@{1} {2}' format: { Processor activePriority. message }) crLog ]. p1 := [ trace value: 'Process 1a waits for signal on semaphore'. semaphore wait. trace value: 'Process 1b received signal' ] forkAt: 30. p2 := [ trace value: 'Process 2a signals semaphore'. semaphore signal. trace value: 'Process 2b continues' ] forkAt: 20.
==> '@30 Process 2a signals semaphore' '@30 Process 2b continues' '@20 Process 1a waits for signal on semaphore' '@20 Process 1b received signal'
cheers -ben
On Thu, 9 Jan 2020 at 13:01, ducasse <stepharo@netcourrier.com> wrote:
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
The way this is phrased seems to imply that 'p2' will always be displayed before 'p1', however in Pharo this is not guaranteed (when the processes are at the same priority, as they are this example). As Eliot implied in another reply, Pharo has #processPreemptionYields set to true, which means that any time a higher priority process preempts, the current process will be moved to the back of the queue. So in the case above, after p2 signals the semaphore, if a timer was delivered or keystroke pressed, p2 would be suspended and moved to the back of the queue. When the timer / keystroke / etc. had finished processing p1 would be at the front of the queue and would complete first. Since time and input events are (for practical purposes) unpredictable it means that the execution order of processes at a given priority is also unpredictable. While this isn't likely to happen in the example above, I have seen it regularly with TaskIt and multiple entries being run concurrently. I agree with Eliot that changing #processPreemptionYields to true by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment. Running the following variant, and then typing in to another window, demonstrates the behaviour: | semaphore p1 p2 | semaphore := Semaphore new. [ 100 timesRepeat: [ p1 := [ | z | semaphore wait. z := SmallInteger maxVal. 10000000 timesRepeat: [ z := z + 1 ]. 'p1' crTrace ] fork. p2 := [ | z | 1 second wait. semaphore signal. z := SmallInteger maxVal. 10000000 timesRepeat: [ z := z + 1 ]. 'p2' crTrace ] fork. 1 second wait. ] ] fork. The tail of transcript: 'p2' 'p1' 'p1' 'p1' 'p1' 'p2' 'p2' 'p2' 'p1' 'p1' 'p2' 'p1' 'p2' 'p2' 'p1' 'p1' 'p2' 'p1' Cheers, Alistair
Hi alistair
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
The way this is phrased seems to imply that 'p2' will always be displayed before 'p1', however in Pharo this is not guaranteed (when the processes are at the same priority, as they are this example).
No this is not what I implied. I will reread and rephrase the chapters.
As Eliot implied in another reply, Pharo has #processPreemptionYields set to true, which means that any time a higher priority process preempts, the current process will be moved to the back of the queue.
Yes this is explained in the next chapter. What you should see is that we cannot explain everything in a single chapter. At least I cannot.
So in the case above, after p2 signals the semaphore, if a timer was delivered or keystroke pressed, p2 would be suspended and moved to the back of the queue. When the timer / keystroke / etc. had finished processing p1 would be at the front of the queue and would complete first.
Since time and input events are (for practical purposes) unpredictable it means that the execution order of processes at a given priority is also unpredictable.
While this isn't likely to happen in the example above, I have seen it regularly with TaskIt and multiple entries being run concurrently.
I agree with Eliot that changing #processPreemptionYields to true by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment.
If this would be that easy. :) Now what would be a consequence: we should revisit all the processes of the system and understand if/where they should yield. Because now there is an implicit yield. So in an ideal world the new semantics is probably better. Now are we ready to get new bugs and chase them when an old logic like for example an hidden process in calypso worked under the assumption that it was implicitly yielding after preemption? This is the question that we asked ourselves and we do not really know. So far Pharo worked this way during 12 years with this semantics (I does not mean that we cannot change because of our Motto - but the point is to understand the impact and control). Contrary to what people may think we are not changing without assessing impact. So to us this is not an easy decision (while doing it take one single line of one assignment).
Running the following variant, and then typing in to another window,
I like the idea of the input to show the interaction between processes. I will investigate what I can do with it.
demonstrates the behaviour:
| semaphore p1 p2 | semaphore := Semaphore new. [ 100 timesRepeat: [ p1 := [ | z | semaphore wait. z := SmallInteger maxVal. 10000000 timesRepeat: [ z := z + 1 ]. 'p1' crTrace ] fork.
p2 := [ | z | 1 second wait. semaphore signal. z := SmallInteger maxVal. 10000000 timesRepeat: [ z := z + 1 ]. 'p2' crTrace ] fork. 1 second wait. ] ] fork.
The tail of transcript:
'p2' 'p1' 'p1' 'p1' 'p1' 'p2' 'p2' 'p2' 'p1' 'p1' 'p2' 'p1' 'p2' 'p2' 'p1' 'p1' 'p2' 'p1'
Cheers, Alistair
Hi Stef, On Sun, 12 Jan 2020 at 11:28, ducasse <stepharo@netcourrier.com> wrote:
Hi alistair
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
The way this is phrased seems to imply that 'p2' will always be displayed before 'p1', however in Pharo this is not guaranteed (when the processes are at the same priority, as they are this example).
No this is not what I implied. I will reread and rephrase the chapters.
As Eliot implied in another reply, Pharo has #processPreemptionYields set to true, which means that any time a higher priority process preempts, the current process will be moved to the back of the queue.
Yes this is explained in the next chapter. What you should see is that we cannot explain everything in a single chapter. At least I cannot.
Agreed. Maybe just a footnote indicating that process scheduling will be explained in later chapters.
So in the case above, after p2 signals the semaphore, if a timer was delivered or keystroke pressed, p2 would be suspended and moved to the back of the queue. When the timer / keystroke / etc. had finished processing p1 would be at the front of the queue and would complete first.
Since time and input events are (for practical purposes) unpredictable it means that the execution order of processes at a given priority is also unpredictable.
While this isn't likely to happen in the example above, I have seen it regularly with TaskIt and multiple entries being run concurrently.
I agree with Eliot that changing #processPreemptionYields to true by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment.
As Sven kindly pointed out, I meant to say set #processPreemptionYields to false.
If this would be that easy. :) Now what would be a consequence: we should revisit all the processes of the system and understand if/where they should yield. Because now there is an implicit yield. So in an ideal world the new semantics is probably better. Now are we ready to get new bugs and chase them when an old logic like for example an hidden process in calypso worked under the assumption that it was implicitly yielding after preemption? This is the question that we asked ourselves and we do not really know. So far Pharo worked this way during 12 years with this semantics (I does not mean that we cannot change because of our Motto - but the point is to understand the impact and control). Contrary to what people may think we are not changing without assessing impact. So to us this is not an easy decision (while doing it take one single line of one assignment).
I also wasn't implying that we just go ahead and change it. But if it is a possibility then I'll try changing it in my image and see how it helps with tracking down inter-process interactions that we're currently experiencing, and if it does introduce any showstopper issues. Cheers, Alistair
I also wasn't implying that we just go ahead and change it. But if it is a possibility then I'll try changing it in my image and see how it helps with tracking down inter-process interactions that we're currently experiencing, and if it does introduce any showstopper issues.
Yes this is clearly something that we should evaluate. Now we have more urgent problems to fix :). At least with the concurrent booklet we will set the stage and create material that everybody can read and understand. S.
Hi Alistair,
On 12 Jan 2020, at 09:33, Alistair Grant <akgrant0710@gmail.com> wrote:
On Thu, 9 Jan 2020 at 13:01, ducasse <stepharo@netcourrier.com> wrote:
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
The way this is phrased seems to imply that 'p2' will always be displayed before 'p1', however in Pharo this is not guaranteed (when the processes are at the same priority, as they are this example).
As Eliot implied in another reply, Pharo has #processPreemptionYields set to true, which means that any time a higher priority process preempts, the current process will be moved to the back of the queue.
So in the case above, after p2 signals the semaphore, if a timer was delivered or keystroke pressed, p2 would be suspended and moved to the back of the queue. When the timer / keystroke / etc. had finished processing p1 would be at the front of the queue and would complete first.
Since time and input events are (for practical purposes) unpredictable it means that the execution order of processes at a given priority is also unpredictable.
While this isn't likely to happen in the example above, I have seen it regularly with TaskIt and multiple entries being run concurrently.
I agree with Eliot that changing #processPreemptionYields to true by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment.
I don't understand, in your second paragraph you say 'Pharo has #processPreemptionYields set to true' and now you say it should become the default. Is that already the case or not then ?
Running the following variant, and then typing in to another window, demonstrates the behaviour:
I am not sure what you want to demonstrate: that it is totally random depending on external factors ;-) ? Which is pretty bad: how should semaphores be used (safely) ? What are good examples of real world correct semaphore usage ? Right now, all the explanations around scheduling of processes and their priorities make it seem as if the answer is 'it all depends' and 'there is no way to be 100% sure what will happen'. Sven
| semaphore p1 p2 | semaphore := Semaphore new. [ 100 timesRepeat: [ p1 := [ | z | semaphore wait. z := SmallInteger maxVal. 10000000 timesRepeat: [ z := z + 1 ]. 'p1' crTrace ] fork.
p2 := [ | z | 1 second wait. semaphore signal. z := SmallInteger maxVal. 10000000 timesRepeat: [ z := z + 1 ]. 'p2' crTrace ] fork. 1 second wait. ] ] fork.
The tail of transcript:
'p2' 'p1' 'p1' 'p1' 'p1' 'p2' 'p2' 'p2' 'p1' 'p1' 'p2' 'p1' 'p2' 'p2' 'p1' 'p1' 'p2' 'p1'
Cheers, Alistair
Hi Sven, In line below... Cheers, Alistair (on phone) On Sun., 12 Jan. 2020, 13:00 Sven Van Caekenberghe, <sven@stfx.eu> wrote:
Hi Alistair,
On 12 Jan 2020, at 09:33, Alistair Grant <akgrant0710@gmail.com> wrote:
I agree with Eliot that changing #processPreemptionYields to true by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment.
I don't understand, in your second paragraph you say 'Pharo has #processPreemptionYields set to true' and now you say it should become the default. Is that already the case or not then ?
Oops, typo, sorry. I meant to say 'false'. (I shouldn't ever reply in a hurry).
Running the following variant, and then typing in to another window, demonstrates the behaviour:
I am not sure what you want to demonstrate: that it is totally random depending on external factors ;-) ?
If processPreemptionYields false the output should be: ... p2 p1 p2 p1 p2 p1 ... i.e. it is regular. What we're actually seeing is: ... 'p1' 'p2' 'p1' 'p2' 'p2' ... Which showing that p2 might get to signal multiple times before p1 gets to complete a single loop.
Which is pretty bad: how should semaphores be used (safely) ? What are good examples of real world correct semaphore usage ?
Given a set of processes at the same priority the amount of time allocated to any one of the processes is unpredictable. All the semaphore usage is working as described. My point wasn't that the semaphores are being used incorrectly, but that you can't make assumptions about the order in which processes will complete if they are CPU bound.
Right now, all the explanations around scheduling of processes and their priorities make it seem as if the answer is 'it all depends' and 'there is no way to be 100% sure what will happen'.
The semaphore operations are clear, but which process, within a single process priority, gets the most CPU time is less so. HTH, Alistair
If processPreemptionYields false the output should be:
... p2 p1 p2 p1 p2 p1 ...
i.e. it is regular. What we're actually seeing is:
... 'p1' 'p2' 'p1' 'p2' 'p2' ...
Which showing that p2 might get to signal multiple times before p1 gets to complete a single loop.
Yes this is why I prefer the new preemption semantics. The old semantics looks like bringing more âchaosâ. On one hand, it may give a chance to some processes to execute but I wonder also if code did not get more complex to protect against extra race conditions (since the old way could schedule more randomly). Now we were discussing some weeks ago about the risks to use the new semantics and also if we can identify the processes that we will have to change to explicitly yield. S.
While working on the booklet this week-end I was wondering (now Iâm too dead to think and going to sleep) whether there is a semantics difference between resuming a suspended process using the message resume and âresuming a processâ as a side effect of a wait signalled. S.
On Sun, 12 Jan 2020 at 20:00, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi Alistair,
On 12 Jan 2020, at 09:33, Alistair Grant <akgrant0710@gmail.com> wrote:
On Thu, 9 Jan 2020 at 13:01, ducasse <stepharo@netcourrier.com> wrote:
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of
signal.
The way this is phrased seems to imply that 'p2' will always be displayed before 'p1', however in Pharo this is not guaranteed (when the processes are at the same priority, as they are this example).
As Eliot implied in another reply, Pharo has #processPreemptionYields set to true, which means that any time a higher priority process preempts, the current process will be moved to the back of the queue.
So in the case above, after p2 signals the semaphore, if a timer was delivered or keystroke pressed, p2 would be suspended and moved to the back of the queue. When the timer / keystroke / etc. had finished processing p1 would be at the front of the queue and would complete first.
Since time and input events are (for practical purposes) unpredictable it means that the execution order of processes at a given priority is also unpredictable.
While this isn't likely to happen in the example above, I have seen it regularly with TaskIt and multiple entries being run concurrently.
I agree with Eliot that changing #processPreemptionYields to true by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment.
I don't understand, in your second paragraph you say 'Pharo has #processPreemptionYields set to true' and now you say it should become the default. Is that already the case or not then ?
Running the following variant, and then typing in to another window, demonstrates the behaviour:
I am not sure what you want to demonstrate: that it is totally random depending on external factors ;-) ?
Which is pretty bad: how should semaphores be used (safely) ? What are good examples of real world correct semaphore usage ?
Bad depends on the assumptions you are working with. The issue is its generally promoted that our scheduling is "preemptive-across-priorities, cooperative-within-priorities" but thats not entirely true for Pharo, which is "preemptive-across-priorities, mostly-cooperative-within-priorities". The former is arguably a simpler model to reason about, and having consistent implicit-behaviour between same-priority-processes lessens the need for Semaphores between them. However if you naively "assume" the former you may get burnt in Pharo since behaviour between same-priority-processes is random depending on "when" higher priority processes are scheduled. But if you "assume" the latter (i.e. that your process can be preempted any time) you'd use Semaphores as-needed and have no problems. So to reply directly to your last line. Semaphores can always be used safely. Its poor assumptions about when Semaphores aren't required that is bad. Now a new consideration for whether Pharo might change the default processPreemptionYields to false is ThreadedFFI. Presumably it will be common for a callback to be defined at same priority as an in-image process. I can't quite think through the implications myself. So a question... if a callback is a lower-priority than the current process, does it wait before grabbing the VM lock (IIUC how that is meant to work)?
Right now, all the explanations around scheduling of processes and their priorities make it seem as if the answer is 'it all depends' and 'there is no way to be 100% sure what will happen'.
Reasoning about processes at different-priorities its easy and explicit. Between processes at the same-priority you are correct, currently 'there is no way to be 100% sure what will happen' (without Semaphores). Examples showing same-priority processes interacting like its cooperative will lead to student confusion when their results differ from the book. Currently Pharo must be taught with examples presuming a fully preemptive system (at restricted locations like backward jumps). cheers -ben P.S. Now I wonder about the impact of upcoming Idle-VM. Currently same-priority-processes are effectively round-robin scheduled because the high priority DelayScheduler triggers often, bumping the current process to the back of its runQueue. When it triggers less often, anything relying on this implicit behaviour may act differently.
Hi Ben, On Sun, 12 Jan 2020 at 15:26, Ben Coman <btc@openinworld.com> wrote:
Now a new consideration for whether Pharo might change the default processPreemptionYields to false is ThreadedFFI. Presumably it will be common for a callback to be defined at same priority as an in-image process. I can't quite think through the implications myself. So a question... if a callback is a lower-priority than the current process, does it wait before grabbing the VM lock (IIUC how that is meant to work)?
The version of Threaded FFI I'm using at the moment is about a month old, but assuming nothing has changed... The callback queue is currently run at priority 70 (vs the UI process at 40). The reasoning as explained by Esteban (from memory) is that you may want callbacks to do some small amount of work and respond quickly.
P.S. Now I wonder about the impact of upcoming Idle-VM. Currently same-priority-processes are effectively round-robin scheduled because the high priority DelayScheduler triggers often, bumping the current process to the back of its runQueue. When it triggers less often, anything relying on this implicit behaviour may act differently.
Another reason to consider #processPreemptionYields set to false :-) Cheers, Alistair
Hi Alastair,
On Jan 12, 2020, at 12:34 AM, Alistair Grant <akgrant0710@gmail.com> wrote: On Thu, 9 Jan 2020 at 13:01, ducasse <stepharo@netcourrier.com> wrote:
Hi
I wanted to explain
| semaphore p1 p2 | semaphore := Semaphore new. p1 := [ semaphore wait. 'p1' crTrace ] fork.
p2 := [semaphore signal. 'p2' crTrace ] fork.
displays p2 and p1. but I would like explain clearly but it depends on the semantics of signal.
The way this is phrased seems to imply that 'p2' will always be displayed before 'p1', however in Pharo this is not guaranteed (when the processes are at the same priority, as they are this example).
As Eliot implied in another reply, Pharo has #processPreemptionYields set to true, which means that any time a higher priority process preempts, the current process will be moved to the back of the queue.
So in the case above, after p2 signals the semaphore, if a timer was delivered or keystroke pressed, p2 would be suspended and moved to the back of the queue. When the timer / keystroke / etc. had finished processing p1 would be at the front of the queue and would complete first.
Since time and input events are (for practical purposes) unpredictable it means that the execution order of processes at a given priority is also unpredictable.
While this isn't likely to happen in the example above, I have seen it regularly with TaskIt and multiple entries being run concurrently.
I agree with Eliot that changing #processPreemptionYields to true by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment
You mean to write that âI agree with Eliot that changing #processPreemptionYields to false by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment.â Preemption by a higher priority process should not cause a yield.
Running the following variant, and then typing in to another window, demonstrates the behaviour:
| semaphore p1 p2 | semaphore := Semaphore new. [ 100 timesRepeat: [ p1 := [ | z | semaphore wait. z := SmallInteger maxVal. 10000000 timesRepeat: [ z := z + 1 ]. 'p1' crTrace ] fork.
p2 := [ | z | 1 second wait. semaphore signal. z := SmallInteger maxVal. 10000000 timesRepeat: [ z := z + 1 ]. 'p2' crTrace ] fork. 1 second wait. ] ] fork.
The tail of transcript:
'p2' 'p1' 'p1' 'p1' 'p1' 'p2' 'p2' 'p2' 'p1' 'p1' 'p2' 'p1' 'p2' 'p2' 'p1' 'p1' 'p2' 'p1'
Cheers, Alistair
Cheers, Alistair! _,,,^..^,,,_ (phone)
Hi Eliot, On Sun, 12 Jan 2020 at 18:16, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Alastair,
On Jan 12, 2020, at 12:34 AM, Alistair Grant <akgrant0710@gmail.com> wrote:
I agree with Eliot that changing #processPreemptionYields to true by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment
You mean to write that
âI agree with Eliot that changing #processPreemptionYields to false by default would be an improvement in Pharo. It would make it easier to predict what is happening in a complex environment.â
Preemption by a higher priority process should not cause a yield.
Yes, sorry about that. Cheers, Alistair
participants (9)
-
Alistair Grant -
Ben Coman -
Danil Osipchuk -
ducasse -
Eliot Miranda -
Nicolas Cellier -
Norbert Hartl -
Sean P. DeNigris -
Sven Van Caekenberghe