Re: [Pharo-dev] [Vm-dev] [ANN] ARM Stack VMs available
Hi Juan, On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)" <juanlists@jvuletich.org> wrote:
Hi Folks,
The Squeak process scheduler usually preempts a process only by another one of higher priority. And when the scheduler is ready to go back at the lower priority, it will always resume the same process that was last suspended. The only way for two or more processes of the same priority to share the processor is when the running process calls #wait or #yield.
No. According to the blue book preempting sends the preempted process to the back if its run queue so preempting forces a yield. Obviously this is wrong, but that's the way it was defined. The ObjectWorks & VisualWorks VMs changed this to work as you describe. When I implemented Cog I wanted to fix this but not break old code so I made it optional. The behavior is controlled by a flag in the image. You can see the flags via Smalltalk vmParameterAt: 48. Bit 2 (value 4) is set if preempting puts the preempted process at the head of its list, and hence does not yield. Here's the part of the comment from vmParameterAt: 48 various properties of the Cog VM as an integer encoding an array of bit flags. Bit 0: implies the image's Process class has threadId as its 3rd inst var (zero relative) Bit 1: on Cog VMs asks the VM to set the flag bit in interpreted methods Bit 2: if set, preempting a process puts it to the head of its run queue, not the back, i.e. preempting a process by a higher one will not cause the process to yield to others at the same priority.
This means that using shared state from several processes at the same priority (without using any locking mechanism, such as Semaphores, Monitors or critical sections) is safe, as long as the processes do #wait or #yield only when that shared state is in a consistent state.
Right.
It looks like this ARM VM is somehow not following this convention. If this was the case, the problems Ken sees are to be expected.
Can you check the bit is set in the image? Thanks for raising this. It looks like the Spur images have the bit cleared. Ouch. Ah, it's not a problem of the bootstrap, its at least a problem in trunk. The bit is not set in trunk. I need to a) write some tests to test the bit and b) make sure its set in the relevant places.
HTH, Juan Vuletich
Eliot (phone)
Quoting Douglas McPherson <djm1329@san.rr.com>:
Hi Ken,
On Nov 11, 2014, at 16:28, Ken Dickey <Ken.Dickey@whidbey.com> wrote:
On Sun, 9 Nov 2014 20:56:47 -0800 Douglas McPherson <djm1329@san.rr.com> wrote:
KenD>>
The stkspurlinuxhtARM VM, however, complains: This interpreter (vers. 6521) cannot read image file (vers. 6505).
I presume some conversion is required to get the image into SPUR format? Yes, the Spur VMs require a new image format. There are recent Squeak trunk images in the spur format on Eliotâs site. Iâll let Eliot and others comment about how a Cuis image might be converted.
I tried out the Squeak SPUR image from the Cog site and everything Squeak seems fine.
There is a timing bug with Cuis redraw that shows up on both the stklinuxhtARM vm and the one you sent me earlier.
Basically, I am using DisplayObject>>slideFrom:to:nSteps:delay: to do some animation. When multiple timer events fire during an animation it sometimes appears that the x,y redraw origin is used from the currently drawing morph's position, not 0,0. I.e. part of the redisplay is offset. [Display redraw via menu corrects the display].
This _only_ happens on the Samsung ARM Chromebook, never on Intel CPUs.
Thanks for reporting this. Iâll try to look into it. There are definitely ARM-specific changes to support certain atomic operations, so there may be something not quite right with these.
Can you suggest a method for reproducing this in Squeak? I repeatedly sent morphs to the trash (with appropriate preference set to enable the animation) but didnât see an issue. I suspect Iâm not driving it hard enough.
My life is really, really busy right now, so I have never had time to dig into this. I can redraw the display, so a workaround exists.
[One possible resource. The Apache Portable Runtime, http://apr.apache.org/ , implements compareAndSwap and various other atomic operations across various OSs.]
Thanks for the reference. It may be useful. I also found a library called Mintomic (http://preshing.com/20130505/introducing-mintomic-a-small-portable-lock-free...) which may help.
I am happy to do quick tests, but must apologize for lack of time to dig into this.
No problems, and thanks again for reporting it.
FYI, -KenD
(inline) Quoting Eliot Miranda <eliot.miranda@gmail.com>:
Hi Juan,
On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)" <juanlists@jvuletich.org> wrote:
Hi Folks,
The Squeak process scheduler usually preempts a process only by another one of higher priority. And when the scheduler is ready to go back at the lower priority, it will always resume the same process that was last suspended. The only way for two or more processes of the same priority to share the processor is when the running process calls #wait or #yield.
No. According to the blue book preempting sends the preempted process to the back if its run queue so preempting forces a yield. Obviously this is wrong, but that's the way it was defined. The ObjectWorks & VisualWorks VMs changed this to work as you describe.
Oh, apologies. Thanks for correcting me. But this means that understanding the problems Ken and Tim found on the PI is not that easy, because in other platforms the problem don't occur, even with olders VMs and (I guess) on slower machines. I don't have a PI myself, so I can't do much testing. Maybe something in the VM timer(s), event fetching, or semaphore signaling is different in the PI from others VMs? Or maybe the possible interrupt points are different? Maybe something like this could make a higher priority process wake up at usual times, or under less restricted conditions?
When I implemented Cog I wanted to fix this but not break old code so I made it optional. The behavior is controlled by a flag in the image. You can see the flags via Smalltalk vmParameterAt: 48. Bit 2 (value 4) is set if preempting puts the preempted process at the head of its list, and hence does not yield.
Here's the part of the comment from vmParameterAt:
48 various properties of the Cog VM as an integer encoding an array of bit flags. Bit 0: implies the image's Process class has threadId as its 3rd inst var (zero relative) Bit 1: on Cog VMs asks the VM to set the flag bit in interpreted methods Bit 2: if set, preempting a process puts it to the head of its run queue, not the back, i.e. preempting a process by a higher one will not cause the process to yield to others at the same priority.
This means that using shared state from several processes at the same priority (without using any locking mechanism, such as Semaphores, Monitors or critical sections) is safe, as long as the processes do #wait or #yield only when that shared state is in a consistent state.
Right.
It looks like this ARM VM is somehow not following this convention. If this was the case, the problems Ken sees are to be expected.
Can you check the bit is set in the image?
Thanks for raising this. It looks like the Spur images have the bit cleared. Ouch. Ah, it's not a problem of the bootstrap, its at least a problem in trunk. The bit is not set in trunk. I need to a) write some tests to test the bit and b) make sure its set in the relevant places.
Glad to be of help! Cheers, Juan Vuletich
HTH, Juan Vuletich
Eliot (phone)
Quoting Douglas McPherson <djm1329@san.rr.com>:
Hi Ken,
On Nov 11, 2014, at 16:28, Ken Dickey <Ken.Dickey@whidbey.com> wrote:
On Sun, 9 Nov 2014 20:56:47 -0800 Douglas McPherson <djm1329@san.rr.com> wrote:
KenD>>
The stkspurlinuxhtARM VM, however, complains: This interpreter (vers. 6521) cannot read image file (vers. 6505).
I presume some conversion is required to get the image into SPUR format? Yes, the Spur VMs require a new image format. There are recent Squeak trunk images in the spur format on Eliotâs site. Iâll let Eliot and others comment about how a Cuis image might be converted.
I tried out the Squeak SPUR image from the Cog site and everything Squeak seems fine.
There is a timing bug with Cuis redraw that shows up on both the stklinuxhtARM vm and the one you sent me earlier.
Basically, I am using DisplayObject>>slideFrom:to:nSteps:delay: to do some animation. When multiple timer events fire during an animation it sometimes appears that the x,y redraw origin is used from the currently drawing morph's position, not 0,0. I.e. part of the redisplay is offset. [Display redraw via menu corrects the display].
This _only_ happens on the Samsung ARM Chromebook, never on Intel CPUs.
Thanks for reporting this. Iâll try to look into it. There are definitely ARM-specific changes to support certain atomic operations, so there may be something not quite right with these.
Can you suggest a method for reproducing this in Squeak? I repeatedly sent morphs to the trash (with appropriate preference set to enable the animation) but didnât see an issue. I suspect Iâm not driving it hard enough.
My life is really, really busy right now, so I have never had time to dig into this. I can redraw the display, so a workaround exists.
[One possible resource. The Apache Portable Runtime, http://apr.apache.org/ , implements compareAndSwap and various other atomic operations across various OSs.]
Thanks for the reference. It may be useful. I also found a library called Mintomic (http://preshing.com/20130505/introducing-mintomic-a-small-portable-lock-free...) which may help.
I am happy to do quick tests, but must apologize for lack of time to dig into this.
No problems, and thanks again for reporting it.
FYI, -KenD
On Fri, Dec 5, 2014 at 11:39 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Juan,
On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)" <juanlists@jvuletich.org> wrote:
Hi Folks,
The Squeak process scheduler usually preempts a process only by another one of higher priority. And when the scheduler is ready to go back at the lower priority, it will always resume the same process that was last suspended. The only way for two or more processes of the same priority to share the processor is when the running process calls #wait or #yield.
No. According to the blue book preempting sends the preempted process to the back if its run queue so preempting forces a yield. Obviously this is wrong, but that's the way it was defined. The ObjectWorks & VisualWorks VMs changed this to work as you describe.
Why is that wrong? By sending it to the back, one gets task-switching that acts like real "multitasking" as one would expect in a real multitasking system, without the developer having to put in explicit sends to #yield. Keeping it at the front "enables" multi-processing code to be developed incorrectly so that, one day, when it might ever get to a real multitasking environment, it won't work and it'll be harder to debug since by then its old and probably more complex. #yield is not a good synchronization tool..
Quoting Chris Muller <asqueaker@gmail.com>:
On Fri, Dec 5, 2014 at 11:39 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Juan,
On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)" <juanlists@jvuletich.org> wrote:
Hi Folks,
The Squeak process scheduler usually preempts a process only by another one of higher priority. And when the scheduler is ready to go back at the lower priority, it will always resume the same process that was last suspended. The only way for two or more processes of the same priority to share the processor is when the running process calls #wait or #yield.
No. According to the blue book preempting sends the preempted process to the back if its run queue so preempting forces a yield. Obviously this is wrong, but that's the way it was defined. The ObjectWorks & VisualWorks VMs changed this to work as you describe.
Why is that wrong? By sending it to the back, one gets task-switching that acts like real "multitasking" as one would expect in a real multitasking system, without the developer having to put in explicit sends to #yield. Keeping it at the front "enables" multi-processing code to be developed incorrectly so that, one day, when it might ever get to a real multitasking environment, it won't work and it'll be harder to debug since by then its old and probably more complex. #yield is not a good synchronization tool..
Well, yes. But for that to work we'd need a highest priority process that keeps sleeping for, lets say, 1 mSec, and does nothing. That way all processes would be suspended and shuffled all the time. The way it is right now, this shuffling won't happen often, and ("bad") code that assumes the behavior I described would work "most of the time", breaking randomly, and more often on slower hardware (such as the PI)... Is this worth doing? This "bad" code would fail all the time, and it would be spotted and fixed. But we would break (a rather illusory) back compatibility... For instance, the code Ken is using for animating the game cards (moving morphs around in a separate process) would break for sure. But fixing it would also fix the issues he sees on the PI. It is not hard to imagine that the same would happen to the sound problems Tim found on Scratch on the PI... Cheers, Juan Vuletich
J. Vuletich (mail lists) wrote:
Quoting Chris Muller <asqueaker@gmail.com>:
On Fri, Dec 5, 2014 at 11:39 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Juan,
On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)" <juanlists@jvuletich.org> wrote:
Hi Folks,
The Squeak process scheduler usually preempts a process only by another one of higher priority. And when the scheduler is ready to go back at the lower priority, it will always resume the same process that was last suspended. The only way for two or more processes of the same priority to share the processor is when the running process calls #wait or #yield.
No. According to the blue book preempting sends the preempted process to the back if its run queue so preempting forces a yield. Obviously this is wrong, but that's the way it was defined. The ObjectWorks & VisualWorks VMs changed this to work as you describe.
Why is that wrong? By sending it to the back, one gets task-switching that acts like real "multitasking" as one would expect in a real multitasking system, without the developer having to put in explicit sends to #yield. Keeping it at the front "enables" multi-processing code to be developed incorrectly so that, one day, when it might ever get to a real multitasking environment, it won't work and it'll be harder to debug since by then its old and probably more complex. #yield is not a good synchronization tool..
Well, yes. But for that to work we'd need a highest priority process that keeps sleeping for, lets say, 1 mSec, and does nothing. That way all processes would be suspended and shuffled all the time.
A few times I've seen the idea raised of moving process scheduling into the Image. This sounds interesting. For Pharo** I recently factored the delay scheduling event loop code out from Delay class side to its own DelayScheduler. Now actually I don't see any reason that new class needs to be tied only to Delays. It might also provide process scheduling. Where the VM currently decides which process to run, conceivably the /timingSemepahore/ could be signaled to wake up and decide which process to run next, then call back into the VM to dispatch the selected process. Having process scheduling in the VM would conceivably facilitate development of domain specific process scheduling - e.g. different for interactive UI, web servers, batch processing, robotics, control systems, real-time media. Also this would make a great environment for Computer Science courses to teach process scheduling algorithms. The following looks interesting, although I don't follow all of it... https://computeradventures.wordpress.com/tag/squeak-smalltalk-vm-scheduling/
The way it is right now, this shuffling won't happen often, and ("bad") code that assumes the behavior I described would work "most of the time", breaking randomly, and more often on slower hardware (such as the PI)...
Is this worth doing? This "bad" code would fail all the time, and it would be spotted and fixed. But we would break (a rather illusory) back compatibility...
Less of an issue for Pharo. Maybe this is something to consider for Pharo 5 ? cheers -ben
For instance, the code Ken is using for animating the game cards (moving morphs around in a separate process) would break for sure. But fixing it would also fix the issues he sees on the PI. It is not hard to imagine that the same would happen to the sound problems Tim found on Scratch on the PI...
participants (4)
-
Ben Coman -
Chris Muller -
Eliot Miranda -
J. Vuletich (mail lists)