[Pharo-project] [ANN] A new scheduler + VM changes alpha-release
Hello guys, i hope you are interested in having a rock-solid & safe concurrency in smalltalk, despite it green-threaded. A new implementation of scheduler will allow us to improve stuff and be no longer dependant from VM side scheduling policy. You can find the changes on mantis: http://bugs.squeak.org/view.php?id=7345 Please, give me a feedback, if you can successfully build VM with new changes on all platforms. I used a pretty old VMMaker package version for my VM: VMMaker-dtl.91 Will try to see if it can be built with latest one , available on SqS. Andreas (& others) - do you have any tests/benchmarks, how to measure a new scheduler overhead comparing to old one? Personally, i didn't noticed much speed degradation :) And last thing, i need an advice how to implement a #callbackEnter: properly. Its currently leaved unchanged, using old VM scheduling policy. So, its unsafe to use plugins with callbacks if you running under new scheduler. A callbackEnter remembers the active process, suspends it and then activates it back when callback is done. The problem is, that squeak 3.10 doesn't have any image-side code which i could change to play with callbacks :( A fix for callbacks should be pretty easy: - at entry, deactivate active process (which should be NOT the interrupt process) & switch to interrupt process, so it could schedule next available process. - when leaving switch from interrupt process to previously remembered active process. So, a certain preconditions should be met: - a code which calls a primitives which using callbacks should never run in interruptProcess - a code , which returns from a callback should run in interruptProcess -- Best regards, Igor Stasenko AKA sig.
i added a new changeset in mantis which deals with callbackEnter: code. Unfortunately, a callback handling , to work correctly needs an image-side changes as well. P.S. Having fun to watch how scheduler handling signals, by inspecting Process and selecting its 'signals' ivar :) -- Best regards, Igor Stasenko AKA sig.
A quick comparion for Delays: delay := Delay forMilliseconds: 1. bag := Bag new. 1000 timesRepeat:[bag add: [delay wait] timeToRun]. bag sortedCounts on my 4-core box it yields: - with AdvancedProcessorScheduler install a SortedCollection(951->2 49->1) - with Processor fallbackToOldScheduler a SortedCollection(953->2 47->1) - with old VM a SortedCollection(952->2 47->1 1->3) not much overhead huh? :) But Delay's code now can be refactored, which will simplify it seriously, because it could run in interrupt process and take no care for concurrency issues. 2009/4/30 Igor Stasenko <siguctua@gmail.com>:
i added a new changeset in mantis which deals with callbackEnter: code. Unfortunately, a callback handling , to work correctly needs an image-side changes as well.
P.S. Having fun to watch how scheduler handling signals, by inspecting Process and selecting its 'signals' ivar :)
-- Best regards, Igor Stasenko AKA sig.
-- Best regards, Igor Stasenko AKA sig.
I think that this is a good idea to see how such parts of Ssqueak/ pharo can be improved in the future. On Apr 30, 2009, at 4:15 PM, Igor Stasenko wrote:
A quick comparion for Delays:
delay := Delay forMilliseconds: 1. bag := Bag new. 1000 timesRepeat:[bag add: [delay wait] timeToRun]. bag sortedCounts
on my 4-core box it yields:
- with AdvancedProcessorScheduler install a SortedCollection(951->2 49->1)
- with Processor fallbackToOldScheduler a SortedCollection(953->2 47->1)
- with old VM a SortedCollection(952->2 47->1 1->3)
not much overhead huh? :)
But Delay's code now can be refactored, which will simplify it seriously, because it could run in interrupt process and take no care for concurrency issues.
2009/4/30 Igor Stasenko <siguctua@gmail.com>:
i added a new changeset in mantis which deals with callbackEnter: code. Unfortunately, a callback handling , to work correctly needs an image-side changes as well.
P.S. Having fun to watch how scheduler handling signals, by inspecting Process and selecting its 'signals' ivar :)
-- Best regards, Igor Stasenko AKA sig.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Here the Pharo benchs 100 "processes" benchSwitch: 5. "seconds" 1) "No changes to Processor" '2,520,509 switches/sec' '2,520,590 switches/sec' '2,413,224 switches/sec' 2) "After changes to Processor, but using old VM primitives" '1,106,806 switches/sec' '1,110,870 switches/sec' '1,116,981 switches/sec' '1,204,289 switches/sec' "After installing new Processor" '152,779 switches/sec' '150,913 switches/sec' '152,608 switches/sec' '153,844 switches/sec' Its interesting that transition from 1) to 2) which just increasing a send chain to get to primitive, like: Semaphore>>wait >> primitive Semaphore>>wait >> Processor>>waitForSemaphore:>>Semaphore>>primitiveWait>>primitive downgrades a bench numbers by more than a 2x factor. 2009/4/30 Stéphane Ducasse <stephane.ducasse@inria.fr>:
I think that this is a good idea to see how such parts of Ssqueak/ pharo can be improved in the future.
On Apr 30, 2009, at 4:15 PM, Igor Stasenko wrote:
A quick comparion for Delays:
   delay := Delay forMilliseconds: 1.    bag := Bag new.    1000 timesRepeat:[bag add: [delay wait] timeToRun].    bag sortedCounts
on my 4-core box it yields:
- with AdvancedProcessorScheduler install  a SortedCollection(951->2 49->1)
- with Processor fallbackToOldScheduler a SortedCollection(953->2 47->1)
- with old VM Â a SortedCollection(952->2 47->1 1->3)
not much overhead huh? :)
But Delay's code now can be refactored, which will simplify it seriously, because it could run in interrupt process and take no care for concurrency issues.
2009/4/30 Igor Stasenko <siguctua@gmail.com>:
i added a new changeset in mantis which deals with callbackEnter: code. Unfortunately, a callback handling , to work correctly needs an image-side changes as well.
P.S. Having fun to watch how scheduler handling signals, by inspecting Process and selecting its 'signals' ivar :)
-- Best regards, Igor Stasenko AKA sig.
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
2009/4/30 Igor Stasenko <siguctua@gmail.com>:
Here the Pharo benchs
100 "processes" benchSwitch: 5. "seconds"
1) "No changes to Processor" '2,520,509 switches/sec' '2,520,590 switches/sec' '2,413,224 switches/sec'
2) "After changes to Processor, but using old VM primitives" '1,106,806 switches/sec' '1,110,870 switches/sec' '1,116,981 switches/sec' '1,204,289 switches/sec'
"After installing new Processor"
'152,779 switches/sec' '150,913 switches/sec' '152,608 switches/sec' '153,844 switches/sec'
Its interesting that transition from 1) to 2) which just increasing a send chain to get to primitive, like:
What's even more interesting that i added a bench: AdvancedProcessorScheduler>>bench: seconds | proc count | count := 0. proc := [ [ Processor interruptWith: [count := count + 1] ] repeat ] forkAt: Processor activePriority -1. (Delay forSeconds: seconds) wait. proc terminate. ^(count value // seconds) asStringWithCommas, ' switches/sec' And it shows following results: Processor bench: 5 '1,039,398 switches/sec' '1,060,727 switches/sec' '1,083,370 switches/sec' As you can see, a scheduler loop alone is quite performant and nearly matching the speed of VM based scheduler. Which makes me wonder why semaphore signal/wait causes so much speed degradation... Looks like it does looping where it shouldn't. Need to inspect it more closely. :)
Semaphore>>wait >> primitive Semaphore>>wait >> Processor>>waitForSemaphore:>>Semaphore>>primitiveWait>>primitive
downgrades a bench numbers by more than a 2x factor.
2009/4/30 Stéphane Ducasse <stephane.ducasse@inria.fr>:
-- Best regards, Igor Stasenko AKA sig.
participants (2)
-
Igor Stasenko -
Stéphane Ducasse