Hello. I want to use the benchmarks in my GSoC project. Can somebody write or suggest me some resources about how could I run written benchmarks on CI server and get there information? Best regards, Natalia
Hi Natalia, Here you have an interesting video http://vimeo.com/68494202 and a repository http://smalltalkhub.com/#!/~StefanMarr/SMark 2013/8/30 Natalia Moskovchuk <natalia.moskovchuk@unikernel.net>
Hello. I want to use the benchmarks in my GSoC project. Can somebody write or suggest me some resources about how could I run written benchmarks on CI server and get there information?
Best regards, Natalia
Thanks. I already saw this video. I cann't see configuration on https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it? -- View this message in context: http://forum.world.st/Benchmark-tp4705781p4705818.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
That you just need to run it from the command line which is not related to the benchmark project. Currently it seems like the ci has some troubles, so I cannot get to the configuration either. On 2013-08-30, at 17:25, Natalia Moskovchuk <natalia.moskovchuk@unikernel.net> wrote:
Thanks. I already saw this video. I cann't see configuration on https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?
However benchmark on the CI are not on a dedicated machine so it is not very reliable .... 2013/8/30 Camillo Bruni <camillobruni@gmail.com>
That you just need to run it from the command line which is not related to the benchmark project. Currently it seems like the ci has some troubles, so I cannot get to the configuration either.
On 2013-08-30, at 17:25, Natalia Moskovchuk < natalia.moskovchuk@unikernel.net> wrote:
Thanks. I already saw this video. I cann't see configuration on https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?
yes, it was just a proof of concept, waiting for their own dedicated machine(s) On Aug 30, 2013, at 7:03 PM, Clément Bera <bera.clement@gmail.com> wrote:
However benchmark on the CI are not on a dedicated machine so it is not very reliable ....
2013/8/30 Camillo Bruni <camillobruni@gmail.com> That you just need to run it from the command line which is not related to the benchmark project. Currently it seems like the ci has some troubles, so I cannot get to the configuration either.
On 2013-08-30, at 17:25, Natalia Moskovchuk <natalia.moskovchuk@unikernel.net> wrote:
Thanks. I already saw this video. I cann't see configuration on https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?
Thank all of you for help. I have successfuly added benchmarks to my project. And additionaly I wrote simple step by step tutorial about how to write benchmarks in Pharo for people like me)). Best regards, Natalia -- View this message in context: http://forum.world.st/Benchmark-tp4705781p4707388.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Forgot to give the link(. http://nataliatymchuk.blogspot.com/2013/09/hello.html 10.09.13 02:15, Natalia Moskovchuk ???????(??):
Thank all of you for help. I have successfuly added benchmarks to my project. And additionaly I wrote simple step by step tutorial about how to write benchmarks in Pharo for people like me)). Best regards, Natalia
-- View this message in context: http://forum.world.st/Benchmark-tp4705781p4707388.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Natalia Moskovchuk wrote:
Forgot to give the link(. http://nataliatymchuk.blogspot.com/2013/09/hello.html
10.09.13 02:15, Natalia Moskovchuk ???????(??):
Thank all of you for help. I have successfuly added benchmarks to my project. And additionaly I wrote simple step by step tutorial about how to write benchmarks in Pharo for people like me)). Best regards, Natalia
-- View this message in context: http://forum.world.st/Benchmark-tp4705781p4707388.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Thanks Natalia. That was interesting to learn. btw, I followed the link to Camillo's tutorial, and from there to benchmarkgame.com. It was interesting interesting was VW Smalltalk being faster than C for the thread-ring [2] benchmark. I'm guessing that bodes well for web programs handling many many requests each doing a small task, rather than numerical processing like many of the benchmarks. [1] http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=threadring&la... [2] http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=threadring&la... cheers -ben
Hello Natalia I started to write a chapter on Smark for a future Pharo book. So may be we could write it together :) I can give you access to it if you are interested Stef On Sep 10, 2013, at 1:15 AM, Natalia Moskovchuk <natalia.moskovchuk@unikernel.net> wrote:
Thank all of you for help. I have successfuly added benchmarks to my project. And additionaly I wrote simple step by step tutorial about how to write benchmarks in Pharo for people like me)). Best regards, Natalia
-- View this message in context: http://forum.world.st/Benchmark-tp4705781p4707388.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Hello Stephane. Yes, I'm interested in that and it will be an honor for me)). But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better . Best regards, Natalia
On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <natalia.moskovchuk@unikernel.net> wrote:
Hello Stephane. Yes, I'm interested in that and it will be an honor for me)).
Excellent. If you have a github account let me know and I will add you as a contributor Have a look at https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tre...
But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better .
No stress :) My english sucks too and we should not care for now.
Best regards, Natalia
Hello. I have a question about problemSize in benchmarking. I got the small times from running benchmarks and that's why I try to use problemSize like in the video http://vimeo.com/68494202. But the benchmark for which I rewrote code and used problemSize has given me very big result. Maybe it looks like there is no division on that problemSize. Am I wrong? What can I do? Best regards, Natalia 12.09.13 15:20, Stéphane Ducasse напиÑав(ла):
On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <natalia.moskovchuk@unikernel.net> wrote:
Hello Stephane. Yes, I'm interested in that and it will be an honor for me)). Excellent. If you have a github account let me know and I will add you as a contributor Have a look at
https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tre...
But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better . No stress :) My english sucks too and we should not care for now.
Best regards, Natalia
Short Answer: ------------- problemSize is used to calibrate your benchmark, so usually you adapt this value for your machine. And yes you are right, the result is not divided by the problemSize. Long Answer: ------------ The typical use case is like this: MathBench >> benchLoopSinus 1 to: self problemSize to: [ :i | i sin ] Now you have two parameters to modifiy: 1. the number of samples you take (aka, how many times you measure the time of #benchLoopSinus) 2. the problem size (aka, how many times you run #sin inside #benchLoopSinus) You increase (1) to get a more stable result: MathBench run: 1 "for debugging". MathBench run: 100 "will take a long time, but results are more accurate" You modify (2) to change the duration of your benchmark, in my silly example above, the method is quite small and the benchmark would finish too quickly. Rule of thumb is to get the run time of your benchmark (here #benchLoopSinus) in the range of 10ms and more. This way you don't have to worry about the timer resolution used to capture the duration of your method. Does this answer your question? On 2013-09-14, at 08:06, Natalia Tymchuk <natalia.tymchuk@unikernel.net> wrote:
Hello. I have a question about problemSize in benchmarking. I got the small times from running benchmarks and that's why I try to use problemSize like in the video http://vimeo.com/68494202. But the benchmark for which I rewrote code and used problemSize has given me very big result. Maybe it looks like there is no division on that problemSize. Am I wrong? What can I do? Best regards, Natalia
12.09.13 15:20, Stéphane Ducasse напиÑав(ла):
On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <natalia.moskovchuk@unikernel.net> wrote:
Hello Stephane. Yes, I'm interested in that and it will be an honor for me)). Excellent. If you have a github account let me know and I will add you as a contributor Have a look at
https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tre...
But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better . No stress :) My english sucks too and we should not care for now.
Best regards, Natalia
Thanks, Camillo, for that big and very useful letter. You helped me a lot. Best regards, Natalia 14.09.13 15:16, Camillo Bruni напиÑав(ла):
Short Answer: ------------- problemSize is used to calibrate your benchmark, so usually you adapt this value for your machine. And yes you are right, the result is not divided by the problemSize.
Long Answer: ------------ The typical use case is like this:
MathBench >> benchLoopSinus 1 to: self problemSize to: [ :i | i sin ]
Now you have two parameters to modifiy: 1. the number of samples you take (aka, how many times you measure the time of #benchLoopSinus) 2. the problem size (aka, how many times you run #sin inside #benchLoopSinus)
You increase (1) to get a more stable result:
MathBench run: 1 "for debugging". MathBench run: 100 "will take a long time, but results are more accurate"
You modify (2) to change the duration of your benchmark, in my silly example above, the method is quite small and the benchmark would finish too quickly. Rule of thumb is to get the run time of your benchmark (here #benchLoopSinus) in the range of 10ms and more. This way you don't have to worry about the timer resolution used to capture the duration of your method.
Does this answer your question?
On 2013-09-14, at 08:06, Natalia Tymchuk <natalia.tymchuk@unikernel.net> wrote:
Hello. I have a question about problemSize in benchmarking. I got the small times from running benchmarks and that's why I try to use problemSize like in the video http://vimeo.com/68494202. But the benchmark for which I rewrote code and used problemSize has given me very big result. Maybe it looks like there is no division on that problemSize. Am I wrong? What can I do? Best regards, Natalia
12.09.13 15:20, Stéphane Ducasse напиÑав(ла):
On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <natalia.moskovchuk@unikernel.net> wrote:
Hello Stephane. Yes, I'm interested in that and it will be an honor for me)). Excellent. If you have a github account let me know and I will add you as a contributor Have a look at
https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tre...
But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better . No stress :) My english sucks too and we should not care for now.
Best regards, Natalia
we should add that to the book chapter :) On Sep 14, 2013, at 2:16 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
Short Answer: ------------- problemSize is used to calibrate your benchmark, so usually you adapt this value for your machine. And yes you are right, the result is not divided by the problemSize.
Long Answer: ------------ The typical use case is like this:
MathBench >> benchLoopSinus 1 to: self problemSize to: [ :i | i sin ]
Now you have two parameters to modifiy: 1. the number of samples you take (aka, how many times you measure the time of #benchLoopSinus) 2. the problem size (aka, how many times you run #sin inside #benchLoopSinus)
You increase (1) to get a more stable result:
MathBench run: 1 "for debugging". MathBench run: 100 "will take a long time, but results are more accurate"
You modify (2) to change the duration of your benchmark, in my silly example above, the method is quite small and the benchmark would finish too quickly. Rule of thumb is to get the run time of your benchmark (here #benchLoopSinus) in the range of 10ms and more. This way you don't have to worry about the timer resolution used to capture the duration of your method.
Does this answer your question?
On 2013-09-14, at 08:06, Natalia Tymchuk <natalia.tymchuk@unikernel.net> wrote:
Hello. I have a question about problemSize in benchmarking. I got the small times from running benchmarks and that's why I try to use problemSize like in the video http://vimeo.com/68494202. But the benchmark for which I rewrote code and used problemSize has given me very big result. Maybe it looks like there is no division on that problemSize. Am I wrong? What can I do? Best regards, Natalia
12.09.13 15:20, Stéphane Ducasse напиÑав(ла):
On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <natalia.moskovchuk@unikernel.net> wrote:
Hello Stephane. Yes, I'm interested in that and it will be an honor for me)). Excellent. If you have a github account let me know and I will add you as a contributor Have a look at
https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tre...
But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better . No stress :) My english sucks too and we should not care for now.
Best regards, Natalia
participants (9)
-
btc＠openinworld.com -
Camillo Bruni -
Clément Bera -
Esteban Lorenzano -
Natalia Moskovchuk -
Natalia Tymchuk
-
Natalia Tymchuk -
Sebastian Tleye -
Stéphane Ducasse