Hi camillo/Stefan I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start? I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn. Stef
I started to read the code (and its self-documentation aspect is not totally there). (SMarkLoops new runOnly: #benchFloatLoop) run Stefan I started to comment the code so are you interested in the commits? Stef
Hi camillo/Stefan
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn.
Stef
SMarkLoops profile: #benchArrayAccess does not work I was wondering if this is the class side version of runBenchmark: Stef On Jun 16, 2013, at 12:12 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
I started to read the code (and its self-documentation aspect is not totally there).
(SMarkLoops new runOnly: #benchFloatLoop) run
Stefan I started to comment the code so are you interested in the commits?
Stef
Hi camillo/Stefan
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn.
Stef
ok apparently I was running it on 30 and not 20. Stef On Jun 16, 2013, at 7:16 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
SMarkLoops profile: #benchArrayAccess
does not work
I was wondering if this is the class side version of runBenchmark:
Stef
On Jun 16, 2013, at 12:12 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
I started to read the code (and its self-documentation aspect is not totally there).
(SMarkLoops new runOnly: #benchFloatLoop) run
Stefan I started to comment the code so are you interested in the commits?
Stef
Hi camillo/Stefan
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn.
Stef
#profile: is supposed to open the profiler on the given benchmark. let me fix that :) On 2013-06-16, at 19:16, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
SMarkLoops profile: #benchArrayAccess
does not work
I was wondering if this is the class side version of runBenchmark:
Stef
On Jun 16, 2013, at 12:12 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
I started to read the code (and its self-documentation aspect is not totally there).
(SMarkLoops new runOnly: #benchFloatLoop) run
Stefan I started to comment the code so are you interested in the commits?
Stef
Hi camillo/Stefan
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn.
Stef
actually that is related to a bug in 3.0 :/ => https://pharo.fogbugz.com/default.asp?10920 On 2013-06-16, at 19:50, Camillo Bruni <camillobruni@gmail.com> wrote:
#profile: is supposed to open the profiler on the given benchmark. let me fix that :)
On 2013-06-16, at 19:16, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
SMarkLoops profile: #benchArrayAccess
does not work
I was wondering if this is the class side version of runBenchmark:
Stef
On Jun 16, 2013, at 12:12 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
I started to read the code (and its self-documentation aspect is not totally there).
(SMarkLoops new runOnly: #benchFloatLoop) run
Stefan I started to comment the code so are you interested in the commits?
Stef
Hi camillo/Stefan
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn.
Stef
thanks I will integrate it. Stef On Jun 16, 2013, at 8:00 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
actually that is related to a bug in 3.0 :/
=> https://pharo.fogbugz.com/default.asp?10920
On 2013-06-16, at 19:50, Camillo Bruni <camillobruni@gmail.com> wrote:
#profile: is supposed to open the profiler on the given benchmark. let me fix that :)
On 2013-06-16, at 19:16, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
SMarkLoops profile: #benchArrayAccess
does not work
I was wondering if this is the class side version of runBenchmark:
Stef
On Jun 16, 2013, at 12:12 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
I started to read the code (and its self-documentation aspect is not totally there).
(SMarkLoops new runOnly: #benchFloatLoop) run
Stefan I started to comment the code so are you interested in the commits?
Stef
Hi camillo/Stefan
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn.
Stef
Camillo it looks like SMarkHarness is a good candidate for having a nice commandline handler. May be we could pair program one :) Stef
actually that is related to a bug in 3.0 :/
=> https://pharo.fogbugz.com/default.asp?10920
On 2013-06-16, at 19:50, Camillo Bruni <camillobruni@gmail.com> wrote:
#profile: is supposed to open the profiler on the given benchmark. let me fix that :)
On 2013-06-16, at 19:16, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
SMarkLoops profile: #benchArrayAccess
does not work
I was wondering if this is the class side version of runBenchmark:
Stef
On Jun 16, 2013, at 12:12 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
I started to read the code (and its self-documentation aspect is not totally there).
(SMarkLoops new runOnly: #benchFloatLoop) run
Stefan I started to comment the code so are you interested in the commits?
Stef
Hi camillo/Stefan
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn.
Stef
There is another approach done by JuanPablo for Rizel running on the new Pharo CLI infrastructure. So we can definitely hack together and merge the two approaches :) On 2013-06-16, at 21:07, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Camillo
it looks like SMarkHarness is a good candidate for having a nice commandline handler. May be we could pair program one :)
Stef
actually that is related to a bug in 3.0 :/
=> https://pharo.fogbugz.com/default.asp?10920
On 2013-06-16, at 19:50, Camillo Bruni <camillobruni@gmail.com> wrote:
#profile: is supposed to open the profiler on the given benchmark. let me fix that :)
On 2013-06-16, at 19:16, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
SMarkLoops profile: #benchArrayAccess
does not work
I was wondering if this is the class side version of runBenchmark:
Stef
On Jun 16, 2013, at 12:12 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
I started to read the code (and its self-documentation aspect is not totally there).
(SMarkLoops new runOnly: #benchFloatLoop) run
Stefan I started to comment the code so are you interested in the commits?
Stef
Hi camillo/Stefan
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I'm thinking that I could use the implementation of Trie of Benoit and the one of Cuis as an example to do some benchs. Since I'm not fluent in benchmarks, I think that this the occasion to learn.
Stef
Hi Stef: On 16 Jun 2013, at 12:12, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I am not aware of a blog post or similar things. All we got at the moment are class comments. The entry class is the benchmark harness: SMarkHarness. The benchmark SMarkSuite has also a comment, which gives minimal information of how to use it. Note that I use SMark mostly from the command line. So, it only has a limited set of convenience methods. Most of the time, I use `SMarkHarness run:` as in the example below.
I started to read the code (and its self-documentation aspect is not totally there). Stefan I started to comment the code so are you interested in the commits?
Sure, they can always be improved. Contributions are welcome.
(SMarkLoops new runOnly: #benchFloatLoop) run
Ehm, where did you find that? That is not supposed to work currently, because the benchmark suite is not independent of the rest of the infrastructure and needs to be initialized properly (most notably, it needs a benchmark runner, which actually executes the benchmarks, we use that indirection for instance to warm up the CogVM properly). `SMarkLoops run` is supposed to work. If you want a convenience method like #runOnly: it should be implemented on the class-side like #run. On the other hand, following the class comment of SMarkHarness, you can also use: SMarkHarness run: {'SMarkHarness'. 'SMarkLoops.benchFloatLoop'. 10. 1. 500} But you probably want to open a transcript first, and then execute: `ScriptConsole outputToTranscript: true.`. Otherwise, you will probably not see the output, because it goes normally only to stdout on the command line. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
Hi Stef:
On 16 Jun 2013, at 12:12, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I am not aware of a blog post or similar things.
it is coming :) see the attachment.
All we got at the moment are class comments.
The entry class is the benchmark harness: SMarkHarness.
Ok I missed it.
The benchmark SMarkSuite has also a comment, which gives minimal information of how to use it. Note that I use SMark mostly from the command line. So, it only has a limited set of convenience methods. Most of the time, I use `SMarkHarness run:` as in the example below.
what do you think about initializing the runner of a suite to the be the default one? This way we could do SmarkLoops new runBenchmark: #benchArrayAccess. What is the <omniUnenforced>? "Hint for the OMOP that it is part of the meta infrastructure" I kind of get that this is a kind of annotation for your phd thesis.
I started to read the code (and its self-documentation aspect is not totally there). Stefan I started to comment the code so are you interested in the commits?
Sure, they can always be improved. Contributions are welcome.
(SMarkLoops new runOnly: #benchFloatLoop) run
Ehm, where did you find that? That is not supposed to work currently, because the benchmark suite is not independent of the rest of the infrastructure and needs to be initialized properly (most notably, it needs a benchmark runner, which actually executes the benchmarks, we use that indirection for instance to warm up the CogVM properly).
Yes I saw
`SMarkLoops run` is supposed to work. If you want a convenience method like #runOnly: it should be implemented on the class-side like #run.
This is also what I thought. But since it was only on the class side I tried it on the intstance
On the other hand, following the class comment of SMarkHarness, you can also use:
SMarkHarness run: {'SMarkHarness'. 'SMarkLoops.benchFloatLoop'. 10. 1. 500}
But you probably want to open a transcript first, and then execute: `ScriptConsole outputToTranscript: true.`. Otherwise, you will probably not see the output, because it goes normally only to stdout on the command line.
ok I will continue to read the code.
Best regards Stefan
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
Hi Stef: On 16 Jun 2013, at 20:43, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
I am not aware of a blog post or similar things.
it is coming :) see the attachment.
Just a general remark: I am not sure what Camillo's opinion is, but he had a substantial part in SMark, and if you need to mention names, he should certainly be included.
The benchmark SMarkSuite has also a comment, which gives minimal information of how to use it. Note that I use SMark mostly from the command line. So, it only has a limited set of convenience methods. Most of the time, I use `SMarkHarness run:` as in the example below.
what do you think about initializing the runner of a suite to the be the default one? This way we could do
SmarkLoops new runBenchmark: #benchArrayAccess.
I think you would rather want something like: SMarkSuite class >> #runOnly: aSymbol "aSymbol should refer to a benchmark method. Example: SMarkLoops runOnly: #benchFloatLoop " ^ self defaultRunner execute: (self new runOnly: aSymbol)
What is the <omniUnenforced>? "Hint for the OMOP that it is part of the meta infrastructure" I kind of get that this is a kind of annotation for your phd thesis. Yes. It is a hint for the VM or bytecode transformation that this methods does not need to be subject to the metaobject protocol.
Since it does not really hurt, I did not spend the effort to keep it out of SMark. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
Stefan for your info SMarkHarness run: {'SMarkHarness'. 'SMarkLoops.benchIntLoop'. 1. 1. 5} does not work. Stef
Hi Stef:
On 16 Jun 2013, at 12:12, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I am not aware of a blog post or similar things.
All we got at the moment are class comments.
The entry class is the benchmark harness: SMarkHarness.
The benchmark SMarkSuite has also a comment, which gives minimal information of how to use it. Note that I use SMark mostly from the command line. So, it only has a limited set of convenience methods. Most of the time, I use `SMarkHarness run:` as in the example below.
I started to read the code (and its self-documentation aspect is not totally there). Stefan I started to comment the code so are you interested in the commits?
Sure, they can always be improved. Contributions are welcome.
(SMarkLoops new runOnly: #benchFloatLoop) run
Ehm, where did you find that? That is not supposed to work currently, because the benchmark suite is not independent of the rest of the infrastructure and needs to be initialized properly (most notably, it needs a benchmark runner, which actually executes the benchmarks, we use that indirection for instance to warm up the CogVM properly).
`SMarkLoops run` is supposed to work. If you want a convenience method like #runOnly: it should be implemented on the class-side like #run.
On the other hand, following the class comment of SMarkHarness, you can also use:
SMarkHarness run: {'SMarkHarness'. 'SMarkLoops.benchFloatLoop'. 10. 1. 500}
But you probably want to open a transcript first, and then execute: `ScriptConsole outputToTranscript: true.`. Otherwise, you will probably not see the output, because it goes normally only to stdout on the command line.
Best regards Stefan
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
there is no ScriptConsole but I imagine that using Transcript is ok. On Jun 16, 2013, at 8:45 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
Stefan
for your info SMarkHarness run: {'SMarkHarness'. 'SMarkLoops.benchIntLoop'. 1. 1. 5} does not work.
Stef
Hi Stef:
On 16 Jun 2013, at 12:12, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I am not aware of a blog post or similar things.
All we got at the moment are class comments.
The entry class is the benchmark harness: SMarkHarness.
The benchmark SMarkSuite has also a comment, which gives minimal information of how to use it. Note that I use SMark mostly from the command line. So, it only has a limited set of convenience methods. Most of the time, I use `SMarkHarness run:` as in the example below.
I started to read the code (and its self-documentation aspect is not totally there). Stefan I started to comment the code so are you interested in the commits?
Sure, they can always be improved. Contributions are welcome.
(SMarkLoops new runOnly: #benchFloatLoop) run
Ehm, where did you find that? That is not supposed to work currently, because the benchmark suite is not independent of the rest of the infrastructure and needs to be initialized properly (most notably, it needs a benchmark runner, which actually executes the benchmarks, we use that indirection for instance to warm up the CogVM properly).
`SMarkLoops run` is supposed to work. If you want a convenience method like #runOnly: it should be implemented on the class-side like #run.
On the other hand, following the class comment of SMarkHarness, you can also use:
SMarkHarness run: {'SMarkHarness'. 'SMarkLoops.benchFloatLoop'. 10. 1. 500}
But you probably want to open a transcript first, and then execute: `ScriptConsole outputToTranscript: true.`. Otherwise, you will probably not see the output, because it goes normally only to stdout on the command line.
Best regards Stefan
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
Ok I will load the configuration file. It would be good to rename the configuration: ConfigurationOfSMark like that it is consistent with the project name. I thought that ConfigurationOfBenchmarking was something else. Stef On Jun 16, 2013, at 8:47 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
there is no ScriptConsole but I imagine that using Transcript is ok.
On Jun 16, 2013, at 8:45 PM, Stéphane Ducasse <Stephane.Ducasse@inria.fr> wrote:
Stefan
for your info SMarkHarness run: {'SMarkHarness'. 'SMarkLoops.benchIntLoop'. 1. 1. 5} does not work.
Stef
Hi Stef:
On 16 Jun 2013, at 12:12, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
I would like to know how to use SMark and document it. Do you have a start of a blog/doc/slide that I can use as a start?
I am not aware of a blog post or similar things.
All we got at the moment are class comments.
The entry class is the benchmark harness: SMarkHarness.
The benchmark SMarkSuite has also a comment, which gives minimal information of how to use it. Note that I use SMark mostly from the command line. So, it only has a limited set of convenience methods. Most of the time, I use `SMarkHarness run:` as in the example below.
I started to read the code (and its self-documentation aspect is not totally there). Stefan I started to comment the code so are you interested in the commits?
Sure, they can always be improved. Contributions are welcome.
(SMarkLoops new runOnly: #benchFloatLoop) run
Ehm, where did you find that? That is not supposed to work currently, because the benchmark suite is not independent of the rest of the infrastructure and needs to be initialized properly (most notably, it needs a benchmark runner, which actually executes the benchmarks, we use that indirection for instance to warm up the CogVM properly).
`SMarkLoops run` is supposed to work. If you want a convenience method like #runOnly: it should be implemented on the class-side like #run.
On the other hand, following the class comment of SMarkHarness, you can also use:
SMarkHarness run: {'SMarkHarness'. 'SMarkLoops.benchFloatLoop'. 10. 1. 500}
But you probably want to open a transcript first, and then execute: `ScriptConsole outputToTranscript: true.`. Otherwise, you will probably not see the output, because it goes normally only to stdout on the command line.
Best regards Stefan
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
Stefan note also that ConfigurationOfBenchmarking - refers to SqueakSource - refers to Scripting but scripting is not in the smalltalkhub/StefanMarr/SMark/main Stef
Hi Stef: On 16 Jun 2013, at 20:52, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
note also that ConfigurationOfBenchmarking - refers to SqueakSource - refers to Scripting but scripting is not in the smalltalkhub/StefanMarr/SMark/main
The repo now includes a few commits that haven't been uploaded before. They should also fix the reference and make it point to SmalltalkHub. I don't plan to rename the configuration at the moment. The original intention was to use a more general name, because it includes also the Computer Language Benchmark Game package. Which is not strictly speaking SMark. And, a number of my projects depend on the current configuration. If you feel the need for a specific ConfigurationOfSMark, that for instance does not include the Computer Language Benchmarks, feel free to commit one. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
On 2013-06-16, at 22:28, Stefan Marr <smalltalk@stefan-marr.de> wrote:
Hi Stef:
On 16 Jun 2013, at 20:52, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
note also that ConfigurationOfBenchmarking - refers to SqueakSource - refers to Scripting but scripting is not in the smalltalkhub/StefanMarr/SMark/main
The repo now includes a few commits that haven't been uploaded before. They should also fix the reference and make it point to SmalltalkHub.
I don't plan to rename the configuration at the moment. The original intention was to use a more general name, because it includes also the Computer Language Benchmark Game package. Which is not strictly speaking SMark. And, a number of my projects depend on the current configuration.
If you feel the need for a specific ConfigurationOfSMark, that for instance does not include the Computer Language Benchmarks, feel free to commit one.
humm seems like you introduced some invalid dependencies: https://ci.inria.fr/pharo-contribution/job/SMark/PHARO=20,VERSION=stable,VM=...
Hi Camillo: On 16 Jun 2013, at 23:31, Camillo Bruni <camillobruni@gmail.com> wrote:
humm seems like you introduced some invalid dependencies:
https://ci.inria.fr/pharo-contribution/job/SMark/PHARO=20,VERSION=stable,VM=...
Thanks! Forgot to migrate the NAS Parallel Benchmarks⦠Fixed it. Is there a way, I could get email notifications for broken builds? Thanks Stefan PS: The NPB port currently includes only the integer sorting benchmark, and it's broken on Pharo. My best guess being that we have closures instead of blocks. While the RoarVM images still use blocks. Never got around to fix it. -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
On 2013-06-17, at 00:03, Stefan Marr <smalltalk@stefan-marr.de> wrote:
Hi Camillo:
On 16 Jun 2013, at 23:31, Camillo Bruni <camillobruni@gmail.com> wrote:
humm seems like you introduced some invalid dependencies:
https://ci.inria.fr/pharo-contribution/job/SMark/PHARO=20,VERSION=stable,VM=...
Thanks! Forgot to migrate the NAS Parallel Benchmarksâ¦
Fixed it.
Is there a way, I could get email notifications for broken builds?
jup, I added it, be prepared to get spammed ;)
Thanks Stefan
PS: The NPB port currently includes only the integer sorting benchmark, and it's broken on Pharo. My best guess being that we have closures instead of blocks. While the RoarVM images still use blocks. Never got around to fix it.
participants (3)
-
Camillo Bruni -
Stefan Marr -
Stéphane Ducasse