[Pharo-project] SUnit and Tests inheritance or 'Data' injection
Hi: I have a design problem with my test cases. In my benchmark suite I have a runner class that controls how benchmarks are executed. Now I want to subclass it to provide a runner that automatically finds a reasonable number of iterations for microbenchmarks. Thus, it will first measure the time of a first run and then double the number of iterations until it has some like at least 300ms runtime. Ok, thats the domain design, now the question how do I test that hierarchy with SUnit? It seems like my test hierarchy that parallels the domain hierarchy is not actually working like I would expected it. Thus, my basic test class implements some 9 tests, and my auto-sizing test-class adds two more test. However, when I run the tests, only 11 tests are executed and not 2*9 + 2 = 20 tests. Is there something I am missing? Is there a way that I can force SUnit to also execute the tests of the superclass? Or, something that would actually be nice, is there a way that I can automatically parameterize my tests with data? Then I would not need to us subclassing in my test hierarchy, but could just tell the basic tests to run with all the different runner classes, and for the autosizing runners, I can just run additional tests. Thanks Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525 -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
On Sat, Mar 19, 2011 at 6:52 PM, Stefan Marr <pharo@stefan-marr.de> wrote:
Hi:
I have a design problem with my test cases.
In my benchmark suite I have a runner class that controls how benchmarks are executed. Now I want to subclass it to provide a runner that automatically finds a reasonable number of iterations for microbenchmarks. Thus, it will first measure the time of a first run and then double the number of iterations until it has some like at least 300ms runtime.
Ok, thats the domain design, now the question how do I test that hierarchy with SUnit? It seems like my test hierarchy that parallels the domain hierarchy is not actually working like I would expected it.
Thus, my basic test class implements some 9 tests, and my auto-sizing test-class adds two more test. However, when I run the tests, only 11 tests are executed and not 2*9 + 2 = 20 tests. Is there something I am missing? Is there a way that I can force SUnit to also execute the tests of the superclass?
Yes the tests defined in the superclass are not executed. Another way is to define a Trait with your 9 tests, remove the superclass and have the other test classes use that Trait - tests for collections use this patterns, ProfStef tutorial tests too.
Or, something that would actually be nice, is there a way that I can automatically parameterize my tests with data?
Could you explain a little more ? Laurent
Then I would not need to us subclassing in my test hierarchy, but could just tell the basic tests to run with all the different runner classes, and for the autosizing runners, I can just run additional tests.
Thanks Stefan
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
Hi Laurent: On 19 Mar 2011, at 19:28, laurent laffont wrote:
Yes the tests defined in the superclass are not executed.
Another way is to define a Trait with your 9 tests, remove the superclass and have the other test classes use that Trait - tests for collections use this patterns, ProfStef tutorial tests too. Traits are unfortunately not really feasible, I think. I need to be Squeak 3.7 compatible.
Or, something that would actually be nice, is there a way that I can automatically parameterize my tests with data?
Could you explain a little more ? I guess thats basically what data-driven testing is about.
In my scenario it would be sufficient to be able to tell the test runner, here is the class MyTest, please run it with a set of input parameters. And then I give it an array {foo. bar. baz} and tell it to inject for each iteration one of the values into property current_foobar. So MyTest is executed three times instead of 1 time. Are there examples for such use cases somewhere? Thanks Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
Thus, my basic test class implements some 9 tests, and my auto-sizing test-class adds two more test. However, when I run the tests, only 11 tests are executed and not 2*9 + 2 = 20 tests. Is there something I am missing? Is there a way that I can force SUnit to also execute the tests of the superclass?
See the method TestCase class>>#shouldInheritSelectors.
Or, something that would actually be nice, is there a way that I can automatically parameterize my tests with data?
In your test class create a test method that takes the parameter(s) as argument. Create several test methods that call this method with different arguments. Lukas -- Lukas Renggli www.lukas-renggli.ch
Hi Lukas: On 19 Mar 2011, at 20:19, Lukas Renggli wrote:
Thus, my basic test class implements some 9 tests, and my auto-sizing test-class adds two more test. However, when I run the tests, only 11 tests are executed and not 2*9 + 2 = 20 tests. Is there something I am missing? Is there a way that I can force SUnit to also execute the tests of the superclass?
See the method TestCase class>>#shouldInheritSelectors. Thanks, thats what I was looking for :)
Or, something that would actually be nice, is there a way that I can automatically parameterize my tests with data?
In your test class create a test method that takes the parameter(s) as argument. Create several test methods that call this method with different arguments. Ehm, that approach does not scale...
My test has currently 9 #testMethods, if I understand your proposal correctly, than I would end up with n*9 methods, no? With #shouldInheritSelectors it still feels like a hack, but at least I avoid having to create all those methods. Laurent, the @DataProvider feature of TestNG is basically what I was looking for: http://testng.org/doc/documentation-main.html#parameters Thanks Stefan
Lukas
-- Lukas Renggli www.lukas-renggli.ch
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
I was suggesting the same as lukas. and yes somebody should design a new Testing frameworks and we keep Sunit for backward compatibility Now have you tried to use traits and inject the parameters as fixture. Because this is what we did for the collection hierarchy. Stef On Mar 19, 2011, at 8:46 PM, Stefan Marr wrote:
Hi Lukas:
On 19 Mar 2011, at 20:19, Lukas Renggli wrote:
Thus, my basic test class implements some 9 tests, and my auto-sizing test-class adds two more test. However, when I run the tests, only 11 tests are executed and not 2*9 + 2 = 20 tests. Is there something I am missing? Is there a way that I can force SUnit to also execute the tests of the superclass?
See the method TestCase class>>#shouldInheritSelectors. Thanks, thats what I was looking for :)
Or, something that would actually be nice, is there a way that I can automatically parameterize my tests with data?
In your test class create a test method that takes the parameter(s) as argument. Create several test methods that call this method with different arguments. Ehm, that approach does not scale...
My test has currently 9 #testMethods, if I understand your proposal correctly, than I would end up with n*9 methods, no? With #shouldInheritSelectors it still feels like a hack, but at least I avoid having to create all those methods.
Laurent, the @DataProvider feature of TestNG is basically what I was looking for: http://testng.org/doc/documentation-main.html#parameters
Thanks Stefan
Lukas
-- Lukas Renggli www.lukas-renggli.ch
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
participants (4)
-
laurent laffont -
Lukas Renggli -
Stefan Marr -
Stéphane Ducasse