Test Runner: "Run Coverage" fails or hangs when multithreading is used in test.
Hi, When I want to run the coverage of for example Zincâs Client and Server tests, it will either hang (in the case of Zincâs test cases) or fail because the coverage uses BlockClosure>>valueUnpreemptively for running the tests. The relevant method is TestRunner>>collectCoverageFor:. So when a test is run, it is not able to be preempted by another process, like for example a local server which is needed to run the actual test in Zincâs case. When I use BlockClosure>>valueUninterruptably it works. Can someone tell me if it is wrong to use that instead? Is valueUnpreemptively necessary in this case, and if so, why? To reproduce: Load fresh image, select Zinc's ZnClientTests and ZnServerTests, click on âRun Coverageâ -> hanging image. Thanks, Skip
I never used that, but it seems coverage can only deal with single threaded code, which sounds logical. No ?
On 12 Jan 2016, at 10:53, Skip Lentz <skip.lentz@inria.fr> wrote:
Hi,
When I want to run the coverage of for example Zincâs Client and Server tests, it will either hang (in the case of Zincâs test cases) or fail because the coverage uses BlockClosure>>valueUnpreemptively for running the tests. The relevant method is TestRunner>>collectCoverageFor:.
So when a test is run, it is not able to be preempted by another process, like for example a local server which is needed to run the actual test in Zincâs case.
When I use BlockClosure>>valueUninterruptably it works. Can someone tell me if it is wrong to use that instead? Is valueUnpreemptively necessary in this case, and if so, why?
To reproduce: Load fresh image, select Zinc's ZnClientTests and ZnServerTests, click on âRun Coverageâ -> hanging image.
Thanks,
Skip
You mean as in it can check the coverage effected by the code running in the other thread? Or is this not what you mean? Well I mean, at least it can show the coverage done by what was run in the same thread.. And it will at least not hang or fail.
On Jan 12, 2016, at 10:59 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
I never used that, but it seems coverage can only deal with single threaded code, which sounds logical. No ?
On 12 Jan 2016, at 10:53, Skip Lentz <skip.lentz@inria.fr> wrote:
Hi,
When I want to run the coverage of for example Zincâs Client and Server tests, it will either hang (in the case of Zincâs test cases) or fail because the coverage uses BlockClosure>>valueUnpreemptively for running the tests. The relevant method is TestRunner>>collectCoverageFor:.
So when a test is run, it is not able to be preempted by another process, like for example a local server which is needed to run the actual test in Zincâs case.
When I use BlockClosure>>valueUninterruptably it works. Can someone tell me if it is wrong to use that instead? Is valueUnpreemptively necessary in this case, and if so, why?
To reproduce: Load fresh image, select Zinc's ZnClientTests and ZnServerTests, click on âRun Coverageâ -> hanging image.
Thanks,
Skip
"can NOT check"*, in the first sentence.
On Jan 12, 2016, at 11:05 AM, Skip Lentz <skip.lentz@inria.fr> wrote:
You mean as in it can check the coverage effected by the code running in the other thread? Or is this not what you mean?
Well I mean, at least it can show the coverage done by what was run in the same thread.. And it will at least not hang or fail.
On Jan 12, 2016, at 10:59 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
I never used that, but it seems coverage can only deal with single threaded code, which sounds logical. No ?
On 12 Jan 2016, at 10:53, Skip Lentz <skip.lentz@inria.fr> wrote:
Hi,
When I want to run the coverage of for example Zincâs Client and Server tests, it will either hang (in the case of Zincâs test cases) or fail because the coverage uses BlockClosure>>valueUnpreemptively for running the tests. The relevant method is TestRunner>>collectCoverageFor:.
So when a test is run, it is not able to be preempted by another process, like for example a local server which is needed to run the actual test in Zincâs case.
When I use BlockClosure>>valueUninterruptably it works. Can someone tell me if it is wrong to use that instead? Is valueUnpreemptively necessary in this case, and if so, why?
To reproduce: Load fresh image, select Zinc's ZnClientTests and ZnServerTests, click on âRun Coverageâ -> hanging image.
Thanks,
Skip
The coverage seems to be implemented by 'locking' the current execution thread exclusively, which would mean no other threads can run, at all. So code depending on multi threading, like client/server code, like HTTP, cannot work. Yes that seems to be a limitation, but an understandable one. I am trying to help but I don't known anything about the coverage tool, so maybe I should shut up ;-)
On 12 Jan 2016, at 11:05, Skip Lentz <skip.lentz@inria.fr> wrote:
You mean as in it can check the coverage effected by the code running in the other thread? Or is this not what you mean?
Well I mean, at least it can show the coverage done by what was run in the same thread.. And it will at least not hang or fail.
On Jan 12, 2016, at 10:59 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
I never used that, but it seems coverage can only deal with single threaded code, which sounds logical. No ?
On 12 Jan 2016, at 10:53, Skip Lentz <skip.lentz@inria.fr> wrote:
Hi,
When I want to run the coverage of for example Zincâs Client and Server tests, it will either hang (in the case of Zincâs test cases) or fail because the coverage uses BlockClosure>>valueUnpreemptively for running the tests. The relevant method is TestRunner>>collectCoverageFor:.
So when a test is run, it is not able to be preempted by another process, like for example a local server which is needed to run the actual test in Zincâs case.
When I use BlockClosure>>valueUninterruptably it works. Can someone tell me if it is wrong to use that instead? Is valueUnpreemptively necessary in this case, and if so, why?
To reproduce: Load fresh image, select Zinc's ZnClientTests and ZnServerTests, click on âRun Coverageâ -> hanging image.
Thanks,
Skip
Haha, donât worry. You donât have to shut up, in fact I thank you for replying. I donât know exactly how it has been implemented, I had a look see and it seems to be like this: It seems to collect all methods that will be checked for coverage, and then instantiate a wrapper around each one, and replace the method with the wrapper in the method dictionary of the class (and restore it afterwards, obviously). Maybe it should only lock the threads that were _not_ started by the âtest threadâ? Iâm not even sure that is possible, just suggesting things.
On Jan 12, 2016, at 11:11 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The coverage seems to be implemented by 'locking' the current execution thread exclusively, which would mean no other threads can run, at all. So code depending on multi threading, like client/server code, like HTTP, cannot work. Yes that seems to be a limitation, but an understandable one.
I am trying to help but I don't known anything about the coverage tool, so maybe I should shut up ;-)
On 12 Jan 2016, at 11:05, Skip Lentz <skip.lentz@inria.fr> wrote:
You mean as in it can check the coverage effected by the code running in the other thread? Or is this not what you mean?
Well I mean, at least it can show the coverage done by what was run in the same thread.. And it will at least not hang or fail.
On Jan 12, 2016, at 10:59 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
I never used that, but it seems coverage can only deal with single threaded code, which sounds logical. No ?
On 12 Jan 2016, at 10:53, Skip Lentz <skip.lentz@inria.fr> wrote:
Hi,
When I want to run the coverage of for example Zincâs Client and Server tests, it will either hang (in the case of Zincâs test cases) or fail because the coverage uses BlockClosure>>valueUnpreemptively for running the tests. The relevant method is TestRunner>>collectCoverageFor:.
So when a test is run, it is not able to be preempted by another process, like for example a local server which is needed to run the actual test in Zincâs case.
When I use BlockClosure>>valueUninterruptably it works. Can someone tell me if it is wrong to use that instead? Is valueUnpreemptively necessary in this case, and if so, why?
To reproduce: Load fresh image, select Zinc's ZnClientTests and ZnServerTests, click on âRun Coverageâ -> hanging image.
Thanks,
Skip
Maybe it should only lock the threads that were _not_ started by the âtest threadâ? Iâm not even sure that is possible, just suggesting things.
Have a look at Niall Ross' XProc Patterns. They do the same by - AFAIK - intercepting process creation and checking whether the creating process is one of the "tests threads". HTH, Jan
On Jan 12, 2016, at 11:11 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
The coverage seems to be implemented by 'locking' the current execution thread exclusively, which would mean no other threads can run, at all. So code depending on multi threading, like client/server code, like HTTP, cannot work. Yes that seems to be a limitation, but an understandable one.
I am trying to help but I don't known anything about the coverage tool, so maybe I should shut up ;-)
On 12 Jan 2016, at 11:05, Skip Lentz <skip.lentz@inria.fr> wrote:
You mean as in it can check the coverage effected by the code running in the other thread? Or is this not what you mean?
Well I mean, at least it can show the coverage done by what was run in the same thread.. And it will at least not hang or fail.
On Jan 12, 2016, at 10:59 AM, Sven Van Caekenberghe <sven@stfx. eu> wrote:
I never used that, but it seems coverage can only deal with single threaded code, which sounds logical. No ?
On 12 Jan 2016, at 10:53, Skip Lentz <skip.lentz@inria.fr> wrote:
Hi,
When I want to run the coverage of for example Zincâs Client and Server tests, it will either hang (in the case of Zincâs test cases) or fail because the coverage uses BlockClosure>>valueUnpreemptively for running the tests. The relevant method is TestRunner>>collectCoverageFor:.
So when a test is run, it is not able to be preempted by another process, like for example a local server which is needed to run the actual test in Zincâs case.
When I use BlockClosure>>valueUninterruptably it works. Can someone tell me if it is wrong to use that instead? Is valueUnpreemptively necessary in this case, and if so, why?
To reproduce: Load fresh image, select Zinc's ZnClientTests and ZnServerTests, click on âRun Coverageâ -> hanging image.
Thanks,
Skip
participants (3)
-
Jan Vrany -
Skip Lentz -
Sven Van Caekenberghe