Hello all I have done some experiments, and basically all the questions in my previous mail can be ignored. I think it is all a question of timing, which I sort of understand. Perhaps someone can clarify for me. All my previous tests involved running the whole test suite, to ensure the set-up and tear-down methods were run. As an experiment, I selected the one red method and clicked 'Run Tests'. After about 10 seconds, a halt appeared with the message 'Test took too long'. When I clicked 'Proceed', the green appeared almost immediately. I tried again several times with similar results, although the halt occurred at different places in the code. I went back to my old P6 version and tried running tests on the one method; each time, it ran to green in about 11-12 seconds. So back now to the P7 version; wondered if there is now a setting for permitted test duration. Eventually found it under 'System', default 10 secs. Changed to 15 secs, re-run red test, now comes up green. Re-run whole test suite, all green. So it all comes down to my ignorance of test settings. But Pharo is less than helpful; if you run the whole test suite, there is no way to distinguish a test which failed due to a wrong assertion from one which was terminated for taking too long - or did I miss something? Peter Kenny
Peter, You can override #defaultTimeLimit on the class side of the offending test. Yes, this can be confusing. There is a new test runner under development, DrTest, in Pharo 8. Sven
On 20 Oct 2019, at 16:37, PBKResearch <peter@pbkresearch.co.uk> wrote:
Hello all
I have done some experiments, and basically all the questions in my previous mail can be ignored. I think it is all a question of timing, which I sort of understand. Perhaps someone can clarify for me.
All my previous tests involved running the whole test suite, to ensure the set-up and tear-down methods were run. As an experiment, I selected the one red method and clicked âRun Testsâ. After about 10 seconds, a halt appeared with the message âTest took too longâ. When I clicked âProceedâ, the green appeared almost immediately. I tried again several times with similar results, although the halt occurred at different places in the code. I went back to my old P6 version and tried running tests on the one method; each time, it ran to green in about 11-12 seconds.
So back now to the P7 version; wondered if there is now a setting for permitted test duration. Eventually found it under âSystemâ, default 10 secs. Changed to 15 secs, re-run red test, now comes up green. Re-run whole test suite, all green.
So it all comes down to my ignorance of test settings. But Pharo is less than helpful; if you run the whole test suite, there is no way to distinguish a test which failed due to a wrong assertion from one which was terminated for taking too long â or did I miss something?
Peter Kenny
Sven Thanks - the real problem is my unfamiliarity with P7, so hints like this are helpful. I am happier with familiar systems, but I will switch to the latest Moose suite soon. Peter -----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Sven Van Caekenberghe Sent: 20 October 2019 17:31 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: Re: [Pharo-users] Strategy for investigating test failures Peter, You can override #defaultTimeLimit on the class side of the offending test. Yes, this can be confusing. There is a new test runner under development, DrTest, in Pharo 8. Sven
On 20 Oct 2019, at 16:37, PBKResearch <peter@pbkresearch.co.uk> wrote:
Hello all
I have done some experiments, and basically all the questions in my previous mail can be ignored. I think it is all a question of timing, which I sort of understand. Perhaps someone can clarify for me.
All my previous tests involved running the whole test suite, to ensure the set-up and tear-down methods were run. As an experiment, I selected the one red method and clicked âRun Testsâ. After about 10 seconds, a halt appeared with the message âTest took too longâ. When I clicked âProceedâ, the green appeared almost immediately. I tried again several times with similar results, although the halt occurred at different places in the code. I went back to my old P6 version and tried running tests on the one method; each time, it ran to green in about 11-12 seconds.
So back now to the P7 version; wondered if there is now a setting for permitted test duration. Eventually found it under âSystemâ, default 10 secs. Changed to 15 secs, re-run red test, now comes up green. Re-run whole test suite, all green.
So it all comes down to my ignorance of test settings. But Pharo is less than helpful; if you run the whole test suite, there is no way to distinguish a test which failed due to a wrong assertion from one which was terminated for taking too long â or did I miss something?
Peter Kenny
Confusion of failing test aside - is this something that got worse in Pharo 7? Did these tests pass in under 10s in 6 and take longer in 7? This would be a concrete thing to address, and something to check in 8. Tim Sent from my iPhone
On 20 Oct 2019, at 20:11, PBKResearch <peter@pbkresearch.co.uk> wrote:
Sven
Thanks - the real problem is my unfamiliarity with P7, so hints like this are helpful. I am happier with familiar systems, but I will switch to the latest Moose suite soon.
Peter
-----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Sven Van Caekenberghe Sent: 20 October 2019 17:31 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: Re: [Pharo-users] Strategy for investigating test failures
Peter,
You can override #defaultTimeLimit on the class side of the offending test.
Yes, this can be confusing.
There is a new test runner under development, DrTest, in Pharo 8.
Sven
On 20 Oct 2019, at 16:37, PBKResearch <peter@pbkresearch.co.uk> wrote:
Hello all
I have done some experiments, and basically all the questions in my previous mail can be ignored. I think it is all a question of timing, which I sort of understand. Perhaps someone can clarify for me.
All my previous tests involved running the whole test suite, to ensure the set-up and tear-down methods were run. As an experiment, I selected the one red method and clicked âRun Testsâ. After about 10 seconds, a halt appeared with the message âTest took too longâ. When I clicked âProceedâ, the green appeared almost immediately. I tried again several times with similar results, although the halt occurred at different places in the code. I went back to my old P6 version and tried running tests on the one method; each time, it ran to green in about 11-12 seconds.
So back now to the P7 version; wondered if there is now a setting for permitted test duration. Eventually found it under âSystemâ, default 10 secs. Changed to 15 secs, re-run red test, now comes up green. Re-run whole test suite, all green.
So it all comes down to my ignorance of test settings. But Pharo is less than helpful; if you run the whole test suite, there is no way to distinguish a test which failed due to a wrong assertion from one which was terminated for taking too long â or did I miss something?
Peter Kenny
Tim: No, on P6 the test took 11-12 seconds (judged on my ability to count seconds) and the same on P7. Presumably P6 did not have a limit, or it was over 10s. Peter -----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Tim Mackinnon Sent: 20 October 2019 22:35 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: Re: [Pharo-users] Strategy for investigating test failures Confusion of failing test aside - is this something that got worse in Pharo 7? Did these tests pass in under 10s in 6 and take longer in 7? This would be a concrete thing to address, and something to check in 8. Tim Sent from my iPhone
On 20 Oct 2019, at 20:11, PBKResearch <peter@pbkresearch.co.uk> wrote:
Sven
Thanks - the real problem is my unfamiliarity with P7, so hints like this are helpful. I am happier with familiar systems, but I will switch to the latest Moose suite soon.
Peter
-----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Sven Van Caekenberghe Sent: 20 October 2019 17:31 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: Re: [Pharo-users] Strategy for investigating test failures
Peter,
You can override #defaultTimeLimit on the class side of the offending test.
Yes, this can be confusing.
There is a new test runner under development, DrTest, in Pharo 8.
Sven
On 20 Oct 2019, at 16:37, PBKResearch <peter@pbkresearch.co.uk> wrote:
Hello all
I have done some experiments, and basically all the questions in my previous mail can be ignored. I think it is all a question of timing, which I sort of understand. Perhaps someone can clarify for me.
All my previous tests involved running the whole test suite, to ensure the set-up and tear-down methods were run. As an experiment, I selected the one red method and clicked âRun Testsâ. After about 10 seconds, a halt appeared with the message âTest took too longâ. When I clicked âProceedâ, the green appeared almost immediately. I tried again several times with similar results, although the halt occurred at different places in the code. I went back to my old P6 version and tried running tests on the one method; each time, it ran to green in about 11-12 seconds.
So back now to the P7 version; wondered if there is now a setting for permitted test duration. Eventually found it under âSystemâ, default 10 secs. Changed to 15 secs, re-run red test, now comes up green. Re-run whole test suite, all green.
So it all comes down to my ignorance of test settings. But Pharo is less than helpful; if you run the whole test suite, there is no way to distinguish a test which failed due to a wrong assertion from one which was terminated for taking too long â or did I miss something?
Peter Kenny
Tim: No, on P6 the test took 11-12 seconds (judged on my ability to count seconds) and the same on P7. Presumably P6 did not have a limit, or it was over 10s. Peter -----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Tim Mackinnon Sent: 20 October 2019 22:35 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: Re: [Pharo-users] Strategy for investigating test failures Confusion of failing test aside - is this something that got worse in Pharo 7? Did these tests pass in under 10s in 6 and take longer in 7? This would be a concrete thing to address, and something to check in 8. Tim Sent from my iPhone
On 20 Oct 2019, at 20:11, PBKResearch <peter@pbkresearch.co.uk> wrote:
Sven
Thanks - the real problem is my unfamiliarity with P7, so hints like this are helpful. I am happier with familiar systems, but I will switch to the latest Moose suite soon.
Peter
-----Original Message----- From: Pharo-users <pharo-users-bounces@lists.pharo.org> On Behalf Of Sven Van Caekenberghe Sent: 20 October 2019 17:31 To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: Re: [Pharo-users] Strategy for investigating test failures
Peter,
You can override #defaultTimeLimit on the class side of the offending test.
Yes, this can be confusing.
There is a new test runner under development, DrTest, in Pharo 8.
Sven
On 20 Oct 2019, at 16:37, PBKResearch <peter@pbkresearch.co.uk> wrote:
Hello all
I have done some experiments, and basically all the questions in my previous mail can be ignored. I think it is all a question of timing, which I sort of understand. Perhaps someone can clarify for me.
All my previous tests involved running the whole test suite, to ensure the set-up and tear-down methods were run. As an experiment, I selected the one red method and clicked âRun Testsâ. After about 10 seconds, a halt appeared with the message âTest took too longâ. When I clicked âProceedâ, the green appeared almost immediately. I tried again several times with similar results, although the halt occurred at different places in the code. I went back to my old P6 version and tried running tests on the one method; each time, it ran to green in about 11-12 seconds.
So back now to the P7 version; wondered if there is now a setting for permitted test duration. Eventually found it under âSystemâ, default 10 secs. Changed to 15 secs, re-run red test, now comes up green. Re-run whole test suite, all green.
So it all comes down to my ignorance of test settings. But Pharo is less than helpful; if you run the whole test suite, there is no way to distinguish a test which failed due to a wrong assertion from one which was terminated for taking too long â or did I miss something?
Peter Kenny
participants (3)
-
PBKResearch -
Sven Van Caekenberghe -
Tim Mackinnon