(Copying squeak-dev too). I'm not sold on the whole test timeout thing. When I run tests, I want to know the answer to the question, "is the software working?" Putting a timeout on tests trades a slower, but definitive, "yes" or "no" for a supposedly-faster "maybe". But is getting a "maybe" back really faster? I've just incurred the cost of running a test suite, but left without my answer. I get a "maybe", what am I supposed to do next? Find a faster machine? Hack into the code to fiddle with a timeout pragma? That's not faster.. But, the reason given for the change was not for running tests interactively (the 99% case), rather, all tests form the beginning of time are now saddled with a timeout for the 1% case: "The purpose of the timeout is to catch issues like infinite loops, unexpected user input etc. in automated test environments." If tests are supposed to be quick (and deterministic) anyway, wouldn't an infinite loop or user-input be caught the first time the test was run (interactively)? Seriously, when you make software changes, we run the tests interactively first, and then the purpose of night-time automated test environment is to catch regressions on the merged code.. In that case, the high-level test-controller which spits out the results could and should be responsible for handling "unexpected user input" and/or putting in a timeout, not each and every last test method.. IMO, we want short tests, so let's just write them to be short. If they're too long, then the encouragement to shorten them comes from our own impatience of running them interactively. Running them in batch at night requires no patience, because we're sleeping, and besides, the batch processor should take responsibility for handling those rare scenarios at a higher-level.. Regards, Chris On Sat, May 29, 2010 at 2:53 AM, stephane ducasse <stephane.ducasse@free.fr> wrote:
Hi guys
in Squeak andreas introduced the idea of test time out Do you think that this is interesting?
Stef
SUnit ----- All test cases now have an associated timeout after which the test is considered failed. The purpose of the timeout is to catch issues like infinite loops, unexpected user input etc. in automated test environments. Timeouts can be set on an individual test basis using the <timeout: seconds> tag or for an entire test case by implementing the #defaultTimeout method. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project