2010/6/5 Hernan Wilkinson <hernan.wilkinson@gmail.com>:
On Fri, Jun 4, 2010 at 2:32 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
2010/6/4 Hernan Wilkinson <hernan.wilkinson@gmail.com>:
I dont think it is a necessary... I mean, if you are doing TDD, you write the test and you run it inmediatly, if it takes too much time to run (that is, more than 2 seconds :-)) then you press ctrl+ . and problem solved...
Pretend you are Stephane or Andreas and trying to integrate a bunch of tests that you did not write. How do you know whether the test programmer cared about shortness ? How do you know whether the test is expected to be long or there is a regression somewhere causing an infinite loop ?
From integrator point of view, best strategy is to reject any long test.
So, the problem is not development, it is integration... so, it is not related to TDD but how to run tests when integrating In that case, if you are integrating the first thing you have to do is to verify that the change, package, etc you are about to integrate "runs", that is, the integrator should run all the tests before integrating if not after integrating when a test fails he will not know if that is due to an integration error or an error in the original change set. Therefore if you have to run the test in the change "original" environment you will know how long they will take after running them and also how long they should take after integrating them.
Yes but I doubt Andreas and Steph would make that effort. They are not even aware of "original" environment. Either the test runs in a reasonable time slot, or has an explicit long timeout statement.
Maybe changing the integration procedure avoids the need for this "feature", that could impact when developing because of a need when integrating... I worked in a system that had around 23.000 tests and when integrating never feel the necessity for something like this... but when we started to have so many tests running all of them before integration was taking more time than expected (even thoug the 23.000 tests run in about 10 minutes or so), so we decided to change our procedure in such a way that when a programmer versioned a change, with that change we added the test running information so the integrator did not need to run the test but only see how they ran before closing the version... the time to run the test could be added to that information and make the automatic running of test when integrating take those times into account.
You did not hit the problem probably because you have a well managed environment/application. Andreas added this feature on pragmatic basis after hitting the problem. Automatically generated meta information could be a solution, but far more complex than Andreas simple hack.
If you are a producer and want to produce a long test, then you MUST declare it as such.
Of course, that breaks a few legacy tests...
I'm saying this because the should:notTakeMoreThan: has some interesting implementation details, like running the test in another process and synchronizing them, etc... so running all the test would create a process for each test, etc. tacking more time unnecessarily in most of the cases... unless the implementation takes care of only doing this special "feature" when needed (for example, when you have that pragma) but the, if you have to write the pragma why not just send the message #should:notTakeMoreThan: and problem solved?
Your problem, probably, Andreas and Steph problem, I don't think so.
hmm well again, I thought we were talking about development, but I think that there are more developers that integrators and you spend more time developing than integrating so, I still think that the solution should not affect development but concentrate on the integration part of the development process
Bye, Hernan.
Maybe... I have the feeling that if we go toward a modular image with kind of automated testing we will hit the integration problem more and more. Nicolas
Implementation details should be discussed. also the threshold is arbitrary and hardware dependent. Maybe we could use the Dorado as unit of time ;) but IMO, this feature is worth.
Nicolas
On Sat, May 29, 2010 at 3:42 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
For me the point is that I have the impression that this is an interesting feature to have when we have a test server. Like that you make sure that you do not have tests with infinite recursion now this is also true that on a test server you do not really care if your tests take 30s or 2 min.
Stef
We already have #should:notTakeMoreThan: and friends in TestCase. The complete TestCase can be protected by overriding #runCase, the individual test by wrapping the code of the test method.
Lukas
On 29 May 2010 16:43, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote:
Stef,
Time to expose some of my ignorance (don't worry, I have plenty more waiting where I found this): what is the tag concept? Â That sounds very Tweak-ish, and I am a real believer in doing things "with the language, not TO the language" whenever possible. Â That is not to say that frameworks are bad; in fact, it means that frameworks are good, language extensions are anywhere from suspect to evil.
I have some code that I am still porting, but the basic idea is to be able to write
[ Â "code that might not complete"
] tryForSeconds:10 onTimeOut:[
].
With a robust capability to do such things, it is probably not necessary (or even appropriate) for TestCase to enforce timeouts. Â The timeout block can simply raise an exception or assert false, and there is no need to disable timeouts where they do not belong.
Bill
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Michael Roberts [mike@mjr104.co.uk] Sent: Saturday, May 29, 2010 6:17 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] SUnit Time out
yes think it's a good idea. I'm not sure the granularity that's required though.
mike
On Saturday, May 29, 2010, stephane ducasse <stephane.ducasse@free.fr> wrote:
Hi guys
in Squeak andreas introduced the idea of test time out Do you think that this is interesting?
Stef
SUnit ----- All test cases now have an associated timeout after which the test is considered failed. The purpose of the timeout is to catch issues like infinite loops, unexpected user input etc. in automated test environments. Timeouts can be set on an individual test basis using the <timeout: seconds> tag or for an entire test case by implementing the #defaultTimeout method. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Lukas Renggli www.lukas-renggli.ch
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project