On Apr 21, 2010, at 11:06 AM, Peter Hugosson-Miller wrote:
I'm trying to follow along with this discussion, but now I feel really stupid :-(
I've been using SUnit since 1998 (when it was still called "BeckTestingFramework"),
testingFramework on VW :)
and ever since "expected failures" showed up, I've never understood them. So I wonder if some kind guru-like person could please explain to me what they are useful for? I mean, to my way of thinking, if one writes a test that is expected to fail, then why not invert the test and call it a success instead?
for example:
self should: [answer = 42]
self assert: answer = 42 because [] are for self should: [] raise: Error and friend
...as an expected failure could simply be re-written as
self should: [answer ~= 42]
...right? No, obviously I've missed something really obvious and important, and that's why I'm asking the question now. Please be gentle ;-)
-- Cheers, Peter
On Wed, Apr 21, 2010 at 10:30 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote: We should have a look at what adrian did now the problem is that understanding a large set of changes is more difficult than a couple of simple ones. If somebody want to help we are open. Stef
I think Adrian Kuhn did that in his SUnit work. I also remember he also introduced a difference between expectedFailures and expectedErrors.
Doru
On 21 Apr 2010, at 10:16, Stéphane Ducasse wrote:
On Apr 21, 2010, at 9:51 AM, Adrian Lienhard wrote:
Yea, I agree, the GUI is suboptimal.
I still think, though, that treating this case as a failure is correct. For instance, consider the case where you had added a workaround to a known bug and when the bug is fixed you need to remove the workaround again. Maybe it even leads to a wrong behavior now that the bug is gone. In this case you really want to know that the test does not fail anymore.
yes Now I have the impression that expectedFailures should be like passes, failed, errors: a state of the tests.
Stef
In any case, I think that tagging methods as expected failures should be done with pragmas and not with #expectedFailures. Like this it would also be much easier to understand what's going on when you have a failure in this test although all assertions pass.
Adrian
On Apr 21, 2010, at 08:22 , Stéphane Ducasse wrote:
On Apr 20, 2010, at 11:20 PM, Adrian Lienhard wrote:
Yes, if a test that is expected to fail does not fail, this is treated as a failure. I think that makes sense.
well it depends about the scenario. you put on expectedfailures something that gets in your way now, so after if it works even better. of course you should get notified that the test is green while expected it to failed.
Now it leads to a UI problem where you have a failure that passes so when you click on it nothing happens: no debugger. And you can wonder why the hell do I have a failure when my tests pass.
So I think that this implementation of expectedFailures is a hack.
Adrian
On Apr 20, 2010, at 21:57 , Stéphane Ducasse wrote:
Hi
I tagged some tests as expected failures and I got a strange behavior. On the the tests which was passing was listed under the failures. When I renamed the method without updating the expected failures my bar was green. So expected failures really expect that the tests failed? We cannto have green tests in there?
Stef _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- www.tudorgirba.com
"Live like you mean it."
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project