sunit failures vs errors
Does anybody here distinguish failures and errors for real when doing TDD? I do not really see the point of separating them. Besides the complexity added to the testing framework it is a burden for newcomers. I personally just see: 1) test passes => green, ok 2) test doesn't pass => red, not ok Apparently that has been done in JUnit4 http://www.devx.com/Java/Article/31983/0/page/2
On Wed, Jul 10, 2013 at 2:46 AM, Camillo Bruni <camillobruni@gmail.com>wrote:
Does anybody here distinguish failures and errors for real when doing TDD?
I don't and I would like to see if both can be merged. -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill
On 2013-07-10, at 06:35, Damien Cassou <damien.cassou@gmail.com> wrote:
On Wed, Jul 10, 2013 at 2:46 AM, Camillo Bruni <camillobruni@gmail.com>wrote:
Does anybody here distinguish failures and errors for real when doing TDD?
I don't and I would like to see if both can be merged.
2 pomodoros for a prototype 1 day for a clean implementation ;)
I use them. I always fix the red tests first because they are mostly easier to fix and some of them lead to yellow tests. Maybe not that convincing but I feel the distinction between fail and error is good. Will think about it more. Am 10.07.2013 um 02:46 schrieb Camillo Bruni <camillobruni@gmail.com>:
Does anybody here distinguish failures and errors for real when doing TDD?
I do not really see the point of separating them. Besides the complexity added to the testing framework it is a burden for newcomers.
I personally just see: 1) test passes => green, ok 2) test doesn't pass => red, not ok
Apparently that has been done in JUnit4 http://www.devx.com/Java/Article/31983/0/page/2
On Wed, Jul 10, 2013 at 8:16 AM, Norbert Hartl <norbert@hartl.name> wrote:
I use them. I always fix the red tests first because they are mostly easier to fix and some of them lead to yellow tests
I agree with you. I tend to do that as well. -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error). Please do not do away with this distinction. Cheers, Doru On Jul 10, 2013, at 10:34 AM, Damien Cassou <damien.cassou@gmail.com> wrote:
On Wed, Jul 10, 2013 at 8:16 AM, Norbert Hartl <norbert@hartl.name> wrote: I use them. I always fix the red tests first because they are mostly easier to fix and some of them lead to yellow tests
I agree with you. I tend to do that as well.
-- Damien Cassou http://damiencassou.seasidehosting.st
"Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill
-- www.tudorgirba.com "Beauty is where we see it."
On 2013-07-10, at 13:32, Tudor Girba <tudor@tudorgirba.com> wrote:
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error).
In which sense? As a programmer I have to tackle them both the same way: start debugging. I really do not see the difference. In both cases the code does not behave as expected. One might even say that there is a global contract that the code does not raise exceptions. Tackling the failures first is purely a habit induced by the existing system that errors are worse than failures
On 10 July 2013 12:47, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2013-07-10, at 13:32, Tudor Girba <tudor@tudorgirba.com> wrote:
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error).
In which sense? As a programmer I have to tackle them both the same way: start debugging.
I really do not see the difference. In both cases the code does not behave as expected.
Yes, you do need to approach the problem in the same way - open the debugger - but I don't think that's sufficient reason to lose the distinction. Implementing a new feature also involves opening a debugger, but we keep the distinction between "writing new code" and "debugging existing code".
One might even say that there is a global contract that the code does not raise exceptions.
That last part's not true. I don't know what "global" means, but lots of code will explicitly have as its contract "halt and catch fire".
Tackling the failures first is purely a habit induced by the existing system that errors are worse than failures
Maybe, but it's still a useful distinction. "Oh, this feature broke" isn't the same as "hang on, that's a brand new bug". frank
Maybe we should not throw away the distinction, I am used to it as well. But what Camillo means, I think, is that in the end it does not matter. For example, if I have a unit test for some object's #printString both the case where is does no longer return what I expect and the case that it suddenly throws an error mean the same thing: the object's #printString is broken. Sven On 10 Jul 2013, at 13:55, Frank Shearar <frank.shearar@gmail.com> wrote:
On 10 July 2013 12:47, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2013-07-10, at 13:32, Tudor Girba <tudor@tudorgirba.com> wrote:
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error).
In which sense? As a programmer I have to tackle them both the same way: start debugging.
I really do not see the difference. In both cases the code does not behave as expected.
Yes, you do need to approach the problem in the same way - open the debugger - but I don't think that's sufficient reason to lose the distinction. Implementing a new feature also involves opening a debugger, but we keep the distinction between "writing new code" and "debugging existing code".
One might even say that there is a global contract that the code does not raise exceptions.
That last part's not true. I don't know what "global" means, but lots of code will explicitly have as its contract "halt and catch fire".
Tackling the failures first is purely a habit induced by the existing system that errors are worse than failures
Maybe, but it's still a useful distinction. "Oh, this feature broke" isn't the same as "hang on, that's a brand new bug".
frank
On 2013-07-10, at 14:09, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Maybe we should not throw away the distinction, I am used to it as well.
But what Camillo means, I think, is that in the end it does not matter.
For example, if I have a unit test for some object's #printString both the case where is does no longer return what I expect and the case that it suddenly throws an error mean the same thing: the object's #printString is broken.
Exactly, in the end you can perfectly deduce the type of failing test by the thrown error, no?
On 10 July 2013 13:18, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2013-07-10, at 14:09, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Maybe we should not throw away the distinction, I am used to it as well.
But what Camillo means, I think, is that in the end it does not matter.
For example, if I have a unit test for some object's #printString both the case where is does no longer return what I expect and the case that it suddenly throws an error mean the same thing: the object's #printString is broken.
Exactly, in the end you can perfectly deduce the type of failing test by the thrown error, no?
"In the end" isn't the same as being able to prioritise debugging. Most people think (judging by the participants in this thread at least) that unexpected test failures - errors - are more important than mere failures. frank
Error â defect in the human thought process made while trying to understand given information, solve problems, or to use methods and tools. In the context of software requirements specifications, an error is a basic misconception of the actual needs of a user or customer. Fault â concrete manifestation of an error within the software. One error may cause several faults, and various errors may cause identical faults. Failure â departure of the operational software system behavior from user expected requirements. A particular failure may be caused by several faults and some faults may never cause a failure. IEEE Std 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology, 1990. A systematic literature review to identify and classify software requirement errors. Gursimran Singh Walia, Jeffrey C. Carver. Information and Software Technology, 2009. El 10/07/2013 8:47, Camillo Bruni escribió:
On 2013-07-10, at 13:32, Tudor Girba <tudor@tudorgirba.com> wrote:
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error).
In which sense? As a programmer I have to tackle them both the same way: start debugging.
I really do not see the difference. In both cases the code does not behave as expected. One might even say that there is a global contract that the code does not raise exceptions.
Tackling the failures first is purely a habit induced by the existing system that errors are worse than failures .
Hi, On Jul 10, 2013, at 1:47 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2013-07-10, at 13:32, Tudor Girba <tudor@tudorgirba.com> wrote:
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error).
In which sense?
In the sense that they express two distinct classes of problems.
As a programmer I have to tackle them both the same way: start debugging.
Yes, but not all debugging is created equal :)
I really do not see the difference. In both cases the code does not behave as expected. One might even say that there is a global contract that the code does not raise exceptions.
Yes, but then what is the point of explicit assertions? :)
Tackling the failures first is purely a habit induced by the existing system that errors are worse than failures
Actually, it is the other way around :). Errors are much faster to fix than failures. Errors are often caused by bad initializations and you can see it rather fast when you see the exceptions. On the other hand, failures are often more sensitive to the deep semantics of your code and that is why they require more time to get fixed. Cheers, Doru -- www.tudorgirba.com "Presenting is storytelling."
On 2013-07-10, at 14:47, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
On Jul 10, 2013, at 1:47 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2013-07-10, at 13:32, Tudor Girba <tudor@tudorgirba.com> wrote:
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error).
In which sense?
In the sense that they express two distinct classes of problems.
As a programmer I have to tackle them both the same way: start debugging.
Yes, but not all debugging is created equal :)
I really do not see the difference. In both cases the code does not behave as expected. One might even say that there is a global contract that the code does not raise exceptions.
Yes, but then what is the point of explicit assertions? :)
Tackling the failures first is purely a habit induced by the existing system that errors are worse than failures
Actually, it is the other way around :). Errors are much faster to fix than failures. Errors are often caused by bad initializations and you can see it rather fast when you see the exceptions. On the other hand, failures are often more sensitive to the deep semantics of your code and that is why they require more time to get fixed.
That is fun, because everybody first takes the yellow ones because they come on top... So in the end the results should just be grouped by type of error I think.
Hi, On Jul 10, 2013, at 3:01 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2013-07-10, at 14:47, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
On Jul 10, 2013, at 1:47 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2013-07-10, at 13:32, Tudor Girba <tudor@tudorgirba.com> wrote:
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error).
In which sense?
In the sense that they express two distinct classes of problems.
As a programmer I have to tackle them both the same way: start debugging.
Yes, but not all debugging is created equal :)
I really do not see the difference. In both cases the code does not behave as expected. One might even say that there is a global contract that the code does not raise exceptions.
Yes, but then what is the point of explicit assertions? :)
Tackling the failures first is purely a habit induced by the existing system that errors are worse than failures
Actually, it is the other way around :). Errors are much faster to fix than failures. Errors are often caused by bad initializations and you can see it rather fast when you see the exceptions. On the other hand, failures are often more sensitive to the deep semantics of your code and that is why they require more time to get fixed.
That is fun, because everybody first takes the yellow ones because they come on top...
Really? I think people pick and choose (or at least try to) before fixing a test. So, any hints that help people with this choice is a good thing. This is why the premise of Phexample could bring so much value because besides the fluent interface, it builds a tree of tests.
So in the end the results should just be grouped by type of error I think.
That is actually a rather cool idea. I would be happy to join playing with such an idea. Cheers, Doru -- www.tudorgirba.com "Yesterday is a fact. Tomorrow is a possibility. Today is a challenge."
camillo frankly there are plenty of things that should be improved but for SUnit. First we should look at the main branch in VW because it changes since we forked it Second Smalltalk/X people probably enhance it too. Fourth there is also assess the frameworks of andres that has really nice extensions for benchmarking. And I vote against changing SUnit. If each time somebody ask a question in the mailing-lit we get a change we will never focus on the important items. Stef On Jul 10, 2013, at 1:47 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
On 2013-07-10, at 13:32, Tudor Girba <tudor@tudorgirba.com> wrote:
Precisely. There is a huge difference between invalidating a known contract (i.e., failure) and raising an unexpected error (i.e., error).
In which sense? As a programmer I have to tackle them both the same way: start debugging.
I really do not see the difference. In both cases the code does not behave as expected. One might even say that there is a global contract that the code does not raise exceptions.
Tackling the failures first is purely a habit induced by the existing system that errors are worse than failures
Oh, so programming in the debugger was a myth? I'm disappointed ;) 2013/7/10 Damien Cassou <damien.cassou@gmail.com>
On Wed, Jul 10, 2013 at 8:16 AM, Norbert Hartl <norbert@hartl.name> wrote:
I use them. I always fix the red tests first because they are mostly easier to fix and some of them lead to yellow tests
I agree with you. I tend to do that as well.
-- Damien Cassou http://damiencassou.seasidehosting.st
"Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill
Am 10.07.2013 um 15:33 schrieb Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>:
Oh, so programming in the debugger was a myth? I'm disappointed ;)
Well, I would say it is neither a myth nor has it to be forced. There is always more than one work mode. So you program only in the debugger? Did you uninstall all browsers, inspectors, etc.? Need to think about that approach! :) Norbert
2013/7/10 Damien Cassou <damien.cassou@gmail.com>
On Wed, Jul 10, 2013 at 8:16 AM, Norbert Hartl <norbert@hartl.name> wrote: I use them. I always fix the red tests first because they are mostly easier to fix and some of them lead to yellow tests
I agree with you. I tend to do that as well.
-- Damien Cassou http://damiencassou.seasidehosting.st
"Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill
Hi Camillo: On 10 Jul 2013, at 02:46, Camillo Bruni <camillobruni@gmail.com> wrote:
Does anybody here distinguish failures and errors for real when doing TDD?
To make your live a little more difficult, I actually introduced an additional type of failure on top of that: blue for still to be implemented. I liked that a lot for prototyping to see all my todos in the test runner in a different pane, and to be able to distinguish between unimplemented stuff, and stuff I just broke again. However, to your actual question, I have not noticed that difference in JUnit, so, I guess, I don't really care for the destination between errors and failures. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
On 2013-07-10, at 09:08, Stefan Marr <smalltalk@stefan-marr.de> wrote:
Hi Camillo:
On 10 Jul 2013, at 02:46, Camillo Bruni <camillobruni@gmail.com> wrote:
Does anybody here distinguish failures and errors for real when doing TDD?
To make your live a little more difficult, I actually introduced an additional type of failure on top of that: blue for still to be implemented.
that on the other hand I like :) because a different action is needed. Which exactly is not the case when it is red vs yellow. I have to debug in either case...
I liked that a lot for prototyping to see all my todos in the test runner in a different pane, and to be able to distinguish between unimplemented stuff, and stuff I just broke again.
However, to your actual question, I have not noticed that difference in JUnit, so, I guess, I don't really care for the destination between errors and failures.
On Jul 10, 2013, at 9:08 AM, Stefan Marr <smalltalk@stefan-marr.de> wrote:
Hi Camillo:
On 10 Jul 2013, at 02:46, Camillo Bruni <camillobruni@gmail.com> wrote:
Does anybody here distinguish failures and errors for real when doing TDD?
To make your live a little more difficult, I actually introduced an additional type of failure on top of that: blue for still to be implemented.
I liked that a lot for prototyping to see all my todos in the test runner in a different pane, and to be able to distinguish between unimplemented stuff, and stuff I just broke again.
+ 1
However, to your actual question, I have not noticed that difference in JUnit, so, I guess, I don't really care for the destination between errors and failures.
ah I did not know. I always thought it was the same.
Best regards Stefan
-- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
participants (10)
-
Camillo Bruni -
Damien Cassou -
Frank Shearar -
Hernán Morales Durand -
Nicolas Cellier -
Norbert Hartl -
Stefan Marr -
Stéphane Ducasse -
Sven Van Caekenberghe -
Tudor Girba