I'd like to better understand the semantics of "expected failures" in TestRunner. It seems to me that if you want to ensure that a certain operation fails, in a test you'd wrap it as follows... shouldFailed=false. [ self operationThatShouldFail ] on: Error do: [ shouldFailed := true ]. self assert: shouldFailed. So is tagging methods with pragma <expectedFailure> or in method #expectedFailures a temporary measure used to bypass a failing test when the judgment is that it is not critical to fix immediately? What is the process for tacking and resolving expected failures. It would give a warm fuzzy feeling if no expected failures are reported in TestRunner. Otherwise it leaves some residual uncertainty that something is wrong, even though a failure is "expected". cheers -ben
ben camillo once wrote a nice mail about shouldnot: Error. If I remember correctly. Stef On 25 Feb 2014, at 17:04, btc@openinworld.com wrote:
I'd like to better understand the semantics of "expected failures" in TestRunner. It seems to me that if you want to ensure that a certain operation fails, in a test you'd wrap it as follows...
shouldFailed=false. [ self operationThatShouldFail ] on: Error do: [ shouldFailed := true ]. self assert: shouldFailed.
So is tagging methods with pragma <expectedFailure> or in method #expectedFailures a temporary measure used to bypass a failing test when the judgment is that it is not critical to fix immediately? What is the process for tacking and resolving expected failures. It would give a warm fuzzy feeling if no expected failures are reported in TestRunner. Otherwise it leaves some residual uncertainty that something is wrong, even though a failure is "expected".
cheers -ben
Hi, I tend to wrap then like this: [ self operationThatShouldFail. self fail: 'why!'. ] on: Error do: [ "check that is the right error" ]
-----Mensaje original----- De: Pharo-dev [mailto:pharo-dev-bounces@lists.pharo.org] En nombre de Pharo4Stef Enviado el: Martes, 25 de Febrero de 2014 16:51 Para: Pharo Development List Asunto: Re: [Pharo-dev] expected failures
ben
camillo once wrote a nice mail about shouldnot: Error. If I remember correctly.
Stef On 25 Feb 2014, at 17:04, btc@openinworld.com wrote:
I'd like to better understand the semantics of "expected failures" in TestRunner. It seems to me that if you want to ensure that a certain operation fails, in a test you'd wrap it as follows...
shouldFailed=false. [ self operationThatShouldFail ] on: Error do: [ shouldFailed := true ]. self assert: shouldFailed.
So is tagging methods with pragma <expectedFailure> or in method #expectedFailures a temporary measure used to bypass a failing test when the judgment is that it is not critical to fix immediately? What is the process for tacking and resolving expected failures. It would give a warm fuzzy feeling if no expected failures are reported in TestRunner. Otherwise it leaves some residual uncertainty that something is wrong, even though a failure is "expected".
cheers -ben
On 2014-02-25, at 17:04, btc@openinworld.com wrote:
I'd like to better understand the semantics of "expected failures" in TestRunner. It seems to me that if you want to ensure that a certain operation fails, in a test you'd wrap it as follows...
shouldFailed=false. [ self operationThatShouldFail ] on: Error do: [ shouldFailed := true ]. self assert: shouldFailed.
I prefer: self should: [ self operationThatShouldFail ] raise: ASpecificError Otherwise you can just run the code itself: self operationThatShouldFail the test framework will take care of it and mark the test correctly as a failure.
On 2014-02-26, at 19:11, btc@openinworld.com wrote:
Camillo Bruni wrote:
On 2014-02-25, at 17:04, btc@openinworld.com wrote:
I'd like to better understand the semantics of "expected failures" in TestRunner. It seems to me that if you want to ensure that a certain operation fails, in a test you'd wrap it as follows...
shouldFailed=false. [ self operationThatShouldFail ] on: Error do: [ shouldFailed := true ]. self assert: shouldFailed.
I prefer:
self should: [ self operationThatShouldFail ] raise: ASpecificError
Otherwise you can just run the code itself:
self operationThatShouldFail
the test framework will take care of it and mark the test correctly as a failure.
Camillo, I don't quite follow your second case. Do you refer to using <expectedFailure> ? Doing that might be community convention, but "expected failures" for this makes me uncomfortable. It is not specific about what the failure should be, so the occurrence of non-expected failures is masked. I really like #should:raise since it is specific. It would be nice if as many <expectedFailure>s as possible were converted to #should:raise, with <epxectedFailure> being used only for fubared tests being prioritised to deal with later on, with the ultimate goal of having no <expected Failure>s in the image.
ah sorry, you are right, I didn't answer correctly. I don't use <expectedFailure> since I don't understand what it means :P I prefer #should:raise: too. And in the case to mark a test failing (aka skip) we can do the following: self skip: 'Because it does not work'
participants (4)
-
btc＠openinworld.com -
Camillo Bruni -
Emilio Oca -
Pharo4Stef