"Subject in SUnit Tests" â New blog post on all: objects all: theTime
Dear Pharo users and developers, In my series of blog posts on testing, I added a follow-up post <https://all-objects-all-the-time.st/#/blog/posts/12> about a dedicated name for the object under test in SUnit tests, the âsubject". Happy reading! Ciao, Koen
This makes the assumption that there is, most of the time, a single, unique, object that the test is clearly about. I hasten to admit that my unit testing practice needs a lot to be desired. But it has not been my experience that there *is* any such object much of the time. For example, to test a method, I will sometimes create two equal objects, manipulate one of them in the old trusted way, manipulate the other in the new way, and then see if the objects are equal again. Which then is *the* subject? The answer is neither, because what the test is *about* is not an object, but a method. Or again, I open a stream to a new file, create a complex object, persist it on the stream, close the stream, open another stream reading from the file, reconstitute an object from it, and then check that the two objects are isomorphic. Which is *the* subject? Neither, because what I am testing is the encoding and decoding methods, not the objects. The key paragraph is this one: <quote> One could argue that there is not always a single object under test. In some situations that is correct, for instance when testing the interaction between objects. But in my experience, most unit tests have a single object under test. </quote> And it is *this* that we really need a blog post on. I am very ready to believe that you are a much better tester than I am. (That wouldn't be hard.) But *something* interesting is going on when I think in terms of testing classes and methods to the point where I find the idea of testing a particular object almost incomprehensible, yet you think this is the norm. I take some comfort from the fact that the Gherkin language in Cucumber (and why oh which has nobody bothered to write down a proper grammar, let alone even an informal semantics, for it? How am I supposed to know what is and what is not legal Gherkin?) goes out of its way to conceal the existence of subjects, should they exist, from test reviewers, and calls what a test is about a "feature". Here are two tests for the Quaternion class: testMultiplicationByScalarCommutes sampleExactScalars do: [:x | sampleExactQuaternions do: [:y | self assert: (x*y) = (y*x)]]. testMultiplicationOfPowersCommutes sampleExactQuaternions do: [:x | |y z| y := x squared. z := x cubed. self assert: (y*z) = (z*y)]. These two methods test important properties of the #* method. Here's another one, part of testing the test data.. testExactScalarsAreExactScalars sampleExactScalars do: [:each | self assert: each isScalar. self assert: each isExact]. This tests the *methods* #isScalar and #isExact, and indirectly the code that generates the test data. It's not about any particular object. Nor is it about *one* method. It's very hard to write a test case that tests only one method. In fact my own rather embarrassing experience is that when a test case of mine failed it quite often wasn't the method I thought I was testing that was wrong. Returning to the persistence example, it's not testing #saveOn: (object saveOn: byteStream) or #restore (byteStream restore). It's testing that the two methods are properly related. It would be wrong to call it #testSaveOn: and it would be wrong to call it #testRestore. There's a distinction here: it *is* possible to write methods to test #saveOn: by itself, by writing various objects and checking that you get the right bytes. But if the external representation is changed (and I need to change it), the tests for #saveOn: and #restore as such will have to be rewritten, while the tests for #saveOn: and #restore *together* will not. Cucumber offers an insight here, I think. A scenario in Gherkin is not a test case for an object. Nor is it a test case for a method. It is a test case for a composite *behaviour* of a system, at some contextually appropriate level of detail. Brian Marick had some lovely stickers. One of them reads "An example would be handy about now." I note that your blog post *did* provide an example of testing Date. In which none of your test cases used 'subject'. I entirely agree with you that the three methods have good intention-revealing names. What we actually need is some examples where 'subject' pays off. On Mon, 20 Jan 2025 at 04:04, Koen De Hondt <koen@all-objects-all-the-time.st> wrote:
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post about a dedicated name for the object under test in SUnit tests, the âsubject".
Happy reading!
Ciao, Koen
Folks, I really appreciate (and learn from) these types of conversations - especially on testing where I struggle for consistency and completeness. I tend to agree that my testing focuses on methods, however the majority of my âfailuresâ revolve around insufficient object values that eventually have my tests failing - Iâve defaulted to generating âdataâ as randomly as I can early on, however non-realistic, and running tests a few thousand times to point out assumptions I make in my methods. Thanks for discussions! Russ Whaley whaley.russ@gmail.com On Thu, Jan 23, 2025 at 5:02â¯AM Richard O'Keefe <raoknz@gmail.com> wrote:
This makes the assumption that there is, most of the time, a single, unique, object that the test is clearly about. I hasten to admit that my unit testing practice needs a lot to be desired. But it has not been my experience that there *is* any such object much of the time. For example, to test a method, I will sometimes create two equal objects, manipulate one of them in the old trusted way, manipulate the other in the new way, and then see if the objects are equal again. Which then is *the* subject? The answer is neither, because what the test is *about* is not an object, but a method. Or again, I open a stream to a new file, create a complex object, persist it on the stream, close the stream, open another stream reading from the file, reconstitute an object from it, and then check that the two objects are isomorphic. Which is *the* subject? Neither, because what I am testing is the encoding and decoding methods, not the objects.
The key paragraph is this one: <quote> One could argue that there is not always a single object under test. In some situations that is correct, for instance when testing the interaction between objects. But in my experience, most unit tests have a single object under test. </quote>
And it is *this* that we really need a blog post on. I am very ready to believe that you are a much better tester than I am. (That wouldn't be hard.) But *something* interesting is going on when I think in terms of testing classes and methods to the point where I find the idea of testing a particular object almost incomprehensible, yet you think this is the norm. I take some comfort from the fact that the Gherkin language in Cucumber (and why oh which has nobody bothered to write down a proper grammar, let alone even an informal semantics, for it? How am I supposed to know what is and what is not legal Gherkin?) goes out of its way to conceal the existence of subjects, should they exist, from test reviewers, and calls what a test is about a "feature".
Here are two tests for the Quaternion class: testMultiplicationByScalarCommutes sampleExactScalars do: [:x | sampleExactQuaternions do: [:y | self assert: (x*y) = (y*x)]].
testMultiplicationOfPowersCommutes sampleExactQuaternions do: [:x | |y z| y := x squared. z := x cubed. self assert: (y*z) = (z*y)].
These two methods test important properties of the #* method.
Here's another one, part of testing the test data..
testExactScalarsAreExactScalars sampleExactScalars do: [:each | self assert: each isScalar. self assert: each isExact].
This tests the *methods* #isScalar and #isExact, and indirectly the code that generates the test data. It's not about any particular object. Nor is it about *one* method. It's very hard to write a test case that tests only one method. In fact my own rather embarrassing experience is that when a test case of mine failed it quite often wasn't the method I thought I was testing that was wrong. Returning to the persistence example, it's not testing #saveOn: (object saveOn: byteStream) or #restore (byteStream restore). It's testing that the two methods are properly related. It would be wrong to call it #testSaveOn: and it would be wrong to call it #testRestore. There's a distinction here: it *is* possible to write methods to test #saveOn: by itself, by writing various objects and checking that you get the right bytes. But if the external representation is changed (and I need to change it), the tests for #saveOn: and #restore as such will have to be rewritten, while the tests for #saveOn: and #restore *together* will not.
Cucumber offers an insight here, I think. A scenario in Gherkin is not a test case for an object. Nor is it a test case for a method. It is a test case for a composite *behaviour* of a system, at some contextually appropriate level of detail.
Brian Marick had some lovely stickers. One of them reads "An example would be handy about now."
I note that your blog post *did* provide an example of testing Date. In which none of your test cases used 'subject'. I entirely agree with you that the three methods have good intention-revealing names. What we actually need is some examples where 'subject' pays off.
On Mon, 20 Jan 2025 at 04:04, Koen De Hondt <koen@all-objects-all-the-time.st> wrote:
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post about a
dedicated name for the object under test in SUnit tests, the âsubject".
Happy reading!
Ciao, Koen
I too like to use random data in test cases but you also have to use edge cases that random data tend to avoid. It is painful to recall the millions of successful tests of my Dictionary implementation that failed to detect that "Dictionary new: 0" created a non-functional object. ((self pvtHash: key) \\ capacity) + 1" did not work well when capacity was 0. 0 size, 0 capacity, 0 room left, any edge case. Conversely, you have to avoid special cases sometimes: when testing x ./ y and y \ x for Quaternions, you want to make sure y isZero not. One thing I would like some advice on is testing traits. I understand about testing classes and about testing methods in classes (though I am never thorough enough), but how do you test a method that isn't bound to a particular context, where the thing that reveals a mistake may be a class it will be combined with in the future that doesn't exist yet? What is considered good practice? On Tue, 4 Feb 2025 at 05:25, Russ Whaley <whaley.russ@gmail.com> wrote:
Folks, I really appreciate (and learn from) these types of conversations - especially on testing where I struggle for consistency and completeness.
I tend to agree that my testing focuses on methods, however the majority of my âfailuresâ revolve around insufficient object values that eventually have my tests failing - Iâve defaulted to generating âdataâ as randomly as I can early on, however non-realistic, and running tests a few thousand times to point out assumptions I make in my methods.
Thanks for discussions!
Russ Whaley whaley.russ@gmail.com
On Thu, Jan 23, 2025 at 5:02â¯AM Richard O'Keefe <raoknz@gmail.com> wrote:
This makes the assumption that there is, most of the time, a single, unique, object that the test is clearly about. I hasten to admit that my unit testing practice needs a lot to be desired. But it has not been my experience that there *is* any such object much of the time. For example, to test a method, I will sometimes create two equal objects, manipulate one of them in the old trusted way, manipulate the other in the new way, and then see if the objects are equal again. Which then is *the* subject? The answer is neither, because what the test is *about* is not an object, but a method. Or again, I open a stream to a new file, create a complex object, persist it on the stream, close the stream, open another stream reading from the file, reconstitute an object from it, and then check that the two objects are isomorphic. Which is *the* subject? Neither, because what I am testing is the encoding and decoding methods, not the objects.
The key paragraph is this one: <quote> One could argue that there is not always a single object under test. In some situations that is correct, for instance when testing the interaction between objects. But in my experience, most unit tests have a single object under test. </quote>
And it is *this* that we really need a blog post on. I am very ready to believe that you are a much better tester than I am. (That wouldn't be hard.) But *something* interesting is going on when I think in terms of testing classes and methods to the point where I find the idea of testing a particular object almost incomprehensible, yet you think this is the norm. I take some comfort from the fact that the Gherkin language in Cucumber (and why oh which has nobody bothered to write down a proper grammar, let alone even an informal semantics, for it? How am I supposed to know what is and what is not legal Gherkin?) goes out of its way to conceal the existence of subjects, should they exist, from test reviewers, and calls what a test is about a "feature".
Here are two tests for the Quaternion class: testMultiplicationByScalarCommutes sampleExactScalars do: [:x | sampleExactQuaternions do: [:y | self assert: (x*y) = (y*x)]].
testMultiplicationOfPowersCommutes sampleExactQuaternions do: [:x | |y z| y := x squared. z := x cubed. self assert: (y*z) = (z*y)].
These two methods test important properties of the #* method.
Here's another one, part of testing the test data..
testExactScalarsAreExactScalars sampleExactScalars do: [:each | self assert: each isScalar. self assert: each isExact].
This tests the *methods* #isScalar and #isExact, and indirectly the code that generates the test data. It's not about any particular object. Nor is it about *one* method. It's very hard to write a test case that tests only one method. In fact my own rather embarrassing experience is that when a test case of mine failed it quite often wasn't the method I thought I was testing that was wrong. Returning to the persistence example, it's not testing #saveOn: (object saveOn: byteStream) or #restore (byteStream restore). It's testing that the two methods are properly related. It would be wrong to call it #testSaveOn: and it would be wrong to call it #testRestore. There's a distinction here: it *is* possible to write methods to test #saveOn: by itself, by writing various objects and checking that you get the right bytes. But if the external representation is changed (and I need to change it), the tests for #saveOn: and #restore as such will have to be rewritten, while the tests for #saveOn: and #restore *together* will not.
Cucumber offers an insight here, I think. A scenario in Gherkin is not a test case for an object. Nor is it a test case for a method. It is a test case for a composite *behaviour* of a system, at some contextually appropriate level of detail.
Brian Marick had some lovely stickers. One of them reads "An example would be handy about now."
I note that your blog post *did* provide an example of testing Date. In which none of your test cases used 'subject'. I entirely agree with you that the three methods have good intention-revealing names. What we actually need is some examples where 'subject' pays off.
On Mon, 20 Jan 2025 at 04:04, Koen De Hondt <koen@all-objects-all-the-time.st> wrote:
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post about a dedicated name for the object under test in SUnit tests, the âsubject".
Happy reading!
Ciao, Koen
Agreed. A little knowledge of how it is to be used can go a long way. I seem to have a mental block on creating traits. I am sure my code and applications would benefit - but I just canât seem to understand what they are and how to design/implement them. Russ Whaley whaley.russ@gmail.com On Tue, Feb 4, 2025 at 2:23â¯AM Richard O'Keefe <raoknz@gmail.com> wrote:
I too like to use random data in test cases but you also have to use edge cases that random data tend to avoid. It is painful to recall the millions of successful tests of my Dictionary implementation that failed to detect that "Dictionary new: 0" created a non-functional object. ((self pvtHash: key) \\ capacity) + 1" did not work well when capacity was 0. 0 size, 0 capacity, 0 room left, any edge case. Conversely, you have to avoid special cases sometimes: when testing x ./ y and y \ x for Quaternions, you want to make sure y isZero not.
One thing I would like some advice on is testing traits. I understand about testing classes and about testing methods in classes (though I am never thorough enough), but how do you test a method that isn't bound to a particular context, where the thing that reveals a mistake may be a class it will be combined with in the future that doesn't exist yet? What is considered good practice?
On Tue, 4 Feb 2025 at 05:25, Russ Whaley <whaley.russ@gmail.com> wrote:
Folks, I really appreciate (and learn from) these types of conversations
- especially on testing where I struggle for consistency and completeness.
I tend to agree that my testing focuses on methods, however the majority
of my âfailuresâ revolve around insufficient object values that eventually have my tests failing - Iâve defaulted to generating âdataâ as randomly as I can early on, however non-realistic, and running tests a few thousand times to point out assumptions I make in my methods.
Thanks for discussions!
Russ Whaley whaley.russ@gmail.com
On Thu, Jan 23, 2025 at 5:02â¯AM Richard O'Keefe <raoknz@gmail.com>
wrote:
This makes the assumption that there is, most of the time, a single, unique, object that the test is clearly about. I hasten to admit that my unit testing practice needs a lot to be
desired.
But it has not been my experience that there *is* any such object much of the time. For example, to test a method, I will sometimes create two equal objects, manipulate one of them in the old trusted way, manipulate the other in the new way, and then see if the objects are equal again. Which then is *the* subject? The answer is neither, because what the test is *about* is not an object, but a method. Or again, I open a stream to a new file, create a complex object, persist it on the stream, close the stream, open another stream reading from the file, reconstitute an object from it, and then check that the two objects are isomorphic. Which is *the* subject? Neither, because what I am testing is the encoding and decoding methods, not the objects.
The key paragraph is this one: <quote> One could argue that there is not always a single object under test. In some situations that is correct, for instance when testing the interaction between objects. But in my experience, most unit tests have a single object under test. </quote>
And it is *this* that we really need a blog post on. I am very ready to believe that you are a much better tester than I am. (That wouldn't be hard.) But *something* interesting is going on when I think in terms of testing classes and methods to the point where I find the idea of testing a particular object almost incomprehensible, yet you think this is the norm. I take some comfort from the fact that the Gherkin language in Cucumber (and why oh which has nobody bothered to write down a proper grammar, let alone even an informal semantics, for it? How am I supposed to know what is and what is not legal Gherkin?) goes out of its way to conceal the existence of subjects, should they exist, from test reviewers, and calls what a test is about a "feature".
Here are two tests for the Quaternion class: testMultiplicationByScalarCommutes sampleExactScalars do: [:x | sampleExactQuaternions do: [:y | self assert: (x*y) = (y*x)]].
testMultiplicationOfPowersCommutes sampleExactQuaternions do: [:x | |y z| y := x squared. z := x cubed. self assert: (y*z) = (z*y)].
These two methods test important properties of the #* method.
Here's another one, part of testing the test data..
testExactScalarsAreExactScalars sampleExactScalars do: [:each | self assert: each isScalar. self assert: each isExact].
This tests the *methods* #isScalar and #isExact, and indirectly the code that generates the test data. It's not about any particular object. Nor is it about *one* method. It's very hard to write a test case that tests only one method. In fact my own rather embarrassing experience is that when a test case of mine failed it quite often wasn't the method I thought I was testing that was wrong. Returning to the persistence example, it's not testing #saveOn: (object saveOn: byteStream) or #restore (byteStream restore). It's testing that the two methods are properly related. It would be wrong to call it #testSaveOn: and it would be wrong to call it #testRestore. There's a distinction here: it *is* possible to write methods to test #saveOn: by itself, by writing various objects and checking that you get the right bytes. But if the external representation is changed (and I need to change it), the tests for #saveOn: and #restore as such will have to be rewritten, while the tests for #saveOn: and #restore *together* will not.
Cucumber offers an insight here, I think. A scenario in Gherkin is not a test case for an object. Nor is it a test case for a method. It is a test case for a composite *behaviour* of a system, at some contextually appropriate level of detail.
Brian Marick had some lovely stickers. One of them reads "An example would be handy about now."
I note that your blog post *did* provide an example of testing Date. In which none of your test cases used 'subject'. I entirely agree with you that the three methods have good intention-revealing names. What we actually need is some examples where 'subject' pays off.
On Mon, 20 Jan 2025 at 04:04, Koen De Hondt <koen@all-objects-all-the-time.st> wrote:
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post about
a dedicated name for the object under test in SUnit tests, the âsubject".
Happy reading!
Ciao, Koen
participants (3)
-
Koen De Hondt -
Richard O'Keefe -
Russ Whaley