KD
Koen De Hondt
Sun, Jan 19, 2025 3:03 PM
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post https://all-objects-all-the-time.st/#/blog/posts/12 about a dedicated name for the object under test in SUnit tests, the “subject".
Happy reading!
Ciao,
Koen
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post <https://all-objects-all-the-time.st/#/blog/posts/12> about a dedicated name for the object under test in SUnit tests, the “subject".
Happy reading!
Ciao,
Koen
RO
Richard O'Keefe
Thu, Jan 23, 2025 10:02 AM
This makes the assumption that there is, most of the time, a single,
unique, object that the test is clearly about.
I hasten to admit that my unit testing practice needs a lot to be desired.
But it has not been my experience that there is any such object much
of the time.
For example, to test a method, I will sometimes create two equal
objects, manipulate one of them in the
old trusted way, manipulate the other in the new way, and then see if
the objects are equal again.
Which then is the subject?
The answer is neither, because what the test is about is not an
object, but a method.
Or again, I open a stream to a new file, create a complex object,
persist it on the stream, close
the stream, open another stream reading from the file, reconstitute an
object from it, and then
check that the two objects are isomorphic. Which is the subject?
Neither, because what I am testing is the encoding and decoding
methods, not the objects.
The key paragraph is this one:
<quote>
One could argue that there is not always a single object under test.
In some situations that is correct, for instance when testing the
interaction between objects. But in my experience, most unit tests
have a single object under test.
</quote>
And it is this that we really need a blog post on. I am very ready
to believe that you are a much better tester than I am.
(That wouldn't be hard.) But something interesting is going on when
I think in terms of testing classes and methods to
the point where I find the idea of testing a particular object almost
incomprehensible, yet you think this is the norm.
I take some comfort from the fact that the Gherkin language in
Cucumber (and why oh which has nobody bothered to
write down a proper grammar, let alone even an informal semantics, for
it? How am I supposed to know what is and what
is not legal Gherkin?) goes out of its way to conceal the existence
of subjects, should they exist, from test reviewers, and
calls what a test is about a "feature".
Here are two tests for the Quaternion class:
testMultiplicationByScalarCommutes
sampleExactScalars do: [:x |
sampleExactQuaternions do: [:y |
self assert: (xy) = (yx)]].
testMultiplicationOfPowersCommutes
sampleExactQuaternions do: [:x |
|y z|
y := x squared.
z := x cubed.
self assert: (y*z) = (z*y)].
These two methods test important properties of the #* method.
Here's another one, part of testing the test data..
testExactScalarsAreExactScalars
sampleExactScalars do: [:each |
self assert: each isScalar.
self assert: each isExact].
This tests the methods #isScalar and #isExact, and indirectly the
code that generates the test data.
It's not about any particular object. Nor is it about one method.
It's very hard to write a test case that
tests only one method. In fact my own rather embarrassing experience
is that when a test case of mine
failed it quite often wasn't the method I thought I was testing that
was wrong. Returning to the
persistence example, it's not testing #saveOn: (object saveOn:
byteStream) or #restore (byteStream restore).
It's testing that the two methods are properly related. It would be
wrong to call it #testSaveOn: and it
would be wrong to call it #testRestore. There's a distinction here:
it is possible to write methods to test
#saveOn: by itself, by writing various objects and checking that you
get the right bytes. But if the external
representation is changed (and I need to change it), the tests for
#saveOn: and #restore as such will have
to be rewritten, while the tests for #saveOn: and #restore together will not.
Cucumber offers an insight here, I think. A scenario in Gherkin is
not a test case for an object.
Nor is it a test case for a method. It is a test case for a composite
behaviour of a system,
at some contextually appropriate level of detail.
Brian Marick had some lovely stickers. One of them reads "An example
would be handy about now."
I note that your blog post did provide an example of testing Date.
In which none of your test cases used 'subject'.
I entirely agree with you that the three methods have good
intention-revealing names.
What we actually need is some examples where 'subject' pays off.
On Mon, 20 Jan 2025 at 04:04, Koen De Hondt
koen@all-objects-all-the-time.st wrote:
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post about a dedicated name for the object under test in SUnit tests, the “subject".
Happy reading!
Ciao,
Koen
This makes the assumption that there is, most of the time, a single,
unique, object that the test is clearly about.
I hasten to admit that my unit testing practice needs a lot to be desired.
But it has not been my experience that there *is* any such object much
of the time.
For example, to test a method, I will sometimes create two equal
objects, manipulate one of them in the
old trusted way, manipulate the other in the new way, and then see if
the objects are equal again.
Which then is *the* subject?
The answer is neither, because what the test is *about* is not an
object, but a method.
Or again, I open a stream to a new file, create a complex object,
persist it on the stream, close
the stream, open another stream reading from the file, reconstitute an
object from it, and then
check that the two objects are isomorphic. Which is *the* subject?
Neither, because what I am testing is the encoding and decoding
methods, not the objects.
The key paragraph is this one:
<quote>
One could argue that there is not always a single object under test.
In some situations that is correct, for instance when testing the
interaction between objects. But in my experience, most unit tests
have a single object under test.
</quote>
And it is *this* that we really need a blog post on. I am very ready
to believe that you are a much better tester than I am.
(That wouldn't be hard.) But *something* interesting is going on when
I think in terms of testing classes and methods to
the point where I find the idea of testing a particular object almost
incomprehensible, yet you think this is the norm.
I take some comfort from the fact that the Gherkin language in
Cucumber (and why oh which has nobody bothered to
write down a proper grammar, let alone even an informal semantics, for
it? How am I supposed to know what is and what
is not legal Gherkin?) goes out of its way to conceal the existence
of subjects, should they exist, from test reviewers, and
calls what a test is about a "feature".
Here are two tests for the Quaternion class:
testMultiplicationByScalarCommutes
sampleExactScalars do: [:x |
sampleExactQuaternions do: [:y |
self assert: (x*y) = (y*x)]].
testMultiplicationOfPowersCommutes
sampleExactQuaternions do: [:x |
|y z|
y := x squared.
z := x cubed.
self assert: (y*z) = (z*y)].
These two methods test important properties of the #* method.
Here's another one, part of testing the test data..
testExactScalarsAreExactScalars
sampleExactScalars do: [:each |
self assert: each isScalar.
self assert: each isExact].
This tests the *methods* #isScalar and #isExact, and indirectly the
code that generates the test data.
It's not about any particular object. Nor is it about *one* method.
It's very hard to write a test case that
tests only one method. In fact my own rather embarrassing experience
is that when a test case of mine
failed it quite often wasn't the method I thought I was testing that
was wrong. Returning to the
persistence example, it's not testing #saveOn: (object saveOn:
byteStream) or #restore (byteStream restore).
It's testing that the two methods are properly related. It would be
wrong to call it #testSaveOn: and it
would be wrong to call it #testRestore. There's a distinction here:
it *is* possible to write methods to test
#saveOn: by itself, by writing various objects and checking that you
get the right bytes. But if the external
representation is changed (and I need to change it), the tests for
#saveOn: and #restore as such will have
to be rewritten, while the tests for #saveOn: and #restore *together* will not.
Cucumber offers an insight here, I think. A scenario in Gherkin is
not a test case for an object.
Nor is it a test case for a method. It is a test case for a composite
*behaviour* of a system,
at some contextually appropriate level of detail.
Brian Marick had some lovely stickers. One of them reads "An example
would be handy about now."
I note that your blog post *did* provide an example of testing Date.
In which none of your test cases used 'subject'.
I entirely agree with you that the three methods have good
intention-revealing names.
What we actually need is some examples where 'subject' pays off.
On Mon, 20 Jan 2025 at 04:04, Koen De Hondt
<koen@all-objects-all-the-time.st> wrote:
>
> Dear Pharo users and developers,
>
> In my series of blog posts on testing, I added a follow-up post about a dedicated name for the object under test in SUnit tests, the “subject".
>
> Happy reading!
>
> Ciao,
> Koen
>
RW
Russ Whaley
Mon, Feb 3, 2025 4:24 PM
Folks, I really appreciate (and learn from) these types of conversations -
especially on testing where I struggle for consistency and completeness.
I tend to agree that my testing focuses on methods, however the majority of
my ‘failures’ revolve around insufficient object values that eventually
have my tests failing - I’ve defaulted to generating ‘data’ as randomly as
I can early on, however non-realistic, and running tests a few thousand
times to point out assumptions I make in my methods.
Thanks for discussions!
Russ Whaley
whaley.russ@gmail.com
On Thu, Jan 23, 2025 at 5:02 AM Richard O'Keefe raoknz@gmail.com wrote:
This makes the assumption that there is, most of the time, a single,
unique, object that the test is clearly about.
I hasten to admit that my unit testing practice needs a lot to be desired.
But it has not been my experience that there is any such object much
of the time.
For example, to test a method, I will sometimes create two equal
objects, manipulate one of them in the
old trusted way, manipulate the other in the new way, and then see if
the objects are equal again.
Which then is the subject?
The answer is neither, because what the test is about is not an
object, but a method.
Or again, I open a stream to a new file, create a complex object,
persist it on the stream, close
the stream, open another stream reading from the file, reconstitute an
object from it, and then
check that the two objects are isomorphic. Which is the subject?
Neither, because what I am testing is the encoding and decoding
methods, not the objects.
The key paragraph is this one:
<quote>
One could argue that there is not always a single object under test.
In some situations that is correct, for instance when testing the
interaction between objects. But in my experience, most unit tests
have a single object under test.
</quote>
And it is this that we really need a blog post on. I am very ready
to believe that you are a much better tester than I am.
(That wouldn't be hard.) But something interesting is going on when
I think in terms of testing classes and methods to
the point where I find the idea of testing a particular object almost
incomprehensible, yet you think this is the norm.
I take some comfort from the fact that the Gherkin language in
Cucumber (and why oh which has nobody bothered to
write down a proper grammar, let alone even an informal semantics, for
it? How am I supposed to know what is and what
is not legal Gherkin?) goes out of its way to conceal the existence
of subjects, should they exist, from test reviewers, and
calls what a test is about a "feature".
Here are two tests for the Quaternion class:
testMultiplicationByScalarCommutes
sampleExactScalars do: [:x |
sampleExactQuaternions do: [:y |
self assert: (xy) = (yx)]].
testMultiplicationOfPowersCommutes
sampleExactQuaternions do: [:x |
|y z|
y := x squared.
z := x cubed.
self assert: (y*z) = (z*y)].
These two methods test important properties of the #* method.
Here's another one, part of testing the test data..
testExactScalarsAreExactScalars
sampleExactScalars do: [:each |
self assert: each isScalar.
self assert: each isExact].
This tests the methods #isScalar and #isExact, and indirectly the
code that generates the test data.
It's not about any particular object. Nor is it about one method.
It's very hard to write a test case that
tests only one method. In fact my own rather embarrassing experience
is that when a test case of mine
failed it quite often wasn't the method I thought I was testing that
was wrong. Returning to the
persistence example, it's not testing #saveOn: (object saveOn:
byteStream) or #restore (byteStream restore).
It's testing that the two methods are properly related. It would be
wrong to call it #testSaveOn: and it
would be wrong to call it #testRestore. There's a distinction here:
it is possible to write methods to test
#saveOn: by itself, by writing various objects and checking that you
get the right bytes. But if the external
representation is changed (and I need to change it), the tests for
#saveOn: and #restore as such will have
to be rewritten, while the tests for #saveOn: and #restore together will
not.
Cucumber offers an insight here, I think. A scenario in Gherkin is
not a test case for an object.
Nor is it a test case for a method. It is a test case for a composite
behaviour of a system,
at some contextually appropriate level of detail.
Brian Marick had some lovely stickers. One of them reads "An example
would be handy about now."
I note that your blog post did provide an example of testing Date.
In which none of your test cases used 'subject'.
I entirely agree with you that the three methods have good
intention-revealing names.
What we actually need is some examples where 'subject' pays off.
On Mon, 20 Jan 2025 at 04:04, Koen De Hondt
koen@all-objects-all-the-time.st wrote:
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post about a
dedicated name for the object under test in SUnit tests, the “subject".
Happy reading!
Ciao,
Koen
Folks, I really appreciate (and learn from) these types of conversations -
especially on testing where I struggle for consistency and completeness.
I tend to agree that my testing focuses on methods, however the majority of
my ‘failures’ revolve around insufficient object values that eventually
have my tests failing - I’ve defaulted to generating ‘data’ as randomly as
I can early on, however non-realistic, and running tests a few thousand
times to point out assumptions I make in my methods.
Thanks for discussions!
Russ Whaley
whaley.russ@gmail.com
On Thu, Jan 23, 2025 at 5:02 AM Richard O'Keefe <raoknz@gmail.com> wrote:
> This makes the assumption that there is, most of the time, a single,
> unique, object that the test is clearly about.
> I hasten to admit that my unit testing practice needs a lot to be desired.
> But it has not been my experience that there *is* any such object much
> of the time.
> For example, to test a method, I will sometimes create two equal
> objects, manipulate one of them in the
> old trusted way, manipulate the other in the new way, and then see if
> the objects are equal again.
> Which then is *the* subject?
> The answer is neither, because what the test is *about* is not an
> object, but a method.
> Or again, I open a stream to a new file, create a complex object,
> persist it on the stream, close
> the stream, open another stream reading from the file, reconstitute an
> object from it, and then
> check that the two objects are isomorphic. Which is *the* subject?
> Neither, because what I am testing is the encoding and decoding
> methods, not the objects.
>
> The key paragraph is this one:
> <quote>
> One could argue that there is not always a single object under test.
> In some situations that is correct, for instance when testing the
> interaction between objects. But in my experience, most unit tests
> have a single object under test.
> </quote>
>
> And it is *this* that we really need a blog post on. I am very ready
> to believe that you are a much better tester than I am.
> (That wouldn't be hard.) But *something* interesting is going on when
> I think in terms of testing classes and methods to
> the point where I find the idea of testing a particular object almost
> incomprehensible, yet you think this is the norm.
> I take some comfort from the fact that the Gherkin language in
> Cucumber (and why oh which has nobody bothered to
> write down a proper grammar, let alone even an informal semantics, for
> it? How am I supposed to know what is and what
> is not legal Gherkin?) goes out of its way to conceal the existence
> of subjects, should they exist, from test reviewers, and
> calls what a test is about a "feature".
>
> Here are two tests for the Quaternion class:
> testMultiplicationByScalarCommutes
> sampleExactScalars do: [:x |
> sampleExactQuaternions do: [:y |
> self assert: (x*y) = (y*x)]].
>
> testMultiplicationOfPowersCommutes
> sampleExactQuaternions do: [:x |
> |y z|
> y := x squared.
> z := x cubed.
> self assert: (y*z) = (z*y)].
>
> These two methods test important properties of the #* method.
>
> Here's another one, part of testing the test data..
>
> testExactScalarsAreExactScalars
> sampleExactScalars do: [:each |
> self assert: each isScalar.
> self assert: each isExact].
>
> This tests the *methods* #isScalar and #isExact, and indirectly the
> code that generates the test data.
> It's not about any particular object. Nor is it about *one* method.
> It's very hard to write a test case that
> tests only one method. In fact my own rather embarrassing experience
> is that when a test case of mine
> failed it quite often wasn't the method I thought I was testing that
> was wrong. Returning to the
> persistence example, it's not testing #saveOn: (object saveOn:
> byteStream) or #restore (byteStream restore).
> It's testing that the two methods are properly related. It would be
> wrong to call it #testSaveOn: and it
> would be wrong to call it #testRestore. There's a distinction here:
> it *is* possible to write methods to test
> #saveOn: by itself, by writing various objects and checking that you
> get the right bytes. But if the external
> representation is changed (and I need to change it), the tests for
> #saveOn: and #restore as such will have
> to be rewritten, while the tests for #saveOn: and #restore *together* will
> not.
>
> Cucumber offers an insight here, I think. A scenario in Gherkin is
> not a test case for an object.
> Nor is it a test case for a method. It is a test case for a composite
> *behaviour* of a system,
> at some contextually appropriate level of detail.
>
> Brian Marick had some lovely stickers. One of them reads "An example
> would be handy about now."
>
> I note that your blog post *did* provide an example of testing Date.
> In which none of your test cases used 'subject'.
> I entirely agree with you that the three methods have good
> intention-revealing names.
> What we actually need is some examples where 'subject' pays off.
>
>
>
> On Mon, 20 Jan 2025 at 04:04, Koen De Hondt
> <koen@all-objects-all-the-time.st> wrote:
> >
> > Dear Pharo users and developers,
> >
> > In my series of blog posts on testing, I added a follow-up post about a
> dedicated name for the object under test in SUnit tests, the “subject".
> >
> > Happy reading!
> >
> > Ciao,
> > Koen
> >
>
RO
Richard O'Keefe
Tue, Feb 4, 2025 7:23 AM
I too like to use random data in test cases but you also have to use
edge cases that random data tend to avoid.
It is painful to recall the millions of successful tests of my
Dictionary implementation that failed to detect that
"Dictionary new: 0" created a non-functional object. ((self pvtHash:
key) \ capacity) + 1" did not work well when
capacity was 0. 0 size, 0 capacity, 0 room left, any edge case.
Conversely, you have to avoid special cases
sometimes: when testing x ./ y and y \ x for Quaternions, you want to
make sure y isZero not.
One thing I would like some advice on is testing traits. I understand
about testing classes and about testing
methods in classes (though I am never thorough enough), but how do you
test a method that isn't bound to
a particular context, where the thing that reveals a mistake may be a
class it will be combined with in the
future that doesn't exist yet? What is considered good practice?
On Tue, 4 Feb 2025 at 05:25, Russ Whaley whaley.russ@gmail.com wrote:
Folks, I really appreciate (and learn from) these types of conversations - especially on testing where I struggle for consistency and completeness.
I tend to agree that my testing focuses on methods, however the majority of my ‘failures’ revolve around insufficient object values that eventually have my tests failing - I’ve defaulted to generating ‘data’ as randomly as I can early on, however non-realistic, and running tests a few thousand times to point out assumptions I make in my methods.
Thanks for discussions!
Russ Whaley
whaley.russ@gmail.com
On Thu, Jan 23, 2025 at 5:02 AM Richard O'Keefe raoknz@gmail.com wrote:
This makes the assumption that there is, most of the time, a single,
unique, object that the test is clearly about.
I hasten to admit that my unit testing practice needs a lot to be desired.
But it has not been my experience that there is any such object much
of the time.
For example, to test a method, I will sometimes create two equal
objects, manipulate one of them in the
old trusted way, manipulate the other in the new way, and then see if
the objects are equal again.
Which then is the subject?
The answer is neither, because what the test is about is not an
object, but a method.
Or again, I open a stream to a new file, create a complex object,
persist it on the stream, close
the stream, open another stream reading from the file, reconstitute an
object from it, and then
check that the two objects are isomorphic. Which is the subject?
Neither, because what I am testing is the encoding and decoding
methods, not the objects.
The key paragraph is this one:
<quote>
One could argue that there is not always a single object under test.
In some situations that is correct, for instance when testing the
interaction between objects. But in my experience, most unit tests
have a single object under test.
</quote>
And it is this that we really need a blog post on. I am very ready
to believe that you are a much better tester than I am.
(That wouldn't be hard.) But something interesting is going on when
I think in terms of testing classes and methods to
the point where I find the idea of testing a particular object almost
incomprehensible, yet you think this is the norm.
I take some comfort from the fact that the Gherkin language in
Cucumber (and why oh which has nobody bothered to
write down a proper grammar, let alone even an informal semantics, for
it? How am I supposed to know what is and what
is not legal Gherkin?) goes out of its way to conceal the existence
of subjects, should they exist, from test reviewers, and
calls what a test is about a "feature".
Here are two tests for the Quaternion class:
testMultiplicationByScalarCommutes
sampleExactScalars do: [:x |
sampleExactQuaternions do: [:y |
self assert: (xy) = (yx)]].
testMultiplicationOfPowersCommutes
sampleExactQuaternions do: [:x |
|y z|
y := x squared.
z := x cubed.
self assert: (y*z) = (z*y)].
These two methods test important properties of the #* method.
Here's another one, part of testing the test data..
testExactScalarsAreExactScalars
sampleExactScalars do: [:each |
self assert: each isScalar.
self assert: each isExact].
This tests the methods #isScalar and #isExact, and indirectly the
code that generates the test data.
It's not about any particular object. Nor is it about one method.
It's very hard to write a test case that
tests only one method. In fact my own rather embarrassing experience
is that when a test case of mine
failed it quite often wasn't the method I thought I was testing that
was wrong. Returning to the
persistence example, it's not testing #saveOn: (object saveOn:
byteStream) or #restore (byteStream restore).
It's testing that the two methods are properly related. It would be
wrong to call it #testSaveOn: and it
would be wrong to call it #testRestore. There's a distinction here:
it is possible to write methods to test
#saveOn: by itself, by writing various objects and checking that you
get the right bytes. But if the external
representation is changed (and I need to change it), the tests for
#saveOn: and #restore as such will have
to be rewritten, while the tests for #saveOn: and #restore together will not.
Cucumber offers an insight here, I think. A scenario in Gherkin is
not a test case for an object.
Nor is it a test case for a method. It is a test case for a composite
behaviour of a system,
at some contextually appropriate level of detail.
Brian Marick had some lovely stickers. One of them reads "An example
would be handy about now."
I note that your blog post did provide an example of testing Date.
In which none of your test cases used 'subject'.
I entirely agree with you that the three methods have good
intention-revealing names.
What we actually need is some examples where 'subject' pays off.
On Mon, 20 Jan 2025 at 04:04, Koen De Hondt
koen@all-objects-all-the-time.st wrote:
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post about a dedicated name for the object under test in SUnit tests, the “subject".
Happy reading!
Ciao,
Koen
I too like to use random data in test cases but you also have to use
edge cases that random data tend to avoid.
It is painful to recall the millions of successful tests of my
Dictionary implementation that failed to detect that
"Dictionary new: 0" created a non-functional object. ((self pvtHash:
key) \\ capacity) + 1" did not work well when
capacity was 0. 0 size, 0 capacity, 0 room left, any edge case.
Conversely, you have to avoid special cases
sometimes: when testing x ./ y and y \ x for Quaternions, you want to
make sure y isZero not.
One thing I would like some advice on is testing traits. I understand
about testing classes and about testing
methods in classes (though I am never thorough enough), but how do you
test a method that isn't bound to
a particular context, where the thing that reveals a mistake may be a
class it will be combined with in the
future that doesn't exist yet? What is considered good practice?
On Tue, 4 Feb 2025 at 05:25, Russ Whaley <whaley.russ@gmail.com> wrote:
>
> Folks, I really appreciate (and learn from) these types of conversations - especially on testing where I struggle for consistency and completeness.
>
> I tend to agree that my testing focuses on methods, however the majority of my ‘failures’ revolve around insufficient object values that eventually have my tests failing - I’ve defaulted to generating ‘data’ as randomly as I can early on, however non-realistic, and running tests a few thousand times to point out assumptions I make in my methods.
>
> Thanks for discussions!
>
> Russ Whaley
> whaley.russ@gmail.com
>
>
> On Thu, Jan 23, 2025 at 5:02 AM Richard O'Keefe <raoknz@gmail.com> wrote:
>>
>> This makes the assumption that there is, most of the time, a single,
>> unique, object that the test is clearly about.
>> I hasten to admit that my unit testing practice needs a lot to be desired.
>> But it has not been my experience that there *is* any such object much
>> of the time.
>> For example, to test a method, I will sometimes create two equal
>> objects, manipulate one of them in the
>> old trusted way, manipulate the other in the new way, and then see if
>> the objects are equal again.
>> Which then is *the* subject?
>> The answer is neither, because what the test is *about* is not an
>> object, but a method.
>> Or again, I open a stream to a new file, create a complex object,
>> persist it on the stream, close
>> the stream, open another stream reading from the file, reconstitute an
>> object from it, and then
>> check that the two objects are isomorphic. Which is *the* subject?
>> Neither, because what I am testing is the encoding and decoding
>> methods, not the objects.
>>
>> The key paragraph is this one:
>> <quote>
>> One could argue that there is not always a single object under test.
>> In some situations that is correct, for instance when testing the
>> interaction between objects. But in my experience, most unit tests
>> have a single object under test.
>> </quote>
>>
>> And it is *this* that we really need a blog post on. I am very ready
>> to believe that you are a much better tester than I am.
>> (That wouldn't be hard.) But *something* interesting is going on when
>> I think in terms of testing classes and methods to
>> the point where I find the idea of testing a particular object almost
>> incomprehensible, yet you think this is the norm.
>> I take some comfort from the fact that the Gherkin language in
>> Cucumber (and why oh which has nobody bothered to
>> write down a proper grammar, let alone even an informal semantics, for
>> it? How am I supposed to know what is and what
>> is not legal Gherkin?) goes out of its way to conceal the existence
>> of subjects, should they exist, from test reviewers, and
>> calls what a test is about a "feature".
>>
>> Here are two tests for the Quaternion class:
>> testMultiplicationByScalarCommutes
>> sampleExactScalars do: [:x |
>> sampleExactQuaternions do: [:y |
>> self assert: (x*y) = (y*x)]].
>>
>> testMultiplicationOfPowersCommutes
>> sampleExactQuaternions do: [:x |
>> |y z|
>> y := x squared.
>> z := x cubed.
>> self assert: (y*z) = (z*y)].
>>
>> These two methods test important properties of the #* method.
>>
>> Here's another one, part of testing the test data..
>>
>> testExactScalarsAreExactScalars
>> sampleExactScalars do: [:each |
>> self assert: each isScalar.
>> self assert: each isExact].
>>
>> This tests the *methods* #isScalar and #isExact, and indirectly the
>> code that generates the test data.
>> It's not about any particular object. Nor is it about *one* method.
>> It's very hard to write a test case that
>> tests only one method. In fact my own rather embarrassing experience
>> is that when a test case of mine
>> failed it quite often wasn't the method I thought I was testing that
>> was wrong. Returning to the
>> persistence example, it's not testing #saveOn: (object saveOn:
>> byteStream) or #restore (byteStream restore).
>> It's testing that the two methods are properly related. It would be
>> wrong to call it #testSaveOn: and it
>> would be wrong to call it #testRestore. There's a distinction here:
>> it *is* possible to write methods to test
>> #saveOn: by itself, by writing various objects and checking that you
>> get the right bytes. But if the external
>> representation is changed (and I need to change it), the tests for
>> #saveOn: and #restore as such will have
>> to be rewritten, while the tests for #saveOn: and #restore *together* will not.
>>
>> Cucumber offers an insight here, I think. A scenario in Gherkin is
>> not a test case for an object.
>> Nor is it a test case for a method. It is a test case for a composite
>> *behaviour* of a system,
>> at some contextually appropriate level of detail.
>>
>> Brian Marick had some lovely stickers. One of them reads "An example
>> would be handy about now."
>>
>> I note that your blog post *did* provide an example of testing Date.
>> In which none of your test cases used 'subject'.
>> I entirely agree with you that the three methods have good
>> intention-revealing names.
>> What we actually need is some examples where 'subject' pays off.
>>
>>
>>
>> On Mon, 20 Jan 2025 at 04:04, Koen De Hondt
>> <koen@all-objects-all-the-time.st> wrote:
>> >
>> > Dear Pharo users and developers,
>> >
>> > In my series of blog posts on testing, I added a follow-up post about a dedicated name for the object under test in SUnit tests, the “subject".
>> >
>> > Happy reading!
>> >
>> > Ciao,
>> > Koen
>> >
RW
Russ Whaley
Tue, Feb 4, 2025 12:38 PM
Agreed. A little knowledge of how it is to be used can go a long way.
I seem to have a mental block on creating traits. I am sure my code and
applications would benefit - but I just can’t seem to understand what they
are and how to design/implement them.
Russ Whaley
whaley.russ@gmail.com
On Tue, Feb 4, 2025 at 2:23 AM Richard O'Keefe raoknz@gmail.com wrote:
I too like to use random data in test cases but you also have to use
edge cases that random data tend to avoid.
It is painful to recall the millions of successful tests of my
Dictionary implementation that failed to detect that
"Dictionary new: 0" created a non-functional object. ((self pvtHash:
key) \ capacity) + 1" did not work well when
capacity was 0. 0 size, 0 capacity, 0 room left, any edge case.
Conversely, you have to avoid special cases
sometimes: when testing x ./ y and y \ x for Quaternions, you want to
make sure y isZero not.
One thing I would like some advice on is testing traits. I understand
about testing classes and about testing
methods in classes (though I am never thorough enough), but how do you
test a method that isn't bound to
a particular context, where the thing that reveals a mistake may be a
class it will be combined with in the
future that doesn't exist yet? What is considered good practice?
On Tue, 4 Feb 2025 at 05:25, Russ Whaley whaley.russ@gmail.com wrote:
Folks, I really appreciate (and learn from) these types of conversations
- especially on testing where I struggle for consistency and completeness.
I tend to agree that my testing focuses on methods, however the majority
of my ‘failures’ revolve around insufficient object values that eventually
have my tests failing - I’ve defaulted to generating ‘data’ as randomly as
I can early on, however non-realistic, and running tests a few thousand
times to point out assumptions I make in my methods.
This makes the assumption that there is, most of the time, a single,
unique, object that the test is clearly about.
I hasten to admit that my unit testing practice needs a lot to be
But it has not been my experience that there is any such object much
of the time.
For example, to test a method, I will sometimes create two equal
objects, manipulate one of them in the
old trusted way, manipulate the other in the new way, and then see if
the objects are equal again.
Which then is the subject?
The answer is neither, because what the test is about is not an
object, but a method.
Or again, I open a stream to a new file, create a complex object,
persist it on the stream, close
the stream, open another stream reading from the file, reconstitute an
object from it, and then
check that the two objects are isomorphic. Which is the subject?
Neither, because what I am testing is the encoding and decoding
methods, not the objects.
The key paragraph is this one:
<quote>
One could argue that there is not always a single object under test.
In some situations that is correct, for instance when testing the
interaction between objects. But in my experience, most unit tests
have a single object under test.
</quote>
And it is this that we really need a blog post on. I am very ready
to believe that you are a much better tester than I am.
(That wouldn't be hard.) But something interesting is going on when
I think in terms of testing classes and methods to
the point where I find the idea of testing a particular object almost
incomprehensible, yet you think this is the norm.
I take some comfort from the fact that the Gherkin language in
Cucumber (and why oh which has nobody bothered to
write down a proper grammar, let alone even an informal semantics, for
it? How am I supposed to know what is and what
is not legal Gherkin?) goes out of its way to conceal the existence
of subjects, should they exist, from test reviewers, and
calls what a test is about a "feature".
Here are two tests for the Quaternion class:
testMultiplicationByScalarCommutes
sampleExactScalars do: [:x |
sampleExactQuaternions do: [:y |
self assert: (xy) = (yx)]].
testMultiplicationOfPowersCommutes
sampleExactQuaternions do: [:x |
|y z|
y := x squared.
z := x cubed.
self assert: (y*z) = (z*y)].
These two methods test important properties of the #* method.
Here's another one, part of testing the test data..
testExactScalarsAreExactScalars
sampleExactScalars do: [:each |
self assert: each isScalar.
self assert: each isExact].
This tests the methods #isScalar and #isExact, and indirectly the
code that generates the test data.
It's not about any particular object. Nor is it about one method.
It's very hard to write a test case that
tests only one method. In fact my own rather embarrassing experience
is that when a test case of mine
failed it quite often wasn't the method I thought I was testing that
was wrong. Returning to the
persistence example, it's not testing #saveOn: (object saveOn:
byteStream) or #restore (byteStream restore).
It's testing that the two methods are properly related. It would be
wrong to call it #testSaveOn: and it
would be wrong to call it #testRestore. There's a distinction here:
it is possible to write methods to test
#saveOn: by itself, by writing various objects and checking that you
get the right bytes. But if the external
representation is changed (and I need to change it), the tests for
#saveOn: and #restore as such will have
to be rewritten, while the tests for #saveOn: and #restore together
Cucumber offers an insight here, I think. A scenario in Gherkin is
not a test case for an object.
Nor is it a test case for a method. It is a test case for a composite
behaviour of a system,
at some contextually appropriate level of detail.
Brian Marick had some lovely stickers. One of them reads "An example
would be handy about now."
I note that your blog post did provide an example of testing Date.
In which none of your test cases used 'subject'.
I entirely agree with you that the three methods have good
intention-revealing names.
What we actually need is some examples where 'subject' pays off.
On Mon, 20 Jan 2025 at 04:04, Koen De Hondt
koen@all-objects-all-the-time.st wrote:
Dear Pharo users and developers,
In my series of blog posts on testing, I added a follow-up post about
a dedicated name for the object under test in SUnit tests, the “subject".
Happy reading!
Ciao,
Koen
Agreed. A little knowledge of how it is to be used can go a long way.
I seem to have a mental block on creating traits. I am sure my code and
applications would benefit - but I just can’t seem to understand what they
are and how to design/implement them.
Russ Whaley
whaley.russ@gmail.com
On Tue, Feb 4, 2025 at 2:23 AM Richard O'Keefe <raoknz@gmail.com> wrote:
> I too like to use random data in test cases but you also have to use
> edge cases that random data tend to avoid.
> It is painful to recall the millions of successful tests of my
> Dictionary implementation that failed to detect that
> "Dictionary new: 0" created a non-functional object. ((self pvtHash:
> key) \\ capacity) + 1" did not work well when
> capacity was 0. 0 size, 0 capacity, 0 room left, any edge case.
> Conversely, you have to avoid special cases
> sometimes: when testing x ./ y and y \ x for Quaternions, you want to
> make sure y isZero not.
>
> One thing I would like some advice on is testing traits. I understand
> about testing classes and about testing
> methods in classes (though I am never thorough enough), but how do you
> test a method that isn't bound to
> a particular context, where the thing that reveals a mistake may be a
> class it will be combined with in the
> future that doesn't exist yet? What is considered good practice?
>
> On Tue, 4 Feb 2025 at 05:25, Russ Whaley <whaley.russ@gmail.com> wrote:
> >
> > Folks, I really appreciate (and learn from) these types of conversations
> - especially on testing where I struggle for consistency and completeness.
> >
> > I tend to agree that my testing focuses on methods, however the majority
> of my ‘failures’ revolve around insufficient object values that eventually
> have my tests failing - I’ve defaulted to generating ‘data’ as randomly as
> I can early on, however non-realistic, and running tests a few thousand
> times to point out assumptions I make in my methods.
> >
> > Thanks for discussions!
> >
> > Russ Whaley
> > whaley.russ@gmail.com
> >
> >
> > On Thu, Jan 23, 2025 at 5:02 AM Richard O'Keefe <raoknz@gmail.com>
> wrote:
> >>
> >> This makes the assumption that there is, most of the time, a single,
> >> unique, object that the test is clearly about.
> >> I hasten to admit that my unit testing practice needs a lot to be
> desired.
> >> But it has not been my experience that there *is* any such object much
> >> of the time.
> >> For example, to test a method, I will sometimes create two equal
> >> objects, manipulate one of them in the
> >> old trusted way, manipulate the other in the new way, and then see if
> >> the objects are equal again.
> >> Which then is *the* subject?
> >> The answer is neither, because what the test is *about* is not an
> >> object, but a method.
> >> Or again, I open a stream to a new file, create a complex object,
> >> persist it on the stream, close
> >> the stream, open another stream reading from the file, reconstitute an
> >> object from it, and then
> >> check that the two objects are isomorphic. Which is *the* subject?
> >> Neither, because what I am testing is the encoding and decoding
> >> methods, not the objects.
> >>
> >> The key paragraph is this one:
> >> <quote>
> >> One could argue that there is not always a single object under test.
> >> In some situations that is correct, for instance when testing the
> >> interaction between objects. But in my experience, most unit tests
> >> have a single object under test.
> >> </quote>
> >>
> >> And it is *this* that we really need a blog post on. I am very ready
> >> to believe that you are a much better tester than I am.
> >> (That wouldn't be hard.) But *something* interesting is going on when
> >> I think in terms of testing classes and methods to
> >> the point where I find the idea of testing a particular object almost
> >> incomprehensible, yet you think this is the norm.
> >> I take some comfort from the fact that the Gherkin language in
> >> Cucumber (and why oh which has nobody bothered to
> >> write down a proper grammar, let alone even an informal semantics, for
> >> it? How am I supposed to know what is and what
> >> is not legal Gherkin?) goes out of its way to conceal the existence
> >> of subjects, should they exist, from test reviewers, and
> >> calls what a test is about a "feature".
> >>
> >> Here are two tests for the Quaternion class:
> >> testMultiplicationByScalarCommutes
> >> sampleExactScalars do: [:x |
> >> sampleExactQuaternions do: [:y |
> >> self assert: (x*y) = (y*x)]].
> >>
> >> testMultiplicationOfPowersCommutes
> >> sampleExactQuaternions do: [:x |
> >> |y z|
> >> y := x squared.
> >> z := x cubed.
> >> self assert: (y*z) = (z*y)].
> >>
> >> These two methods test important properties of the #* method.
> >>
> >> Here's another one, part of testing the test data..
> >>
> >> testExactScalarsAreExactScalars
> >> sampleExactScalars do: [:each |
> >> self assert: each isScalar.
> >> self assert: each isExact].
> >>
> >> This tests the *methods* #isScalar and #isExact, and indirectly the
> >> code that generates the test data.
> >> It's not about any particular object. Nor is it about *one* method.
> >> It's very hard to write a test case that
> >> tests only one method. In fact my own rather embarrassing experience
> >> is that when a test case of mine
> >> failed it quite often wasn't the method I thought I was testing that
> >> was wrong. Returning to the
> >> persistence example, it's not testing #saveOn: (object saveOn:
> >> byteStream) or #restore (byteStream restore).
> >> It's testing that the two methods are properly related. It would be
> >> wrong to call it #testSaveOn: and it
> >> would be wrong to call it #testRestore. There's a distinction here:
> >> it *is* possible to write methods to test
> >> #saveOn: by itself, by writing various objects and checking that you
> >> get the right bytes. But if the external
> >> representation is changed (and I need to change it), the tests for
> >> #saveOn: and #restore as such will have
> >> to be rewritten, while the tests for #saveOn: and #restore *together*
> will not.
> >>
> >> Cucumber offers an insight here, I think. A scenario in Gherkin is
> >> not a test case for an object.
> >> Nor is it a test case for a method. It is a test case for a composite
> >> *behaviour* of a system,
> >> at some contextually appropriate level of detail.
> >>
> >> Brian Marick had some lovely stickers. One of them reads "An example
> >> would be handy about now."
> >>
> >> I note that your blog post *did* provide an example of testing Date.
> >> In which none of your test cases used 'subject'.
> >> I entirely agree with you that the three methods have good
> >> intention-revealing names.
> >> What we actually need is some examples where 'subject' pays off.
> >>
> >>
> >>
> >> On Mon, 20 Jan 2025 at 04:04, Koen De Hondt
> >> <koen@all-objects-all-the-time.st> wrote:
> >> >
> >> > Dear Pharo users and developers,
> >> >
> >> > In my series of blog posts on testing, I added a follow-up post about
> a dedicated name for the object under test in SUnit tests, the “subject".
> >> >
> >> > Happy reading!
> >> >
> >> > Ciao,
> >> > Koen
> >> >
>
