[Pharo-users] [Pharo-dev] TestAsserter>>assertCollection:hasSameElements:

jtuchel at objektfabrik.de jtuchel at objektfabrik.de
Mon Oct 26 03:27:28 EDT 2015


Am 25.10.15 um 16:33 schrieb Peter Uhnák:
>
>     > assert:equals:  it's just more typing than #= with no additional outcome
>
>
> I also disagree, but that may be also because maybe we write tests for 
> different purpse.
> If you write tests just to test your code, then whatever... I don't do 
> that so I can't comment on that.
>
> However if you do TDD, then tests are to provide feedback, and from 
> that perspective you want to see immediately what is the difference. 
> If I see that the assertion failed I have to start digging to find out 
> what is the difference, which is extra work and bad from feedback 
> perspective. If I instead immediately see what I need to see it allows 
> me to faster deduce the source of the problem and resolve it.
>
I'm sorry, but what you are saying doesn't make any sense. Even if I 
"only" want to test code (which is exactly what you do in TDD, btw.), I 
need good feedback.
That is, btw., the reason why I suggested an extension to SUnit a few 
years ago that would help keep test results in a collection for use in 
CI environments.

So I agree there is the problem of short and precise ways of giving 
precise and helpful feedback in SUnit. But solving this by adding more 
and more variants of assert: to the framework, just to give nicer error 
strings is like healing a headache by decapitating the patient.

What is needed is some way to make providing feedback easier and 
friendlier, and we should think about ways to achieve that. Adding lots 
of sloppily-defined assertion methods is just the wrong solution.

We could think about subclassing TestFailure and a way to hand 
information to the TestFailure so that a nice String can be produced. 
Like a method like cull: that adds arguments' printString representation 
into the failure description.

> And this is actually quite generic idea... imagine that you are 
> filling a web form and when you click "submit" the website tells you 
> "the form is invalid" and nothing else; that is not helpful at all.
> If it instead tells you "The date must be in format MM/DD/YYYY" and 
> highlights the incriminating field that I immediately can see the 
> problem and can fix it quickly.
>
> So the different #assert:whatever: provide context for the assertion 
> to be able to get more valuable feedback.
Please step back for a second and think again: these are two very 
different things. The job of an Assertion is to make a problem visible. 
The representation of the problem is something else, even if these are 
closely related. This is object thinking lesson #2 or so.

>
> It is a world of difference.

I agree. Good and exact feedback is at least as important as finding a 
problem. Because knowing there is a problem without understanding the 
problem is not enough. But that does not support the theory of "more 
assertion variants make our life so much easier". It only says we need a 
way to provide good feedback that is short and precise enough to not be 
ignored to our laziness.

But: adding more and more misnamed and misleading assertion methods 
makes the use of SUnit frustrating and will make it obsolete over time. 
If I have to hunt for design problems in SUnit because it assumes 
something to be wrong even though my understanding of waht I tested is 
different, I lose way more time than I am ready to accept. This doesn't 
happen to me often. If finding that I misunderstood an assertion method 
means I lost a few hours, the best thing that may happen is that I never 
use that method again. In the worst case, I decide I think SUnit is 
useless for me. That would be really bad, don't you think?
>
>  > assertCollection:hasSameElements:
>
>     > So, let's start by asking what the question really means. Does
>     it mean that one collection is a subset of the other? What about
>     ordering then? Does it mean both contain the same elements at the
>     same position or just the fact that if they both were Sets
>     contained the exact same elements. The question itself is not
>     exact, so how could an answer possibly be?
>
>
> There is no question about this method, since this is implemented in 
> code there is nothing ambiguous. This method effectively converts both 
> arguments to sets and compares them like that.

Sorry to say that, but this is ambiguity by design: you define 
hasSameElements: as "both result in the same Set". So the name of this 
assertion method is a great example of bad naming, IMO.

> The question is whether such thing is useful for non-sets, and there 
> are definitely cases where such assertion is not appropriate (when you 
> want to ensure that there are specific items or specific positions), 
> that's why I suggested asserts specifically for that.
This is just nonsense. You name a method after a general collection 
class and try to tell me that it doesn't matter that it is only suitable 
for Sets and that is okay?

>     self assert: result asOrderedCollection asSortedCollection equals:
>     (1 to: 10) asOrderedCollection
>
>
> This is what I usually do now (although I convert to Array, not 
> OrderedCollection, because the expected one is usually created by hand 
> with #() or {} ).
>
I don't really care. If what you try t say is that the testing code can 
be ugly and long, then I agree. If you need tests like this very often 
and want something to make this easier, I understand and agree that some 
additions to SUnit can be helpful. But the way this has been tried so 
far seems completely wrong to me.

>     > Just a few weeks ago, we discussed something similar about the
>     equality of two Collections. Endless discussions where people try
>     convince others that their definition of equality and/or sameness
>     is correct with no outcome.
>
>
> I don't see a problem with that because collections truly can have 
> different equalities based on the context and their purpose. And while 
> you can call this rat poison, it effectively tells what kind of 
> behavior you expect from the collections, which seems ok.
So what, again, was the point of naming a method after a general 
Collection class and use a question that is very unspecific? A 
Collection has the same elements as another does not necessarily mean 
they both result in the same set. Can we agree on that? All the question 
asks if all Elements in Collection A can also be found in Collection B. 
The method name states nothing more than that.

>     > So we could start by providing an abstract superclass for
>     comparing collections based on more than just #= and #==. Then we
>     can add subclasses for that which have a real intention revealing
>     name like
>     CollectionHasExactlyTheSameElementsAsAnotherAtTheSamePosition.
>
>
> well this one specifically is quite regular array comparison, 
> questions are more about the other cases (without order, etc...). And 
> I don't see why new class instead of a selector somehow solves this.
This is a misconception we also discussed in this other discussion. It 
simply isn't true. And even if it was true, it still wouldn't match with 
the chosen method name.

My point here is that a general purpose framework like SUnit should be 
free of such debatable things. SUnit has to be reliable and understandable.
There is nothing wrong with providing some "plugins" for problems like 
Collections that make life easier.
It would be desirable to have more control over SUnit's feedback with 
little typing.

But there is a lot that's wrong in polluting a general purpose framework 
like SUnit with unclear method names that are misleading, just plain 
wrong or debatable.

So we are discussing existing, real problems that exist in SUnit, and we 
are discussing about one of the possible ways to make SUnit friendlier 
that - in my opinion - is counter-productive.


>
>     please keep these highly fragile
>
> why is it fragile?
Because my misunderstanding of what the author of 
assertCollection:hasSameElementsAs: thought the question means can cause 
me a hunt for bugs in his misnaming rather than my code. I may throw 
SUnit into the corner or just not trust in it. That's fragile, not agile.


>     and debatable things
>
> that's why I initiated this debate in the first place.
Which is a good thing and I hope I could explain exactly enough why I 
think the addition of more and more assert: variations for the sake of 
better failure texts is the wrong path.

We should look for better, more precise ways to achieve what is needed. 
One step, IMO, is to agree on the fact that formulating an assertion and 
providing a helpful failure text are two strongly-related, but separate 
concerns. So a redesign of SUnit should be based on the question of how 
we can extend the feedback providing side of SUnit, and not on the 
question of what else people might be testing for and what the resulting 
text should be looking like and then find the best possible assertion 
method name based on that.

By writing this last sentence, I found another argument: not only teh 
new assertion method name could be misleading and debatable, but also 
the failure text you provide in the SUNit code base. This adds a third 
level of problems. What if your failure text was just "Not all Elements 
of $A are in $B" - does it match your implementation? I think it matches 
the method name, but not your implementation...


Nuff said,


Joachim





-- 
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel          mailto:jtuchel at objektfabrik.de
Fliederweg 1                         http://www.objektfabrik.de
D-71640 Ludwigsburg                  http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-users_lists.pharo.org/attachments/20151026/cfd472f2/attachment.html>


More information about the Pharo-users mailing list