Peter,

I see you clearly understand what I wanted to get into and don't give up because of my aggressive tone in the first place.

Am 26.10.15 um 23:53 schrieb Peter Uhn��k:

On Mon, Oct 26, 2015 at 8:27 AM, jtuchel@objektfabrik.de��<jtuchel@objektfabrik.de>��wrote:

Am 25.10.15 um 16:33 schrieb Peter Uhn��k:

> assert:equals:�� it's just more typing than #= with no additional outcome

I also disagree, but that may be also because maybe we write tests for different purpse.

If you write tests just to test your code, then whatever... I don't do that so I can't comment on that.

However if you do TDD, then tests are to provide feedback, and from that perspective you want to see immediately what is the difference. If I see that the assertion failed I have to start digging to find out what is the difference, which is extra work and bad from feedback perspective. If I instead immediately see what I need to see it allows me to faster deduce the source of the problem and resolve it.

I'm sorry, but what you are saying doesn't make any sense. Even if I "only" want to test code (which is exactly what you do in TDD, btw.), I need good feedback.��

Well, that was a bit of generalization on my part. The point was, that many people write tests as an afterthought... which means they have different means of getting feedback from the system (e.g. logging, running manually the code, etc.). In such situation they often end up testing already mostly working code, and thus the feedback from the tests is not as important, because you end up less digging up problems (since large portion were found and resolved during manual testing).

I don't think there really is a difference between test first and tests as an afterthought. You always need good and precise feedback. What you say about having learned much more during development of a body of code is not necessarily true in teams or when you maintain software that has grown for years or decades, Remember, there are Smalltalk systems out there whose development started in the late 80s or early 90s. In fact, most commercial Smalltalk projects out in the wild are rather old.
If you have to make sure you don't break anything by putting a system under test, you need precise feeddback, because errors can happen in areas you haven't touched in the last 5 or 10 years or even never before.

There is, btw, another area in which SUnit can be extremely helpful: Understanding an existing body of code and proving your theories about code to yourself while you dive deeper. Here, feedback is also very important.

So let's just assume that feedback of SUnit can never be too specific and is one of the very most important aspects of the framework.

��

We could think about subclassing TestFailure and a way to hand information to the TestFailure so that a nice String can be produced. Like a method like cull: that adds arguments' printString representation into the failure description.��

You mean #assert:description: ? Because we already have that.

��

Well, I actually mean a new generation of #assert:description: that concentrates more on the #description: part. Remember: some assertion methods have been added as a way to provide "more suitable" feedback, like handing in an object or two to be mentioned in the description string. So the problem at hand wasn't that the standard assert: method was insufficient, just its output.
I clearly understand how this can happen. You want to provide a better log output and since you are testing something that even others might need, you just add that damn method that writes a nice "expecte $A but got $B".
In fact, this solves a problem in the wrong way, because it will result in many more or less well-named assertion methods that basically are there to provide a better failure String.

So what we have is a way to display a description without parameters. But we sometimes want more than that. In case of #assert:equals: we wanted a String that says "Expected $A and got $B". Let's not speculate whether this String really saves us much time if we are in a TDD scenario or on a build server. Let's just assume it is never bad to know as much as possible about a problem to do something about it.

Let's also not forget that Kent Beck had this clear idea from the very beginning of SUnit that it must be lightweight and easy to understand to not shy people away from trying. You must be able to use SUnit within an hour and it must be easy and provide instant feedback. If I have to type too much, or keep too many things in mind, I won't use it.

Please step back for a second and think again: these are two very different things. The job of an Assertion is to make a problem visible. The representation of the problem is something else, even if these are closely related. This is object thinking lesson #2 or so.

Object thinking lesson #3 tells me that I should not care about what is going on behind the curtains.

Well, that is probably not really true for unit tests, because you may want to test something in a very specific way to prove your thesis about how to solve a problem is true under several circumstances.

And while I could explicitly separate the two, if I do it all the time I don't see what's bad about having a convenience method. (And by looking at Object or String protocol, Pharo is a lot about having convenience over engineering rigidness).

Convenience methods are not bad per se, as long as they add value and clearly communicate what they do. Pharo and other Smalltalks are stuffed with lots of bad examples here. Sometimes Convenience methods are a symptom of design problems or the lack of multiple inheritance. Sometimes they are just a good idea.

But: adding more and more misnamed and misleading assertion methods makes the use of SUnit frustrating and will make it obsolete over time. If I have to hunt for design problems in SUnit because it assumes something to be wrong even though my understanding of waht I tested is different, I lose way more time than I am ready to accept. This doesn't happen to me often. If finding that I misunderstood an assertion method means I lost a few hours, the best thing that may happen is that I never use that method again. In the worst case, I decide I think SUnit is useless for me. That would be really bad, don't you think?

I am not sure if we are talking about the same SUnit. Sure, there are 32 methods in the "asserting" protocol, however most of them are either opposites of one another "#assert: vs #deny:", or they provide some customization such has "#assert:description:", "#should:raise:" ... so if I count only meaningfully different methods the number 7 (not to mention that some of the methods are not even used). But if you have trouble understanding the purpose of seven methods, then the problem is on your end, and don't blame SUnit for it.

I think should:/shouldnt are great examples of convenience methods that are not named good enough. I know nobody who can use them without taking a second to think about them.
So what you are saying is that SUnit already has roughly 25 convenience methods that aren't used or hard to understand. And the ones we understand are used because they are a good shortcut to a better description string (assert:equals:). So we are not far away from each other... ;-)

��> assertCollection:hasSameElements:

> So, let's start by asking what the question really means. Does it mean that one collection is a subset of the other? What about ordering then? Does it mean both contain the same elements at the same position or just the fact that if they both were Sets contained the exact same elements. The question itself is not exact, so how could an answer possibly be?

There is no question about this method, since this is implemented in code there is nothing ambiguous. This method effectively converts both arguments to sets and compares them like that.

Sorry to say that, but this is ambiguity by design: you define hasSameElements: as "both result in the same Set". So the name of this assertion method is a great example of bad naming, IMO.

Yes, the naming is confusing. My point was, that instead of philosophizing about the meaning you can look at the code. Of course if you use the method for the first time (like I did), you will get burned by it (as I did).

Hmm - just by reading these 3 last sentences, you see that naming is important. But we don't disagree here. Let's forget about the concrete naming of the method from now on. It's an example.
My point was that if the naming of a method opens room for speculation and philosophy, it is most likely either named badly or it is a bad idea in itself. Maybe it is not a candidate for addition as a core method in SUnit. SUnit should make you think about your code, not about itself.

��

self assert: result asOrderedCollection asSortedCollection equals: (1 to: 10) asOrderedCollection

This is what I usually do now (although I convert to Array, not OrderedCollection, because the expected one is usually created by hand with #() or {} ).

I don't really care. If what you try t say is that the testing code can be ugly and long, then I agree. If you need tests like this very often and want something to make this easier, I understand and agree that some additions to SUnit can be helpful. But the way this has been tried so far seems completely wrong to me.��

How would you test it then? Some problem domains deal with certain kind of problems more than others and thus benefit more from appropriate assertions.

This is not about how to test for something. Our discussion is about the question whether a certain wayto test something should be added to the framework or not. Is the test generic and important enough so that it should make its way into SUnit. And if it should be added, what is a good name for it, so that others find the method as the right thing to use for their purpose or will it disturb them in their proces more than it helps.

You can always use helper objects and non-SUnit-classes in your test packages for generalizations of your project's needs. If you need to walk trees a lot for your purposes, you can add Visitors and stuff, to your test package, put them under test to make sure they work and use them in your assertions. This is not necessarily a candidate for inclusion in SUnit, just because chances are others need to test Trees as well.

��

> Just a few weeks ago, we discussed something similar about the equality of two Collections. Endless discussions where people try convince others that their definition of equality and/or sameness is correct with no outcome.

I don't see a problem with that because collections truly can have different equalities based on the context and their purpose. And while you can call this rat poison, it effectively tells what kind of behavior you expect from the collections, which seems ok.

So what, again, was the point of naming a method after a general Collection class and use a question that is very unspecific? A Collection has the same elements as another does not necessarily mean they both result in the same set. Can we agree on that? All the question asks if all Elements in Collection A can also be found in Collection B. The method name states nothing more than that.��

We agree that the method is badly named. (However what equality of two Collections means is context-dependent.)

Phew. Thank you. I am glad we agree on this. So let's carry on with the SUnit related stuff.

��

My point here is that a general purpose framework like SUnit should be free of such debatable things. SUnit has to be reliable and understandable.
There is nothing wrong with providing some "plugins" for problems like Collections that make life easier.
It would be desirable to have more control over SUnit's feedback with little typing.

Well but we need a way to provide context for the assertion. If I could type less then I would be happier user, however currently I don't see a way how to make it more general (so I don't need to type) and more precise (it still understands context) at the same time.

Now I have you in the corner I want you in ;-)
We should talk about ways to make it easier to provide clearer feedback Strings

So going back to your earlier suggestion with the class... you imagine something like this?

self assert: a equals: b strategy: CollectionHasExactlyTheSameElementsAsAnotherAtTheSamePosition

No. For several reasons (based on how I interpret the names you provide as an example):

* You are back to the idea that you need a way to specify a specialized testing strategy. That is exactly the wrong thought here. What and how to test is highly dependent on your code base and needs. A simple #asserts: and your very special way of testing formulated either by writing complex code or using your own specialized helpers is the only way to keep SUnit lean and mean (BTW, there is nobody keeping us from providing SUnit helper packages that make life easier, like the Tree example - but not as part of core SUnit)

* what if I need a test to prove/check that two collections are NOT the same in a very special case? would you add a #deny:equals:strategy: also? It will blow the testing protocol up from 32 to 100 very soon.

I would like to start a discussion on ways to separate the assertion stuff from the feedback generation. The first steps I can think of are not sufficient in the "little typing" arena, and I must admit I have not yet found a good idea.

But to start with something, I'd like to start with the assumption that we only have #assert: and all we want is a second parameter, like in #assert:description:

So I am now just brainstorming, not saying this is a good solution already.

Let's assume we have some way to add parameters into Strings, like #bindWith: and friends (or Mustache, or whatever). Let's also assume there is something like Block>>#cull: that makes it easy to fill such a String with any number of arguments (e.g. 'I accept up to %1 arguments' cull: {arg1, arg2}).
Couldn't we just put some machinery into the test runner that uses this facility to provide better feedback? like in: #assert:failureText: 'Expected %1 but got %2'.

At first sight, this is not so easy, because we need a place to store the arguments into, and since assert: just accepts a Block or simply the result of executing an expression, there is no such thing at hand (if it was easy, SUnit would have had such a feature from the beginning).

So the assertion needs to be at least a Block or an object which offers its arguments to the "feedback machine" as well.

The api might change from assert: to something like #assertExecuting: aBlockOrObject with: listOfArguments resultsIn: expectedResult
��
Typing this, I see that this is not nice, because you cannot just write some expression like self assert: myObject *3 = 15. You'd have to write assertExecuting: [:myObject| myObject * 3] with: 5 resultsIn: 15
Adding the feedback generation into makes the API even more complicated, because you'd have to ar least pass a String as a fourth parameter.
So we'd end up with:

assertExecuting: aBlock with: listOfArguments resultsIn: expectedResult otherwise: failureString.

In this form, I am sure nobody would use it. Or just for very special and important tests. So all that I can come up with for now is insufficient, unfortunately. Sure, there can be variants that provide some standard parameters for most of the arguments, but the base problem remains that the API is clumsy.

So what I am just showing is that what has happened with the assert: variants was the best thing one could possibly do without making SUnit heavy-weight and unpleasant to use.
Does it mean I am wrong? Does it mean it would be a good idea to put some more thought into this? Does it mean we should just go on and add more convenience methods? Does it mean we shouldn't change SUnit and keep it as it is?

I think it means we shouldn't just go on adding new convenience methods, but think about ways to improve its internals and make it more flexible in its feedback generation facilities.
Maybe we'll end up in something that you can use as a base implementation for today's #assert: method...

1. #assertCollection:hasSameElements: is a bad name, because it treats arguments as sets, not generic collections

it could be renamed or removed (no code in Pharo itself uses it (only Spec, but the use there is wrong))

let's put a check mark on this one. It was just an example

2. Providing selectors such as #assertCollection:equals: and similar just to get different description may not be the best idea, which leads to point 3

I think I made it clear that this will be a less than perfect solution that doesn't jump far enough

3. An easy way to extend printing/assertions/plugins; but what would that look like?

I am convinced this is the way to go. But I have no good answer to it. Just a spark for now.

Maybe

#assert: a = b description: [ Diff collectionDiffBetween: a and: b ]

Possibly the best thing we can come up with for now. By repeating the arguments in the feedback generation part, we keep the complexity out of the test runner's internals

or

#assert: a equals: b strategy: CollectionHasExactlyTheSameElementsAsAnotherAtTheSamePosition

See above. Better than another assert:-variation, but still too heavy

or

something else...

Peter

Joachim

-- 
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel          mailto:jtuchel@objektfabrik.de
Fliederweg 1�� �� �� �� �� �� �� �� �� �� �� �� ��http://www.objektfabrik.de
D-71640 Ludwigsburg                  http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0�� �� �� �� ��Fax: +49 7141 56 10 86 1