Hi adrian I like the idea of cleaning. Now here is what I would like - get the same API (I think that sharing code amongst smalltalk is a too large constraint but having the same API is important) - integrate key features of extensions so that SUNIT and SUNITExtended is cooler.
Niko and me are thinking about doing some work on SUnit.
Background is that we are working on Phexample, which extends SUnit with test that expand on one another [1]. We got test dependencies working with the current SUnit, but had to do some quick ugly hacks. So we thought about cleaning the internals SUnit up, and then include test dependencies in the core.
Things we would possible change (we will certainly identify more as we clean up SUnit...)
- store test resources in a dynamic variable rather than a static one
for this one I would not rely on dynamic variables. Keep it for later. And you will have to really argue. Static scoping is nice.
- refactor the assert: protocol to a trait, such that they can be used in setup and teardown of test resources as well (actually I already did that, you can find a version on my squeaksource account)
:) Easy I'm biaised :)
- apply the fixes to test resources in Niall's slide deck that Lukas showed us
Pleaseeeeeeeee. I would really like to also - check what keith did on his Sunit extensions - have expected failures - have the possibility to know whether a test was red or green when it was saved and now when I run it, is it my fault or what is just before like that
- add expectations matchers as used in behavior-driven testing Please not keep this separated or in another package. I do not like this "bla be something" kind of violation of law of demeter
- break testcase into two classes, and to run the test and one to store the result (no more #cleanUpInstanceVariables hack to avoid memory leaks)
what would be the impact on existing subclasses?
- add test dependencies to test results, such that one test can extend on the return value of another test I originally did not like the idea but I can be convinced easily :) I think that if it can help other without getting in our way then this is cool.
- fix expected failures such that they are a first-class result (no more #shouldPass with O(n^2) or worse behavior)
So you see, the idea is twofold: first to clean up what is there, and second to add test dependencies as a new feature. For those that dont know test dependencies. They had first been proposed by Markus Gälli based on an analysis of all Squeak tests years ago. He found that most tests cover a superset of another test's coverage set. A prototype in Java as been implemented later by Lea Hänsenberger and me. We published a case study at XP 2008, which lead to PHP folks to include test dependencies in the latest PHPUnit. The idea of explicit test dependencies is to shift the burden of isolating test cases from the developers to the framework. You avoid duplicate test code and you get less red test cases per failure. Please refer to Niko's blog post for more details [2].
We got some more ideas (such as API baseline testing, search in test results, etc...) but they will not likely make it into a first clean up.
Yes clean up first!
One thing we do not know is how many folks are out there that depend on internal representation of SUnit. I talked to some folks and identified two requirements, first that legacy tests should keep running and second that contributions to other sunit forks should keep being mergeable. The first seems feasible, the second sounds like it boils down to not cleaning anything :)
Merging does not mean that this is done automatically. I think that having a good testing framework is important. Now good does not mean that it has a lot of not that useful features. I suggest the following - clean and get a strong core - make sure that all the tests still run - get a set of extensions that people can load
What do you think?
I'm interested :) but people will have to discuss your proposal :) Stef
cheers, AA
[1]: http://www.squeaksource.com/phexample.html [2]: http://smalltalkthoughts.blogspot.com/2009/11/phexample-because-examples-exp...
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project