Ideally, we should not tag long tests. The system should be smart enough to characterize a test as long. If it takes more than 200ms, then it is long. Autotest should then offer me to include long test or not in the automatic test execution.

Using tags means that I have to go over each method test and tag them. This is a costly effort that is likely to not work in practice.

We have
-=-=-=-=-=-=-=-=-=-=-=-=
TestCase class>>newTestDictionary
�� �� �� ��^ Dictionary new at: #timeStamp put: TimeStamp now;
�� �� �� �� �� �� �� ��at: #passed put: Set new;
�� �� �� �� �� �� �� ��at: #failures put: Set new;
�� �� �� �� �� �� �� ��at: #errors put: Set new;
�� �� �� �� �� �� �� ��yourself
-=-=-=-=-=-=-=-=-=-=-=-=

Maybe it should be extended with
-=-=-=-=-=-=-=-=-=-=-=-=
�� �� �� �� �� �� �� ��at: #long put: Set new
-=-=-=-=-=-=-=-=-=-=-=-=

I guess the tweak in SUnit to identify long tests should not be that hard to implement.

Cheers,
Alexandre

On 26 Jul 2010, at 07:44, laurent laffont wrote:

>
> On Sat, Jul 24, 2010 at 10:16 AM, Alexandre Bergel <alexandre@bergel.eu> wrote:
> I have been using Autotest for a while already. I greatly reduce the window switching. This is cool.
> However, I have to disable Autotest when the tests take time to execute.
>
>
> An idea is to be able to tag tests with pragmas to put them in categories. So we can have a LongRunning category which is not run by default. Like NUnit (http://www.nunit.org/index.php?p=category&r=2.2).
>
> Then SUnit could be enhanced to show these categories. Useful to quickly select all tests related to a project for example; or run only performance tests.
>
>
> How would you name the tag ? Where the implementation should go ? (TestCase ?)
>
>
> Laurent.
>
>
> Alexandre
>
>
> On 15 Jun 2010, at 21:41, laurent laffont wrote:
>
> > Hi Alexandre,
> >
> > I've reorganized Autotest to write tests. The tests now run in a thread with lower priority. Thanks for feedback.
> >
> > Cheers,
> >
> > Laurent Laffont
> >
> > http://pharocasts.blogspot.com/
> > http://magaloma.blogspot.com/
> >
> >
> > On Mon, Jun 14, 2010 at 4:52 PM, Alexandre Bergel <alexandre@bergel.eu> wrote:
> > Hi Laurent,
> >
> > I tried to program while having AutoTest running.
> > More importantly than the interface, I experienced some problem with long executing tests. Basically, these tests should not be executed while I am programming. Or at least in a thread of a lesser priority. Am I the only one to experience this?
> >
> > Cheers,
> > Alexandre
> >
> >
> > On 11 Jun 2010, at 09:02, laurent laffont wrote:
> >
> > > On Fri, Jun 11, 2010 at 2:29 PM, Alexandre Bergel <alexandre@bergel.eu> wrote:
> > > Hi Laurent,
> > >
> > > I like Autotest. It is true that I always execute the test after modifying it.
> > > There are three horizontal panes. Why so? Is it just to open a debugger when necessary?
> > >
> > > I want to be able to open a debugger from autotest. I agree the GUI is crap now.
> > >
> > >
> > > We could imagine one standalone button instead with a green color to say the test I just edited is green, and yellow or red when it fails. In that case, clicking on the button open a debugger.
> > >
> > > I just feel the window of autotest takes a lot of space in the screen, without having a real benefit.
> > >
> > > Yes it's true. I'm still thinking on a good GUI. But I'm learning how to make GUI now :)
> > >
> > > If you look at Autotest package, AutotestView is just the GUI, so we can implement several GUI and see the best solution. I need to take the time to do it, repository is read / write so feel free to commit :)
> > >
> > > What I want to have is a dashboard docked on one side of the screen which acts like ��you're driving a car. It's always visible, you have to be able to look quickly at it as when you check the speed of your car, adjust your drive, ....
> > >
> > > Also another problem with the current version if that if a test fails, change it and fails again, you don't see it has run (nothing moves in the GUI).
> > >
> > > Finally, another problem is that you see that a test has failed, but you don't know why (SUnit TestRunner has the same problem). I want to display the exception message too.
> > >
> > > Thanks a lot for feedback.
> > >
> > > Laurent Laffont
> > >
> > > http://pharocasts.blogspot.com/
> > > http://magaloma.blogspot.com/
> > >
> > >
> > >
> > > Cheers,
> > > Alexandre
> > >
> > >
> > > On 3 Jun 2010, at 17:11, laurent laffont wrote:
> > >
> > > > Hi,
> > > >
> > > > I've written a proof-of-concept for Autotest. Draft/crappy code and no tests (exploration mode :) but it loads on PharoCore-1.1-11383-beta image.
> > > >
> > > > Load it:
> > > > Gofer new
> > > > �� �� �� squeaksource: 'Autotest';
> > > > �� �� �� package: 'Autotest';
> > > > �� �� �� load
> > > >
> > > > Open it:
> > > > AutotestView open
> > > > (there's ��en entry in WorldMenu > Tools)
> > > >
> > > > And change a tested method to see the results.
> > > >
> > > > There's a bug I need to find, maybe someone knows:
> > > > - Change Bag>>occurrencesOf:
> > > > - Autotest gives:
> > > >
> > > > 284 run, 281 passes, 0 expected failures, 1 failures, 2 errors, 0 unexpected passes
> > > > Failures:
> > > > CollectionRootTest>>#test0FixtureIterateTest
> > > >
> > > > Errors:
> > > > CollectionRootTest>>#testBasicCollect
> > > > CollectionRootTest>>#testDoWithout
> > > >
> > > > but in SUnit CollectionRootTest gives
> > > > 0 run, 0 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes ??
> > > >
> > > >
> > > > Cheers,
> > > >
> > > > Laurent Laffont
> > > >
> > > > http://pharocasts.blogspot.com/
> > > > http://magaloma.blogspot.com/
> > > >
> > > >
> > > > On Thu, Jun 3, 2010 at 6:09 PM, Alexandre Bergel <alexandre@bergel.eu> wrote:
> > > > The idea is excellent.
> > > >
> > > > Cheers,
> > > > Alexandre
> > > >
> > > >
> > > > On 3 Jun 2010, at 10:22, laurent laffont wrote:
> > > >
> > > > >
> > > > >
> > > > > On Thu, Jun 3, 2010 at 3:42 PM, Alexandre Bergel <alexandre@bergel.eu> wrote:
> > > > > > You may have a lot of noise.
> > > > > >
> > > > > > I guess that Ruby uses files as a unit of development/deployment. The closest Smalltalk/Pharo has is the class and the package.
> > > > > >
> > > > > > I would suggest that TestCase which would use this feature use some pragma/method to identify/declare which classes/packages this test depends upon. Then the "autotest" framework would register such tests and listen for changes in the given classes/packages, launching required tests whenever a change happen.
> > > > > >
> > > > > > Additionally, one could declare such a pragma on a single test method, when this test should be run for a specific class.
> > > > > >
> > > > > > Of course, you also to take care of long running tests, which you probably want to exclude from auto-testing.
> > > > >
> > > > > I see autotest in Pharo in a slighly different way: When I press save in the Monticello browser, I have a popup menu which asks me whether (i) I want to run all the tests or (ii) only the tests that cover that I changed from the last version.
> > > > >
> > > > > Does this make sense?
> > > > >
> > > > > Please no popup :) What I like in ruby autotest is that I can quickly look at test results if I want (or not) without stop writing. Often you want to see your tests failing, as you type / save code. I don't have to stop writing, click a button, wait test results, go again.... testing is done in background and I just see notifications whether it's OK or not.
> > > > >
> > > > > So test log in a Transcript is OK for me.
> > > > >
> > > > >
> > > > > For autotest unit of work is file: it runs the test file which has the same name as the code file, but you can customize this behavior. For autotest-rails:
> > > > > "A simplified version of Autotest heuristics in this mode would be:
> > > > > When changing a test file, only this file is run (e.g. test/unit/foo_test.rb ���test/unit/foo_test.rb).
> > > > > When changing a model file, only associated unit test file is run (e.g.app/models/foo.rb ��� test/unit/foo_test.rb).
> > > > > When changing a controller file, associated functional test file is run (e.g.app/controllers/foo_controller.rb ���test/functional/foo_controller_test.rb).
> > > > > When changing a fixture file, associated unit test and functional test are run (e.g.app/fixtures/foos.yml ��� test/unit/foo_test.rb +test/functional/foo_controller_test.rb).
> > > > > When changing a helper file, associated functional test file is run (e.g.app/helpers/foo_helper.rb ���test/functional/foo_controller_test.rb).
> > > > > When changing application_helper.rb file all functional test files are run (e.g.application_helper.rb ��� test/functional/*_test.rb).
> > > > > When changing a file under the config directory, all tests are run."
> > > > >
> > > > > Laurent
> > > > >
> > > > >
> > > > >
> > > > > Cheers,
> > > > > Alexandre
> > > > > --
> > > > > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
> > > > > Alexandre Bergel ��http://www.bergel.eu
> > > > > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Pharo-project mailing list
> > > > > Pharo-project@lists.gforge.inria.fr
> > > > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> > > > >
> > > > > _______________________________________________
> > > > > Pharo-project mailing list
> > > > > Pharo-project@lists.gforge.inria.fr
> > > > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> > > >
> > > > --
> > > > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
> > > > Alexandre Bergel ��http://www.bergel.eu
> > > > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Pharo-project mailing list
> > > > Pharo-project@lists.gforge.inria.fr
> > > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> > > >
> > > > _______________________________________________
> > > > Pharo-project mailing list
> > > > Pharo-project@lists.gforge.inria.fr
> > > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> > >
> > > --
> > > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
> > > Alexandre Bergel ��http://www.bergel.eu
> > > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
> > >
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Pharo-project mailing list
> > > Pharo-project@lists.gforge.inria.fr
> > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> > >
> > > _______________________________________________
> > > Pharo-project mailing list
> > > Pharo-project@lists.gforge.inria.fr
> > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >
> > --
> > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
> > Alexandre Bergel ��http://www.bergel.eu
> > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Pharo-project mailing list
> > Pharo-project@lists.gforge.inria.fr
> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >
> > _______________________________________________
> > Pharo-project mailing list
> > Pharo-project@lists.gforge.inria.fr
> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
>
> --
> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
> Alexandre Bergel ��http://www.bergel.eu
> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
>
>
>
>
>
>
> _______________________________________________
> Pharo-project mailing list
> Pharo-project@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> Pharo-project@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel ��http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.






_______________________________________________
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project