Fwd: exception handling problem when running tests in Nautilus
I realised I may have posted this on the wrong list. Is there anyone that can help with this one please? ---------- Forwarded message ---------- From: Otto Behrens <otto@finworks.biz> Date: Sun, Apr 19, 2015 at 10:08 AM Subject: exception handling problem when running tests in Nautilus To: pharo-users@lists.pharo.org Hi, Thanks everyone for the new Pharo and all the work done. I like Spotter! I gave Pharo 4 a try today and ran into a problem with running tests in a TestCase (test class -> run tests). I just get the debugger popping up with WARequestContextNotFound. In the setUp of our test case, we call WACurrentRequestContext value, but handle the error when we do not have a context, for example: [ WACurrentRequestContext value. true ] on: WARequestContextNotFound do: [ :e | false ] This handler: "[ :e | false ]" is never called. What influences this is the exception handling in: AbstractNautilusUI >> #runTestsOfClass:notifying: Firstly, I think that it may be slighly better to call "e pass" or "e outer" in stead of "e defaultAction", as this would give another outer handler a chance to do something. The defaultAction is called when there are no more outer handlers anyway. After trying this, it did not help my cause. And I think that this is because the WACurrentRequestContext is a Notification and not an Error. I then replaced Exception with Error in the handler in AbstractNautilusUI. This seem to work. What also works is to completely remove the exception handling in AbstractNautilusUI >> #runTestsOfClass:notifying: What is a bit strange is the inconsistency in this exception handling when running a suite or an individual test. Perhaps this kind of code could be better on the TestSuite and not in the Nautilus UI? Can anyone please help? Should I be doing something else? Cheers Otto
Indeed, Nautilus logic to run test differs a bit from the TestRunner logic. For example, if your test fails, it is run a second time to open a debugger but it could do side effects. Why don't you just run your tests from the test runner or using MyTestClass run: #testSelector ? We will need to clean this kind of logic that should not be part of Nautilus. Could you open a bug entry for that? Regards, Christophe. Le 21 avr. 2015 à 15:34, Otto Behrens a écrit :
I realised I may have posted this on the wrong list. Is there anyone that can help with this one please?
---------- Forwarded message ---------- From: Otto Behrens <otto@finworks.biz> Date: Sun, Apr 19, 2015 at 10:08 AM Subject: exception handling problem when running tests in Nautilus To: pharo-users@lists.pharo.org
Hi,
Thanks everyone for the new Pharo and all the work done. I like Spotter!
I gave Pharo 4 a try today and ran into a problem with running tests in a TestCase (test class -> run tests). I just get the debugger popping up with WARequestContextNotFound.
In the setUp of our test case, we call WACurrentRequestContext value, but handle the error when we do not have a context, for example:
[ WACurrentRequestContext value. true ] on: WARequestContextNotFound do: [ :e | false ]
This handler: "[ :e | false ]" is never called. What influences this is the exception handling in:
AbstractNautilusUI >> #runTestsOfClass:notifying:
Firstly, I think that it may be slighly better to call "e pass" or "e outer" in stead of "e defaultAction", as this would give another outer handler a chance to do something. The defaultAction is called when there are no more outer handlers anyway.
After trying this, it did not help my cause. And I think that this is because the WACurrentRequestContext is a Notification and not an Error.
I then replaced Exception with Error in the handler in AbstractNautilusUI. This seem to work. What also works is to completely remove the exception handling in AbstractNautilusUI >> #runTestsOfClass:notifying:
What is a bit strange is the inconsistency in this exception handling when running a suite or an individual test. Perhaps this kind of code could be better on the TestSuite and not in the Nautilus UI?
Can anyone please help? Should I be doing something else?
Cheers Otto
2015-04-21 16:03 GMT+02:00 Christophe Demarey <Christophe.Demarey@inria.fr>:
Indeed, Nautilus logic to run test differs a bit from the TestRunner logic.
We will need to clean this kind of logic that should not be part of Nautilus.
Just for reference: 13024 <https://pharo.fogbugz.com/default.asp?13024> Discrepancy between TestRunner and Nautilus test running for method #shouldNotImplementMethod 14864 <https://pharo.fogbugz.com/default.asp?14864> Nautillus do not launch the test cases on some classes 12851 <https://pharo.fogbugz.com/default.asp?12851> Test is executed twice when failling
Regards, Christophe.
Le 21 avr. 2015 à 15:34, Otto Behrens a écrit :
I realised I may have posted this on the wrong list. Is there anyone that can help with this one please?
---------- Forwarded message ---------- From: Otto Behrens <otto@finworks.biz> Date: Sun, Apr 19, 2015 at 10:08 AM Subject: exception handling problem when running tests in Nautilus To: pharo-users@lists.pharo.org
Hi,
Thanks everyone for the new Pharo and all the work done. I like Spotter!
I gave Pharo 4 a try today and ran into a problem with running tests in a TestCase (test class -> run tests). I just get the debugger popping up with WARequestContextNotFound.
In the setUp of our test case, we call WACurrentRequestContext value, but handle the error when we do not have a context, for example:
[ WACurrentRequestContext value. true ] on: WARequestContextNotFound do: [ :e | false ]
This handler: "[ :e | false ]" is never called. What influences this is the exception handling in:
AbstractNautilusUI >> #runTestsOfClass:notifying:
Firstly, I think that it may be slighly better to call "e pass" or "e outer" in stead of "e defaultAction", as this would give another outer handler a chance to do something. The defaultAction is called when there are no more outer handlers anyway.
After trying this, it did not help my cause. And I think that this is because the WACurrentRequestContext is a Notification and not an Error.
I then replaced Exception with Error in the handler in AbstractNautilusUI. This seem to work. What also works is to completely remove the exception handling in AbstractNautilusUI >> #runTestsOfClass:notifying:
What is a bit strange is the inconsistency in this exception handling when running a suite or an individual test. Perhaps this kind of code could be better on the TestSuite and not in the Nautilus UI?
Can anyone please help? Should I be doing something else?
Cheers Otto
Thanks. I do <alt>jc on a method sometimes when I have changed a method. So sometimes I so <alt>jm and then immediately <alt>jc to check all the tests in the class.
Indeed, Nautilus logic to run test differs a bit from the TestRunner logic.
Does it really have to be?
For example, if your test fails, it is run a second time to open a debugger but it could do side effects.
This has been a problem for me for a long time. I would *love* it if Nautilus runs a single test in debug mode (and not a second time). It is especially painful when running selenium tests in our environment, because firefox is launched on every test suite run, which makes running a single test relatively expensive. I suppose solving the problem of tests running a long time is first prize.
Why don't you just run your tests from the test runner or using MyTestClass run: #testSelector ?
Because I normally run the tests that I work on while I'm in the Nautilus browser (using shortcut keys). Why would I want to launch a different browser, search for the class I'm working on and run the tests there? I could be missing it totally here. I'm quite interested to know how other people work. A sort of work flow, to the point of running tests. To run all my tests take a long time. There are quite a few.
We will need to clean this kind of logic that should not be part of Nautilus. Could you open a bug entry for that?
https://pharo.fogbugz.com/f/cases/15374/exception-handling-when-running-test... Is this the right place?
Regards, Christophe.
Le 21 avr. 2015 à 15:34, Otto Behrens a écrit :
I realised I may have posted this on the wrong list. Is there anyone that can help with this one please?
---------- Forwarded message ---------- From: Otto Behrens <otto@finworks.biz> Date: Sun, Apr 19, 2015 at 10:08 AM Subject: exception handling problem when running tests in Nautilus To: pharo-users@lists.pharo.org
Hi,
Thanks everyone for the new Pharo and all the work done. I like Spotter!
I gave Pharo 4 a try today and ran into a problem with running tests in a TestCase (test class -> run tests). I just get the debugger popping up with WARequestContextNotFound.
In the setUp of our test case, we call WACurrentRequestContext value, but handle the error when we do not have a context, for example:
[ WACurrentRequestContext value. true ] on: WARequestContextNotFound do: [ :e | false ]
This handler: "[ :e | false ]" is never called. What influences this is the exception handling in:
AbstractNautilusUI >> #runTestsOfClass:notifying:
Firstly, I think that it may be slighly better to call "e pass" or "e outer" in stead of "e defaultAction", as this would give another outer handler a chance to do something. The defaultAction is called when there are no more outer handlers anyway.
After trying this, it did not help my cause. And I think that this is because the WACurrentRequestContext is a Notification and not an Error.
I then replaced Exception with Error in the handler in AbstractNautilusUI. This seem to work. What also works is to completely remove the exception handling in AbstractNautilusUI >> #runTestsOfClass:notifying:
What is a bit strange is the inconsistency in this exception handling when running a suite or an individual test. Perhaps this kind of code could be better on the TestSuite and not in the Nautilus UI?
Can anyone please help? Should I be doing something else?
Cheers Otto
Le 21 avr. 2015 à 16:32, Otto Behrens a écrit :
Thanks. I do <alt>jc on a method sometimes when I have changed a method. So sometimes I so <alt>jm and then immediately <alt>jc to check all the tests in the class.
Indeed, Nautilus logic to run test differs a bit from the TestRunner logic.
Does it really have to be?
No, it should not.
For example, if your test fails, it is run a second time to open a debugger but it could do side effects.
This has been a problem for me for a long time. I would *love* it if Nautilus runs a single test in debug mode (and not a second time). It is especially painful when running selenium tests in our environment, because firefox is launched on every test suite run, which makes running a single test relatively expensive. I suppose solving the problem of tests running a long time is first prize.
Why don't you just run your tests from the test runner or using MyTestClass run: #testSelector ?
Because I normally run the tests that I work on while I'm in the Nautilus browser (using shortcut keys). Why would I want to launch a different browser, search for the class I'm working on and run the tests there?
It was a suggestion for your use case. If you only want to run the test you are working on, Nautilus helps.
I could be missing it totally here. I'm quite interested to know how other people work. A sort of work flow, to the point of running tests.
We miss a functionality in Pharo to easily run (a shorcut) a set of selected tests (ex: your project tests). This way, you can have a quick feedback on your updates. As it is not so easy, I often fallback to run tests from Nautilus (a test class or sometimes only a test method).
To run all my tests take a long time. There are quite a few.
We will need to clean this kind of logic that should not be part of Nautilus. Could you open a bug entry for that?
https://pharo.fogbugz.com/f/cases/15374/exception-handling-when-running-test...
Yes.
We miss a functionality in Pharo to easily run (a shorcut) a set of selected tests (ex: your project tests). This way, you can have a quick feedback on your updates. As it is not so easy, I often fallback to run tests from Nautilus (a test class or sometimes only a test method).
This may be an idea: I once wrote a bit of Smalltalk code inspired by this: http://www.junitmax.com/ We ran it in our dev environment for about 2 years, and took it out when we upgraded to Pharo 3.0. I still miss it. Other members on the project do not, generally. When accepting a method, we ran some tests: 1. lint (on the method) 2. if the changed method is a test method, run it (debug) 3. if not, run at most 5 tests in the vicinity of the changed method (look for some appropriate senders) What we got for this: 1. TDD really beautiful. Accept a test method -> debugger pops up -> write the code in the debugger -> restart -> rinse and repeat. 2. Lint complained instantly. Very useful. As and when. 3. Change domain code: breaking test pops up -> edit code in the debugger! Problems we had: 1. Monkey patching OmniBrowser to do stuff when accepting a method. Work to figure out how in Nautilus :-( 2. Lint became slow. Lots of work to optimise and perhaps exclude some. 3. The responsiveness on accepting a method was just a slight bit slow which put people off 4. Editing in the debugger and lists of methods: no refactoring, no auto-formatting; pain Solutions: 1. Make it run in the background (continuously). Protect from running too often. 2. Get the method list browser and debugger editors to behave the same as Nautilus (at a method level) 3. Record information about failing tests, and run the ones that tend to fail more often 4. Use changes to help with "run tests for stuff that I am working on"
Hi otto we really want that the infrastructure of Pharo tools lets you define such process. So this is important that you - notify us - potentially contribute with code. Remember pharo is not just an external tool, this is your system. Le 21/4/15 18:29, Otto Behrens a écrit :
We miss a functionality in Pharo to easily run (a shorcut) a set of selected tests (ex: your project tests). This way, you can have a quick feedback on your updates. As it is not so easy, I often fallback to run tests from Nautilus (a test class or sometimes only a test method). This may be an idea: I once wrote a bit of Smalltalk code inspired by this:
We ran it in our dev environment for about 2 years, and took it out when we upgraded to Pharo 3.0. I still miss it. Other members on the project do not, generally.
When accepting a method, we ran some tests: 1. lint (on the method) Now we should have that. I always wanted to have that :)
2. if the changed method is a test method, run it (debug) 3. if not, run at most 5 tests in the vicinity of the changed method (look for some appropriate senders) Cool :)
What we got for this: 1. TDD really beautiful. Accept a test method -> debugger pops up -> write the code in the debugger -> restart -> rinse and repeat. 2. Lint complained instantly. Very useful. As and when. 3. Change domain code: breaking test pops up -> edit code in the debugger! :) It would be really nice to have a package with a setup like that. The tools should be open and customizable for such process. If you want to push this we will really help.
Problems we had: 1. Monkey patching OmniBrowser to do stuff when accepting a method. Work to figure out how in Nautilus :-( We should do a pass on nautilus but normally there should be an event for that. 2. Lint became slow. Lots of work to optimise and perhaps exclude some. We have false positive support and we can filter the rules. So with the work on uko if should be getting better and better.
3. The responsiveness on accepting a method was just a slight bit slow which put people off did you check recently because we did some bench like running all the rules on a method and it was under a second.
4. Editing in the debugger and lists of methods: no refactoring, no auto-formatting; pain We started to add refactorings in all the senders/implementors and others. Now if you see a place where there are missing, again send us the code.
Solutions: 1. Make it run in the background (continuously). Protect from running too often. 2. Get the method list browser and debugger editors to behave the same as Nautilus (at a method level) 3. Record information about failing tests, and run the ones that tend to fail more often 4. Use changes to help with "run tests for stuff that I am working on"
Le 21/4/15 17:32, Otto Behrens a écrit :
Thanks. I do <alt>jc on a method sometimes when I have changed a method. So sometimes I so <alt>jm and then immediately <alt>jc to check all the tests in the class.
Indeed, Nautilus logic to run test differs a bit from the TestRunner logic. Does it really have to be?
For example, if your test fails, it is run a second time to open a debugger but it could do side effects. This has been a problem for me for a long time. I would *love* it if Nautilus runs a single test in debug mode (and not a second time).
Me too :)
It is especially painful when running selenium tests in our environment, because firefox is launched on every test suite run, which makes running a single test relatively expensive. I suppose solving the problem of tests running a long time is first prize.
Why don't you just run your tests from the test runner or using MyTestClass run: #testSelector ? Because I normally run the tests that I work on while I'm in the Nautilus browser (using shortcut keys). Why would I want to launch a different browser, search for the class I'm working on and run the tests there?
I could be missing it totally here. I'm quite interested to know how other people work. A sort of work flow, to the point of running tests.
To run all my tests take a long time. There are quite a few.
We will need to clean this kind of logic that should not be part of Nautilus. Could you open a bug entry for that? https://pharo.fogbugz.com/f/cases/15374/exception-handling-when-running-test...
Is this the right place?
Regards, Christophe.
Le 21 avr. 2015 à 15:34, Otto Behrens a écrit :
I realised I may have posted this on the wrong list. Is there anyone that can help with this one please?
---------- Forwarded message ---------- From: Otto Behrens <otto@finworks.biz> Date: Sun, Apr 19, 2015 at 10:08 AM Subject: exception handling problem when running tests in Nautilus To: pharo-users@lists.pharo.org
Hi,
Thanks everyone for the new Pharo and all the work done. I like Spotter!
I gave Pharo 4 a try today and ran into a problem with running tests in a TestCase (test class -> run tests). I just get the debugger popping up with WARequestContextNotFound.
In the setUp of our test case, we call WACurrentRequestContext value, but handle the error when we do not have a context, for example:
[ WACurrentRequestContext value. true ] on: WARequestContextNotFound do: [ :e | false ]
This handler: "[ :e | false ]" is never called. What influences this is the exception handling in:
AbstractNautilusUI >> #runTestsOfClass:notifying:
Firstly, I think that it may be slighly better to call "e pass" or "e outer" in stead of "e defaultAction", as this would give another outer handler a chance to do something. The defaultAction is called when there are no more outer handlers anyway.
After trying this, it did not help my cause. And I think that this is because the WACurrentRequestContext is a Notification and not an Error.
I then replaced Exception with Error in the handler in AbstractNautilusUI. This seem to work. What also works is to completely remove the exception handling in AbstractNautilusUI >> #runTestsOfClass:notifying:
What is a bit strange is the inconsistency in this exception handling when running a suite or an individual test. Perhaps this kind of code could be better on the TestSuite and not in the Nautilus UI?
Can anyone please help? Should I be doing something else?
Cheers Otto
participants (4)
-
Christophe Demarey -
Nicolai Hess -
Otto Behrens -
stepharo