+100 Couldn't have said it better myself! -- Cheers, Peter. On 27 feb 2011, at 08:27, laurent laffont <laurent.laffont@gmail.com> wrote:

On Sun, Feb 27, 2011 at 4:52 AM, Schwab,Wilhelm K <bschwab@anest.ufl.edu> wrote: Hello all,

Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source.

If you develop the TDD way (which I always do at work) then I would say this question is not valid. Your production code will be there because of tests. In legacy system, new and refactored code will be there because of tests.

IMHO the questions "how much time should I spend writing tests?" or "how many tests should I write ?" are raised when tests are written after. And sadly tests often become a burden in this case and doesn't support the pressure of releases / last minute changes / ... (I've done it this way almost 10 years ago, not good :)

In TDD the question is "which stories / scenarios do we have to implement ?". It's more a Product owner / customer point of view.

The best book on TDD I've read is Dave Astels TDD: a practical guide (Indeed, this one has really changed my mind on how testing should be done, and other books I've read since don't).

Laurent

Bill

On Sun, Feb 27, 2011 at 2:12 PM, Schwab,Wilhelm K <bschwab@anest.ufl.edu>wrote:

Understood. But, doing it correctly, how much of your time do you spend on the test writing/maintenance side?

None. Seriously none. I never pause and say "OK now it's time to write tests and maintain existing ones". It's part of the TDD cycle: write a test, make it green, refactor. write a test, make it green, refactor. The cycle can go from 1mn to 10mn, rarely more. And if you use the Pomodoro technique, you feel it even better. Test writing/maintenance is part of programming. Sometimes you can say "OK before implementing this new story, I would like to refactor here and here (tests + code) because I don't like it". We also do this stuff on legacy code when tests are not here. But it's far easier when we have good test coverage. I found some blog posts, but I was hoping for some peer-reviewed papers that

go on the record.

I am trying to make the point you are making, and it will go over better if I can point to literature and some numbers that I suspect will be seen as sobering. Tests ARE being "written after" in this case, but assuming they change to a proper process, they still need to understand that tests take thought and work.

When tests are written after, it's another kind of story. Then "how much time we should spend writing test" is valid. I don't have any answer. TDD is not easy. Learning to write good tests takes time (years), investment. If someone in your team have good experience with it, he should pair program continuously with other programmers. Participating in coding-dojo , code retreats with experienced developers is great too. Laurent.

________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [ pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Peter Hugosson-Miller [oldmanlink@gmail.com] Sent: Sunday, February 27, 2011 2:58 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing?

+100

Couldn't have said it better myself!

-- Cheers, Peter.

On 27 feb 2011, at 08:27, laurent laffont <laurent.laffont@gmail.com <mailto:laurent.laffont@gmail.com>> wrote:

On Sun, Feb 27, 2011 at 4:52 AM, Schwab,Wilhelm K <<mailto: bschwab@anest.ufl.edu>bschwab@anest.ufl.edu<mailto:bschwab@anest.ufl.edu>> wrote: Hello all,

Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source.

If you develop the TDD way (which I always do at work) then I would say this question is not valid. Your production code will be there because of tests. In legacy system, new and refactored code will be there because of tests.

IMHO the questions "how much time should I spend writing tests?" or "how many tests should I write ?" are raised when tests are written after. And sadly tests often become a burden in this case and doesn't support the pressure of releases / last minute changes / ... (I've done it this way almost 10 years ago, not good :)

In TDD the question is "which stories / scenarios do we have to implement ?". It's more a Product owner / customer point of view.

The best book on TDD I've read is Dave Astels TDD: a practical guide (Indeed, this one has really changed my mind on how testing should be done, and other books I've read since don't).

Laurent

Bill

Sure. But the audience is a group that bangs out code and then fixes it later; I am trying to help them past that, and something quantitative in an ACM or IEEE journal would be harder to shoot down that our advice, not matter how correct it might be. ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Ramon Leon [ramon.leon@allresnet.com] Sent: Sunday, February 27, 2011 11:26 AM To: pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing? On 02/27/2011 06:12 AM, Schwab,Wilhelm K wrote:

Understood. But, doing it correctly, how much of your time do you spend on the test writing/maintenance side?

Far less time than you'll spend manually testing over the course of writing and maintaining the program. You can't avoid testing, nor the time spent doing it, the only question is do want to do it manually and haphazardly with fingers crossed hoping you didn't break anything every publish, or do it with automated tests so you know you didn't break anything every time. -- Ramon Leon http://onsmalltalk.com

Norbert, Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true. I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems. I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests? Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Norbert Hartl [norbert@hartl.name] Sent: Sunday, February 27, 2011 5:41 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing? Hi, On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote:

Hello all,

Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source.

I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation. So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative. I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time. The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :) Norbert

I've asked the question three times because people seem to be trying to convince me (I'm on your side of this) rather than my audience. Your idea of how much test code gets written would be a fine answer to have. One of the points I am going to make in the presentation is that tests are code too (yes, they apparently need to hear that). If expectations were realistic, I would not need numbers. ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Yanni Chiu [yanni@rogers.com] Sent: Sunday, February 27, 2011 11:19 AM To: pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing? On 27/02/11 7:58 AM, Schwab,Wilhelm K wrote:

I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests?

You've asked the question for the third time, so all the TDD explanations seem to have missed the mark. How about asking instead: how much test code do people write? Maybe this way, both the TDD and the "how much time" viewpoints can be accommodated (because volume of test code is roughly proportional to time to write test code). -- Yanni Chiu

I've always felt that "test driving" code actually results in negative time spent on "testing". I spend (and everyone I know that tests well does as well) a lot less time writing code test-first than I ever would writing the same code without the tests first. Also, I spend far less time (almost none) manually testing functionality. So TDD results in net negative time difference. (Apologize if I'm incoherent, the cold and flu drugs are strong in this one.) -Steven On Sun, Feb 27, 2011 at 11:51 AM, Norbert Hartl <norbert@hartl.name> wrote:

On 27.02.2011, at 13:58, Schwab,Wilhelm K wrote:

...

Norbert,

Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true.  I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems.

It seems I wasn't clear on this one. I'm trying to making a point that there is no distinction between "testing" - "no testing" but between "testing" - "writing tests". If you take compilation you test the code for syntax and language quirks. If a developer runs the program he develops than he is testing already. Testing and running a program is the same form that point of view. He takes input parameter and expects output parameter. That is testing. The point is just that developers do it manually and that is not reproducable. So there is no real difference between manual testing all the time and written test case. Only that repeating the testing procedure is boring and therefor supposed to be done by a machine. So if developers could see that the just need to put the work they already doing into a test will ease the work without changing much. They are just changing style and save time. That's it. Talking about 100% test coverage and holistic views about what the perfect testing could be doesn't help.

...

I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests?

I understand your intention but the view is inapropriate. Coding and testing aren't two things. It is something that belongs together (if you changed coding style). So that's why it is hard to estimate a time that should be spent. To me it is more comparabale to this: We all know collections are great. Imagine someone asking you "Can anyone recommend a good reference on the amount of time one should expect to spend writing code that uses collections?". What would you say? There is no answer. That doesn't mean you can't solve the problem. But you won't solving it by saying "It is 38.345 % of the time". If people don't get it you have to convince and/or teach them. I had several times where I could show the developer that he is gaining time from doing it. That works. Everything else is targeted towards excel manipulators.

Norbert

...

________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Norbert Hartl [norbert@hartl.name] Sent: Sunday, February 27, 2011 5:41 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing?

Hi,

On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote:

...

Hello all,

Can anyone recommend a good reference on the amount of time one should expect to spend writing tests?  I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source.

I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation.

So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative.  I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time.

The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :)

Norbert

Oh! I never worry about "proving" it. When I've worked for clients who have insisted that I leave the testing out because they don't want to "pay extra" for the time spent, I simply don't push the tests to their repo. I test for myself, because I have a personal commitment to do the best job possible. -Steven On Sun, Feb 27, 2011 at 12:11 PM, Peter Hugosson-Miller <oldmanlink@gmail.com> wrote:

Steven, you're making perfect sense to me, and I think almost everyone here agrees with you.

Bill's problem (if I've understood him correctly) is that he needs to convince some pointy-haired-boss-type person, by directing him or her to a well-respected "official" statistic that "proves" what we all know to be true from experience. Sadly, I think it will be hard to find it :-(

-- Cheers, Peter

On Sun, Feb 27, 2011 at 9:04 PM, Steven Baker <steven@stevenrbaker.com> wrote:

...

I've always felt that "test driving" code actually results in negative time spent on "testing". I spend (and everyone I know that tests well does as well) a lot less time writing code test-first than I ever would writing the same code without the tests first. Also, I spend far less time (almost none) manually testing functionality. So TDD results in net negative time difference.

(Apologize if I'm incoherent, the cold and flu drugs are strong in this one.)

-Steven

On Sun, Feb 27, 2011 at 11:51 AM, Norbert Hartl <norbert@hartl.name> wrote:

...

On 27.02.2011, at 13:58, Schwab,Wilhelm K wrote:

...

Norbert,

Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true.  I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems.

It seems I wasn't clear on this one. I'm trying to making a point that there is no distinction between "testing" - "no testing" but between "testing" - "writing tests". If you take compilation you test the code for syntax and language quirks. If a developer runs the program he develops than he is testing already. Testing and running a program is the same form that point of view. He takes input parameter and expects output parameter. That is testing. The point is just that developers do it manually and that is not reproducable. So there is no real difference between manual testing all the time and written test case. Only that repeating the testing procedure is boring and therefor supposed to be done by a machine. So if developers could see that the just need to put the work they already doing into a test will ease the work without changing much. They are just changing style and save time. That's it. Talking about 100% test coverage and holistic views about what the perfect testing could be doesn't help.

...

I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests?

I understand your intention but the view is inapropriate. Coding and testing aren't two things. It is something that belongs together (if you changed coding style). So that's why it is hard to estimate a time that should be spent. To me it is more comparabale to this: We all know collections are great. Imagine someone asking you "Can anyone recommend a good reference on the amount of time one should expect to spend writing code that uses collections?". What would you say? There is no answer. That doesn't mean you can't solve the problem. But you won't solving it by saying "It is 38.345 % of the time". If people don't get it you have to convince and/or teach them. I had several times where I could show the developer that he is gaining time from doing it. That works. Everything else is targeted towards excel manipulators.

Norbert

...

________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Norbert Hartl [norbert@hartl.name] Sent: Sunday, February 27, 2011 5:41 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing?

Hi,

On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote:

...

Hello all,

Can anyone recommend a good reference on the amount of time one should expect to spend writing tests?  I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source.

I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation.

So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative.  I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time.

The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :)

Norbert

Let's keep the physical descriptions out of it, ok :) But you have the point: I am looking for published numbers to back up what we all have learned from experience. And it has been surprisingly hard to find. ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Peter Hugosson-Miller [oldmanlink@gmail.com] Sent: Sunday, February 27, 2011 3:11 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing? Steven, you're making perfect sense to me, and I think almost everyone here agrees with you. Bill's problem (if I've understood him correctly) is that he needs to convince some pointy-haired-boss-type person, by directing him or her to a well-respected "official" statistic that "proves" what we all know to be true from experience. Sadly, I think it will be hard to find it :-( -- Cheers, Peter On Sun, Feb 27, 2011 at 9:04 PM, Steven Baker <steven@stevenrbaker.com<mailto:steven@stevenrbaker.com>> wrote: I've always felt that "test driving" code actually results in negative time spent on "testing". I spend (and everyone I know that tests well does as well) a lot less time writing code test-first than I ever would writing the same code without the tests first. Also, I spend far less time (almost none) manually testing functionality. So TDD results in net negative time difference. (Apologize if I'm incoherent, the cold and flu drugs are strong in this one.) -Steven On Sun, Feb 27, 2011 at 11:51 AM, Norbert Hartl <norbert@hartl.name<mailto:norbert@hartl.name>> wrote:

On 27.02.2011, at 13:58, Schwab,Wilhelm K wrote:

...

Norbert,

Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true. I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems.

It seems I wasn't clear on this one. I'm trying to making a point that there is no distinction between "testing" - "no testing" but between "testing" - "writing tests". If you take compilation you test the code for syntax and language quirks. If a developer runs the program he develops than he is testing already. Testing and running a program is the same form that point of view. He takes input parameter and expects output parameter. That is testing. The point is just that developers do it manually and that is not reproducable. So there is no real difference between manual testing all the time and written test case. Only that repeating the testing procedure is boring and therefor supposed to be done by a machine. So if developers could see that the just need to put the work they already doing into a test will ease the work without changing much. They are just changing style and save time. That's it. Talking about 100% test coverage and holistic views about what the perfect testing could be doesn't help.

...

I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests?

I understand your intention but the view is inapropriate. Coding and testing aren't two things. It is something that belongs together (if you changed coding style). So that's why it is hard to estimate a time that should be spent. To me it is more comparabale to this: We all know collections are great. Imagine someone asking you "Can anyone recommend a good reference on the amount of time one should expect to spend writing code that uses collections?". What would you say? There is no answer. That doesn't mean you can't solve the problem. But you won't solving it by saying "It is 38.345 % of the time". If people don't get it you have to convince and/or teach them. I had several times where I could show the developer that he is gaining time from doing it. That works. Everything else is targeted towards excel manipulators.

Norbert

...

________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr> [pharo-project-bounces@lists.gforge.inria.fr<mailto:pharo-project-bounces@lists.gforge.inria.fr>] On Behalf Of Norbert Hartl [norbert@hartl.name<mailto:norbert@hartl.name>] Sent: Sunday, February 27, 2011 5:41 AM To: Pharo-project@lists.gforge.inria.fr<mailto:Pharo-project@lists.gforge.inria.fr> Subject: Re: [Pharo-project] Good reference on time on unit testing?

Hi,

On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote:

...

Hello all,

Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source.

I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation.

So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative. I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time.

The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :)

Norbert

Norbert, Have you ever read of shops where people throw code at features and never much bother to test what they are doing, because it is "testing's" job to catch the bugs ? I had read about it too... I understand what you are saying. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Norbert Hartl [norbert@hartl.name] Sent: Sunday, February 27, 2011 2:51 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing? On 27.02.2011, at 13:58, Schwab,Wilhelm K wrote:

Norbert,

Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true. I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems.

It seems I wasn't clear on this one. I'm trying to making a point that there is no distinction between "testing" - "no testing" but between "testing" - "writing tests". If you take compilation you test the code for syntax and language quirks. If a developer runs the program he develops than he is testing already. Testing and running a program is the same form that point of view. He takes input parameter and expects output parameter. That is testing. The point is just that developers do it manually and that is not reproducable. So there is no real difference between manual testing all the time and written test case. Only that repeating the testing procedure is boring and therefor supposed to be done by a machine. So if developers could see that the just need to put the work they already doing into a test will ease the work without changing much. They are just changing style and save time. That's it. Talking about 100% test coverage and holistic views about what the perfect testing could be doesn't help.

I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests?

I understand your intention but the view is inapropriate. Coding and testing aren't two things. It is something that belongs together (if you changed coding style). So that's why it is hard to estimate a time that should be spent. To me it is more comparabale to this: We all know collections are great. Imagine someone asking you "Can anyone recommend a good reference on the amount of time one should expect to spend writing code that uses collections?". What would you say? There is no answer. That doesn't mean you can't solve the problem. But you won't solving it by saying "It is 38.345 % of the time". If people don't get it you have to convince and/or teach them. I had several times where I could show the developer that he is gaining time from doing it. That works. Everything else is targeted towards excel manipulators. Norbert

________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Norbert Hartl [norbert@hartl.name] Sent: Sunday, February 27, 2011 5:41 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing?

Hi,

On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote:

...

Hello all,

Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source.

I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation.

So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative. I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time.

The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :)

Norbert

Cesar, Alexandre, Thanks. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of csrabak@bol.com.br [csrabak@bol.com.br] Sent: Sunday, February 27, 2011 2:24 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Good reference on time on unit testing? Yes, thus for Bill this mean that for amount of time already expended you need about the same amount for writing tests. YMMV if the cyclomatic complexity (or similar metric) is higher of lower than 'average' programs. In a more commercial scenario, you can cite Gartner's Research Note SPA-19-3568 where they put 10 to 15% of total development time for unit testing and 10 to 15% time for system [integration] testing. These numbers vary a lot, but you can find good references in Software Engineering books or periodicals. HTH -- Cesar Rabak Em 27/02/2011 11:28, Alexandre Bergel < alexandre.bergel@me.com > escreveu: Robert Martin said in his book that one should spend ~50% of its time writing tests. Cheers, Alexandre On 27 Feb 2011, at 00:52, Schwab,Wilhelm K wrote:

Hello all,

Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source.

Bill

-- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.