This is one of the reasons why I don't make testing into a religion but pick a time and a place to add the tests. The 'write your tests first' school is right in that it can create another boost but it comes with the problem that it puts a damper on any exploratory programming. I typically take three tries to get it 'right', and only after the third try with a reasonably stable interface is where I will add tests.
Maybe I'm a lousy designer in that I can't nail down an interface on the first try but on non-trivial code this is unfortunately my experience so far.
First try: very quick, rough and dirty. Second try: a bit slower, mostly right but still needs major rework. Third try: minor tweaks in the longer term but mostly finalized.
The elapsed time is usually something under a day for the first try, a few days to a week for the second and after the third try it goes into 'maintenance' pretty quickly.
I think that what you say makes a lot of sense and I completely agree with doing code-test-code. True TDD (which I have tried) requires too much constant change to be practical.
The churn I was referring to though is less that of short-term interface changes but more to do with long-term multi-month revamping of the product proposition. During these highly exploratory phases, the main risk with errors is not that they multiply but that they never happen because the code isn't run.
I always knew this in theory but like all lessons I needed to learn it the hard way to really appreciate.
This makes me wonder even more though when you put it together with this part of your post:
...I have good hope that in the long run this will tremendously increase my productivity. Even if in the short term it seemed as though my productivity took a hit (after all, writing code that tests stuff does not add any functionality to the program itself).
You seem to be saying that even though you've been doing it for a year, you're still suffering a drop in productivity. Have I misunderstood you? Doesn't that indicate that it's actually not a good idea? Or do you have evidence that there's less customer visible bugs to deal with? (evidence rather than your hopes would be better :)
It doesn't really feel like you've communicated why all that extra testing code is so great. And let's not forget that's what it actually is, code which can be wrong itself. And you're not even doing TDD but adding them way down the line!
There definitely is a drop in initial productivity, after the exploratory phase is over there is a hick-up in the output because I'm writing test cases. Then, as soon as that is done there is a huge boost in productivity because it becomes much (and I can't emphasize enough how much that really is) easier.
So it is more a matter of uneven output than an overall loss, overall it is definitely a net gain. Hard to quantify though because you will never know how much time you would have spent debugging and testing if you had not written those tests first.
It is more of a feeling than that it is measurably slower, in fact it is measurably faster but when writing the test cases it feels slower. I hope that clears it up, sorry for the confusion!
As for fewer customer visible bugs: yes, absolutely, no doubt about that.
The 'write your tests first' school is right in that it can create another boost but it comes with the problem that it puts a damper on any exploratory programming.
As far back as 1999 I can remember the "write your tests first" people saying "... unless you're doing exploratory programming and are likely to throw away the prototype."
> why I don't make testing into a religion but pick a time and a place to add the tests
Quite. Testing is great. "Test Driven Development" where expensive Agile Consultants who haven't written code in ten years tell teams they always have to write tests first for everything is bullshit: http://xrl.us/bmebj3
Many successful devs also preach TDD. TDD is great when your client has a high quality spec, or you are being paid for meeting team spec. TDD is less great when your project goal is "build something neat" with a very loose spec and wide tolerances.
Well it really depends. You can make a religious Process out of TDD where you start with something like Cucumber and work your way down through the design. You can TDD your glue code. You can TDD trivial stuff, like when you put a link in your web app to your new feature, you could write a view test that verifies that the link appears. You can, if you really want to, go through an obnoxious iterative process of writing the same function in a progressively less and less broken state. There might even be value to this approach. I wouldn't know.
Or you could do something like this: whenever you have a function that seems to do an actual algorithm, or process some data, or provide a predictable result given an input, you write up a few asserts and programmatically see whether it seems to work immediately after writing it instead of writing it and wasting your time futzing about with it by hand, or tracking down a bug in it after noticing strange results coming out of the whole program.
Or you could do something like this: whenever you get a bug report, create a test case that reproduces the bug first.
Agile Consultants are like anyone else who tries to sell you enlightenment: they wouldn't eat unless there was actually a kernel to truth behind what they were saying. Writing your tests first is a fantastic idea sometimes.
Even if the components built of units change, and thus trach their tests, won't the tests covering the units still be useful?
Also: I find that a lot of component changes are interface changes -- the test code has to change to match the new interface and return values, but still remains useful.
This is one of the reasons why I don't make testing into a religion but pick a time and a place to add the tests. The 'write your tests first' school is right in that it can create another boost but it comes with the problem that it puts a damper on any exploratory programming. I typically take three tries to get it 'right', and only after the third try with a reasonably stable interface is where I will add tests.
Maybe I'm a lousy designer in that I can't nail down an interface on the first try but on non-trivial code this is unfortunately my experience so far.
First try: very quick, rough and dirty. Second try: a bit slower, mostly right but still needs major rework. Third try: minor tweaks in the longer term but mostly finalized.
The elapsed time is usually something under a day for the first try, a few days to a week for the second and after the third try it goes into 'maintenance' pretty quickly.