Unit Testing Is Boring

Unit testing is really really boring. It is waste of time and productivity, writing unit-tests is hard, takes long and might require time-taking refactorings to the code for it to be unit-testable. Regression testing is anyway a lot more effective in finding mistakes. Besides, unit testing provides very little value as it doesn’t tell anything about the quality of the software. Adding unit tests to code which has already been proven working by functional testing is just stupid.

That may sound a bit exaggerated, but I’m almost serious here. There are times when adding unit tests is really waste of time. There’s a lot of code which is hard or near impossible to unit test properly. But the last part about code being proven working by functional testing is in my opinion bullshit. Functional testing tests functionality and can prove that desired functionality has been achieved, but it cannot prove that the code does what the developer intended it to do. And no matter how much functional tests you add on top of your software, it doesn’t make the code any more maintainable.

Let me tell you  how I define testing and unit testing.

Testing gives quality information about software. Qualities being e.g. functionality, usability, reliability, performance etc. We do testing in order to determine the current quality level of a certain software. This information can be further used to make decisions e.g. whether to release the software now or later (or never which is a brilliant decision in some cases).

Unit testing does not give you any quality information about the software or it’s functionality, but it does give you information about the  quality the code. There is a difference. Unit testing, if done properly, can prove that the code does exactly what the developer intended it to do. Of course there might be mistakes made when deciding what the code should do, but that’s a different story. Unit testing also makes code easier to maintain and further develop.

To be able to unit test code, the code needs to be unit testable. This usually also means the code is cleaner (clean as in Clean Code) which makes it easier to understand and change the code. And when the code is covered with good set of unit tests, it is also safe to change the code. Every code written has a price. There’s a price to develop the code, but there’s (often even larger) price to maintain the code. Clean code which is covered with unit tests has a smaller price tag than code which is not that clean and/or is not covered with unit tests.

Adding unit tests to code is boring. This is true. And if the code will not be further developed, it is complete waste of time as well. Unit tests do not add much (or any) value when written afterwards, specially if long time has passed since the code to be tested was written. As said, adding unit tests to code which was not written to be unit testable in the first place is laborious and often requires big refactorings to be made before unit testing can take place. And as there are no unit tests to cover poor developer’s ass, those refactorings might break things really badly. Luckily there are good old books to help in tasks like this. Do read Martin Fowler’s Refactoring and Michael C. Feather’s Working Effectively with Legacy Code.

So, this far we’ve learned that unit tests often don’t add any value, are difficult to write and the whole thing is also really boring. Forget unit tests completely? No, I’ve found a nice solution which you’ve probably heard earlier as well. It’s called test-first programming or test-driven development (TDD). I personally like most the term red/green/refactor. After I learned to do TDD (as if I meant it) I’ve enjoyed writing code a lot more. Nowadays I don’t often get to write code, but when I do, I use TDD and I love it.

And even if you didn’t like TDD or don’t want to try it (if this is so, I’d like to know why), you can still add the unit tests while you’re writing your code. Then you must think about unit testability before you write the first line of production code. Code design must be done so that adding unit tests is easy.

As a final note, consider Martin Fowler’s words and think about this the next time you’re writing code:

Whenever you are tempted to type something into a print statement or a debugger expression, write it as a test instead.

Advertisements

Get the Most Out of Static Code Analysis

Static code analysis is a cheap way to improve code quality. One might argue with this by stating that tools like Klocwork and Coverity are actually fairly expensive. And here comes one common misconception related to static analysis, some people think it’s about tools while it’s not. Sure good tools help, but there are a lot more important issues related to this. There are several problems related in doing static code analysis and only some of those might be solved with proper tools.

Problems / Solutions

Problem: Too many problems reported

Amount of reported problems is devastating and developers feel overwhelmed. As a result, the problems aren’t fixed. Sometimes a large number of reported problems are actually not real problems but e.g. style issues or even false positives. Good tools help in this but truth is all of them require manual work in tuning the results.

Solution: Define baseline and focus only on new problems

Measure the number of problems reported and define that as a baseline. Then make sure that no new problems are introduced. This ensures that the codebase doesn’t at least get worse. Later on its possible to tighten up the thresholds to get rid of older reported problems as well.

Problem: Tools are not used

It doesn’t really help if you have the best tools money can get but nobody’s using them.

Solution: Run tools automatically

If you’re using continuous integration system such as Jenkins you can run static analysis for every commit. In case the analysis takes long time, run it at least once a day (or night). Make sure all committers are being informed should the analysis reveal new problems. In my opinion, if static analysis reveals new problems, it should be considered a stop-the-line situation. So stop everything, fix problems, and only then continue.

You can also take this to extreme by introducing a pre-receive (or pre-commit) hook in your VCS. There a commit is rejected if static analysis doesn’t pass. In this case analysis needs to be really fast.

Problem: Results are not understood

It might well be that tools produce fine reports pointing out the problems but if developers don’t understand why something should be fixed then it doesn’t get fixed. Commonly heard reasoning is “but the code works why should we fix it?”

Solution: Educate developers, chew the results automatically

In the worst case you need to do like was suggested in the previous solution, declare stop-the-line in case analysis reveals problems or even reject commits to VCS in such a case. I’ve noticed one of the most effective ways is to just show and tell. So show developers a recent problem which has been found, point out the source code location and tell what’s wrong with the code. For example complex code usually doesn’t look to be “wrong” but if you try refactoring it turns out to be difficult. Or if code is not covered by unit tests, retro-fitting them is not easy.

It makes sense to do some pre-analysis for the results. So not to show raw results to developers but try to filter out non-problems and possibly also old problems. The simpler the better.

Problem: Tools are used by wrong people

Some companies have solved some of the previously presented problems by devoting a separate team to perform code analysis tasks. Unfortunately it might be that this team is capable of only finding the problems but not fixing them. Sure it might be more effective if there’s number of people telling that here are these problems which should be fixed but it’s still likely the problems will not get fixed.

Solution: Automatic analysis, everyone responsible for fixing problems

A basic rule of thumb which works in several different issues is “if you break it, you fix it”. Making problems visible by automatic analysis and rapid feedback makes it more likely that problems get fixed. With the tools available it is fairly trivial to point out who caused codebase to become worse. If nothing else helps, public humiliation is the last option to try. And that was a joke, being constructive is of course always really important.

 

So what kind of different static source code analysis tools and methods are out there? I’ll tell something about the ones we are using in the next post. Stay tuned, meanwhile you can for example run some static analysis.