Engineering
Automated tests exist to make change safe, not to reach a coverage number. We test the behaviour that would be expensive to break and leave the rest to review.
Context
The cost of software is dominated by changing it later. Tests are what decide whether the third year of a system is affordable.
Choices
Every choice below has a reason attached. A stack without reasons is a logo wall.
The questions we work through before recommending anything in this area.
Recommending something everywhere is the same as having no opinion.
Where we apply it
No particular number. Coverage measures which lines ran, not whether the important behaviour is protected. A system with 40% coverage concentrated on money and permissions is safer than one at 90% spread evenly.
For anything that will be changed again, yes, but proportionately. A handful of tests around the behaviour that would be costly to break returns more than a comprehensive suite nobody maintains.
Automated tests exist to make change safe, not to reach a coverage number. We test the behaviour that would be expensive to break and leave the rest to review.
The cost of software is dominated by changing it later. Tests are what decide whether the third year of a system is affordable.
Exhaustive tests on a prototype whose purpose is to be thrown away; Chasing a coverage percentage, which rewards testing the code least likely to break; and As a substitute for review; tests confirm behaviour, review catches design problems
What would be most expensive to break without noticing; Whether the team can currently deploy without a manual checklist; and How long the suite takes, because a slow suite gets bypassed
Tests written against what the system does survive refactoring. Tests written against how it does it break every time the code improves, which teaches teams to avoid improving it.
Money movement, data integrity, permissions and anything irreversible. Coverage percentages reward testing trivial code, which is why they are a poor target.
Connected
Bring the existing system and the constraints around it. We will tell you where this fits, where it would be the wrong tool, and what we would leave exactly as it is.
