Test by feature not story13th June 2011
I know it’s been said before, but it’s such an important and misunderstood point I feel the need to reiterate it.
Very often, teams build their functional test suites around stories. Every time a new story is played a brand new test is written. Sometimes, the test will even have the story number in the test name.
This is dangerous for a couple of reasons. Firstly it encourages duplication in your tests. This is because every time a particular application feature is touched, a new test is written, which probably does very similar things to all the other previous tests around the feature. Not only is this a nightmare to maintain, but also it means that your test suite will take forever to run.
The second reason is, stories represent changes to features, not the features themselves. This means that every time you play a story, you are changing how a piece of functionality works. As a result previous story based tests will fail and will possibly even become redundant. Chasing down these failures incurs a great deal of pain.
A far more sensible pattern is to build your functional test suite around the features themselves (there’s a reason why cucumber test files have the “.feature” extension). When a story is played, your test suite is changed or augmented to reflect how the application should work. This means redundant or out of date tests are deleted as soon as a story invalidates them. It also means that you only have one set of tests for a given feature, massively reducing duplication.
Often you’ll find that as you write tests to reflect how the application works, your tests will illustrate the business workflows. It will become not only more maintainable, but more accessible to non-technical users, such as non-technical QAs, BAs and the business themselves. This allows them to become involved in the test writing process, and ensures you’re testing the features that are most important to them.