I’ve been fixing a few bugs in Simple.Data over the last couple of days, and I’m feeling the need to post something about the benefits of a good test suite.
For most feature development on Simple.Data, I do test-driven development. The dynamic nature of my API lends itself really well to this approach; this is one of the real reasons TDD is popular with dynamic languages like Ruby and Python. When I want to add a new Query operator or method, I can write a test that uses the syntax I’m aiming for, and the Behaviour test project will compile and run, and I’ll get a failing test, either with a failed assert or an exception. I really like the latter form of failure, since I get a stack trace and I can just dive into the code at that point and start working out what’s wrong.
So that’s great, but the thing that makes me want to write this post is my “QA” process. When someone reports a bug, like this one, it means my test suite is incomplete. So again, the first thing I do is to create a test which reproduces the bug. Then I fix the code so that test passes. Then I do a Release build, and run the full set of tests (currently 560+ including integration tests) to make sure that the fix hasn’t broken anything else. And then, and this is the really awesome part: if all the tests pass, I package the build and push it to NuGet.
I can do this because I trust those tests to be verifying all the behaviour that Simple.Data users are relying on in their applications. If the tests pass, I’m not going to break anybody’s system when they update to the new version. On the (rare) occasions when this has gone wrong, it’s been because I didn’t have the right tests, and I’ve gone back and added them (would you believe the SQL Server test project didn’t test delete’s until yesterday? Epic fail).
When people look at the Simple.Data repository, and say “wow, you’ve got lots of tests”, it’s as if they think that’s a discipline thing, that I’m just really conscientious about coverage. But that’s not it. I couldn’t do this without the tests. And neither can you, whatever you’re working on.