One day I would like to write a post called “test-driven development for web applications”. So apologies to anyone who came here looking for that. Because I have no idea what that is. But maybe you do? Please feel free to leave a comment.
I would also like to write a tutorial called “how to write unit tests for web applications” because almost every example I’ve come across (and I admit, I have not looked hard) of how to write unit tests tends to involve obvious classes like BankAccount or abstract classes like ObjectFactory. Sheesh.
At work I help lead the development of a web application written in PHP on top of a MySQL database. More than a year ago we decided: “we need unit tests”. So we took the typical approach and used PHPUnit to generate a stub for each of our core “model” classes. Starting with the shortest and easiest classes I began writing tests. What a pain in the ass. The one good thing that came out of it is that I made it possible to do a single command install of our codebase.
Mostly the tests tested whether inserting, updating, and selecting records out of the database and loading them into objects succeeded. Over and over and over again. And every time we had to add a field to a table, we had to update the test in several repetitive, tedious ways that ensured that our first foray into testing was bound to fail. I kept at it for a while. Most of the bugs I found were due to out of date tests. And you probably know how this story ends.
Skip ahead to last Monday, and Ivan informs me that Sebastian Bergmann, the creator of PHPUnit, is giving a talk over at CBS Interactive (nee CNet). I figure why not. Maybe I’d learn something. But when he got to this slide, I realized this talk was not going to get me any closer to the holy grail of “how to write unit tests for web applications”:
I couldn’t contain an outburst at that point, “But isn’t that like every application?” How had it come to this, me a belligerent heckler at a unit testing meetup?
It reminds me that Jonathan recently sent me a link to Tim Bray’s Test-Driven Heresy post with the email subject “we’ve been doing a lot of sinning lately”. Well, I finally had a chance to read it this evening, and I’m glad I got to it late, because it was the comments I found the most illuminating. Paul Houle’s comment gave me the first glimpse of a reality where unit testing just might not apply (in the same way) to web applications.
In some applications, objects are self-contained, activities are sequential, and algorithms are tricky. Automated unit testing is cheap and beneficial. You may have already spent the cost that it takes to break the system into testable units, and know how to make a good architecture with those constraints.
In other cases (say a browser-based app) you’re writing simple programs that work by causing side effects in a large, complex and unreliable external system. In that case the real issue is “does the external system behave the way I want it to” and the nature of your testing is entirely different.
Here’s another tidbit I stumbled across on Ward Cunningham’s patterns wiki:
Once again, web applications AND databases used in a single sentence to describe things that are hard to test, and when they’re restructured to improve testability, they end up needlessly complicated. I swear, I went looking for resources to help me improve web app unit testing, and I ended up with the sense that what I’m looking for might not even be possible.
Update, July 27, 2009: There was another blog post I stumbled upon when I was writing this, and it’s stuck in my mind since then, so I’m going to include an excerpt here. It’s by this guy named Hamlet D’Arcy, called Big Flat Test Cases Suck (and what to do about it):
Beginners always created a test case per class; there was a one to one mapping between production classes and test classes, and I’d guess most people are still doing this.
The enlightened testers were creating one test case per fixture, in which test methods were all in the same test class if they shared the same setUp() method…
The really cool kids, however, were doing test case per feature. This entails grouping test methods together based on what feature or user story they implement.
But don’t be fooled! All three of these test organization methods are essentially the same…Test cases grow. They grow really long. And they all grow in one direction: down the screen.
At least I can skip ahead to the “really cool kids” part!