If you've seen my tweets over the last couple of days, you'll know that I've been taking a look at TestBox, the new testing framework from the ColdBox team.
You might be thinking "Do we really need a new testing framework?" and it's a reasonable question. CFML has had a number of testing frameworks over the years but mostly we've seen each one come and go as maintainers have moved onto other things: cfUnit, then cfcUnit, then a brief resurrection of cfUnit, then MXUnit. Does anyone remember cfSpec? I liked that a lot - I like the Behavior-Driven Development style of writing tests. That's why, at World Singles, we use the Expectations library for almost all of our Clojure-based tests.
So why the interest in TestBox? My initial interest was spurred by the inclusion of the BDD style of tests which I find both more natural to read and a better fit for expressing requirements for software that is yet to be written. Then I read that TestBox has MXUnit compatibility which intrigued me - we have been using MXUnit for several years at World Singles and I don't relish having to rewrite all those tests. If TestBox could allow us to run our existing "xUnit" tests as-is while we write new tests in the BDD style, I'd be very happy.
Adam Cameron has been blogging about Test-Driven Development recently and has also picked up on TestBox. He just blogged about trying TestBox on his extensive MXUnit test suite and you can see he ran into a number of issues, all of which he has reported, and which I expect we'll see fixes for. My experience was more fruitful than his - as I mentioned in my comment on his blog post: just two small changes got the whole of our suite running flawlessly!
My next challenge was integrating TestBox into our automated build / test script, which uses Ant. TestBox provides a runner-template that shows how to do basic Ant integration. I followed that recipe but discovered there was no way for Ant to see that any tests failed and so I couldn't "fail the build" if TestBox found errors or failures. After studying the source code briefly, I realized I could extend the TestBox CFC and provide a way for the runner CFM template to access the details of the TestResult object, and write a property file that Ant could read and fail if a certain property was set. Time to fork the ColdBox Platform Github repo and send a Pull Request to add build failing functionality which Luis merged in within a few hours! Nice!
With that in place, I was able to delete MXUnit from the World Singles repo and rely entirely on TestBox for all our testing needs!
Whether you prefer the "xUnit" style of tests, with assertions, or the "BDD" style with expectations, TestBox can support you, and even if you have a lot invested in MXUnit, migration to TestBox should be straightforward (especially once they address the issues Adam has raised). I think we can be sure of ongoing support and maintenance - based on the well-established track record of the ColdBox Team. I think TestBox is a safe bet. Now you just need to start actually doing some automated testing :)