XUnit Test Patterns

Refactoring Test Code

xUnit Test Patterns - the book

The book has won a Jolt Productivity Award in the Best Technical Book category! Here's what the reviewer Rick Wayne said about why the book won the award:

Unit testing is hardly news, but simply writing a ton of tests guarantees you no bliss. Gerard Meszaros's xUnit Test Patterns distills and codifies the crucial meta-knowledge to take us to the next level. Why do good tests go bad, and how do you fix them--it's as simple and groundbreaking as that. Smells and antipatterns arise in tests that cripple their maintainability. xUnit Test Patterns exhaustively describes those pathologies and provides the prescription in the catalog format familiar since 1994. But fear not - every motivation and pattern includes at least one source-code example and the explanations are couched in clear, direct language. If you're ready to promote your test code to the same level of care and craftsmanship that you devote to production systems, grab a copy of xUnit Test Patterns and get cracking.

Other reviewers have been equally kind:

Awesome book must have for any software developer - A. B. Powell

Our team has been doing TDD for 6 years since the seminal Kent Beck book. We as a team have learnt read and improved our practice over these years but this book has had the most impact on how 2 do TDD properly. It has significantly improved our code and our testing practices. Many of the leasons we learnt where emphasised and standardised in this book. All I can say is that you will save yourself years and years of hard knocks learning if you read this book first. By following the patterns in this book we have been more easily able to implement the principles in Kent's book. -Brett

Availability

The book is now available at retailers and sample chapters are available for download from Addison Wesley Professional's website. It is available in both traditional print form through most booksellers and as a PDF e-book (but only from the AW website.) Note that the material here is now somewhat out of step with the book content because it hasn't been updated based on the results of copy editing.

xUnit Test Patterns - Live Training and Tutorials

I will be presenting tutorials based on this material at various conferences:

---SPLASH 2010 -half-day tutorial -Reno Nevada -October 18, 2010
---Much Ado About Agile 2010 -half-day tutorial -Vancouver BC Canada -November 1st, 2010

I will be doing an open-enrollment training course in Calgary, Alberta November 24-25.

I am available to provide on-site training at your place of business. If you don't have enough people to justify a dedicated class, I can augment with additional students by listing it as an open enrollment course here and through Agile University. Please feel free to suggest venues where you would like to see a course run. Likewise, please send me your suggestions for future conferences. You can reach me by e-mail using the link at the bottom of each page.

Thanks,

Gerard

Sketch Cover-Small embedded from Cover-Small.gif

Larger picture of cover

If you are going to order online, using one of these links will get me a small referral fee.

USA	Canada	UK

Other online booksellers include:

You can also purchase it directly from Addison Wesley Professional at

Hard cover: http://www.awprofessional.com/bookstore/product.asp?isbn=0131495054&rl=1 (2 sample chapters are also available here.)
PDF e-book: http://www.awprofessional.com/bookstore/product.asp?isbn=0321504801&rl=1

Contents Overview

The book is organized in 3 major parts. Part I consists of a series of introductory narratives that describe some aspect of test automation using xUnit. This includes a list of Goals and Principles which are summarized on the left size of this diagram:

Sketch Goals-Smells embedded from Goals-Smells.gif

Part II describes a number of "test smells" that are symptoms of problems with how we are automating our tests. These symptoms are summarized in the right side of the previous diagram.

Part III contains descriptions of the patterns. The chapters correspond to the major boxes in the following diagram:

Sketch Patterns Summary embedded from Patterns Summary.gif

The major boxes in each of the preceding diagrams corresponds to a chapter in the corresponding part of the book; they also correspond the "categories" listed in the top left navigation sidebar on this web site. Selecting a category provides a list of patterns or smells in the category and selecting a specific pattern or smell adds a list of aliases, causes and variations of the chosen pattern or smell.

The following diagram summarizes the notation used in these overview diagrams:

Sketch Visual Summary Legend embedded from Visual Summary Legend.gif

Of course, you can access the material in "book order" by starting in Book Outline. Because so much has changed during the copy editing process, I'll be "abridging" the material on this web site once the book is out. I will, however, keep a brief summary of each pattern & smell on the site as well as many of the cross reference tables.

Why is Test Automation Important?

Automated unit tests (a.k.a. "developer tests") and customer tests (a.k.a. "functional tests") are a cornerstone of many agile development methods (such as eXtreme Programming). The availability of automated, self-checking tests allows developers to be much bolder in how they modify existing software. They allow a more evolutionary form of software development that support incremental delivery of functionality to the customer (motto: Deliver early; deliver often!) that speeds up user feedback and improves the quality (both "fitness for purpose" and "software quality") of the software being built. The techniques are also spreading to less agile development methods via the introduction of "Test Driven Development" as a less extreme process alternative.

Automated test are more repeatable than manual tests because they execute exactly the same way every time. They don't forget things after long weekends and vacations. They don't leave to work on other projects. They don't get sick or run over by a bus.

Automated tests take a lot less effort to run than manual tests. As a result, they are more likely to be run often. The more often the better. Fully assimilated agile developers run their tests pretty well every time they save and compile their code. And any time they need a quick vote of confidence! The tests act as a comforting "safety net" that promises to catch the developer's mistakes. This allows them to work more quickly and with less paranoia and that makes them more productive despite the extra effort involved in writing the tests.

Why is Test Automation Hard?

Cost effective test automation is all about repeatability, maintainability and communication. Repeatability of results requires repeatability of test fixture setup and repeatability of the interactions with the software under test. And that requires interfaces into the software under test that allow you to put it in the right state before the test and to find out what state it is in after the test. And that can be hard. Throw in the need to make the tests easy to understand and easy to maintain and the problem gets even harder.

Automating tests using XUnit is a form of software whether you write the tests before or after the code it tests. But the goals of this test software ("testware") is very different from the software most people are used to writing. Writing it is optional so we can stop writing it or maintaining it at any time. So we need compelling reason to (keep) writing them. And there is a lot to think about when automating tests. How do I interact with the SUT? What is the best way to express the expected outcome? How can I keep tests from breaking each other? How can I ensure the tests will work next week, next month, even next year? How do I test when the software under test depends on software that hasn't been written yet? Or cannot be used in our test environment? Or makes the tests take too long to run? If writing the tests is hard and has little benefit, we just won't do it any more!

Why Test Automation Patterns?

I have been reading the various conference papers and (mostly JUnit-based) books on test automation for quite some time. Each author seems to have a particular area of interest and favorite techniques. While I don't always agree with their practices, I am always trying to understand why they do it a particular way and *when* it would be more appropriate to use their techniques than the ones I already use.

This is one of the major different between prose that merely explains a technique and a pattern. Patterns help the reader understand the WHY behind each practice so they can make intelligent choices between the alternative patterns (and thereby avoid nasty consequences from surprising them later.)

Some projects are using the XUnit tools to automate regression tests. All of these initiatives are more successful when the test developers have experience building tests. And the next best thing to having experience test automaters on your project (which ClearStream Consulting would be more than happy to supply) is a well-documented set of good test automation practices in the form of patterns.

Background of this website

I've assembled this site to catalog the good practices in xUnit test automation I've encountered over the years. It came about as a result of discussions between me (Gerard Meszaros) and Shaun Smith about the testing techniques we found ourselves using over and over again to solve particular xUnit test automation problems.

We started off by writing a paper [IEAT] that we presented at XP2001 in Sardinia, Italy on techniques that we had used to make our tests run much faster. This led to discussions with the inventors of Mock Objects (page X) about whether we were using Mock Objects or some other technique.

We presented another paper [TAM] at XP/Agile Universe 2003 in New Orleans, Louisiana in which we identified a number of smells and the principles we used to avoid them. These built on papers presented in previous XP conferences on the use of Mock Objects and testing of Frameworks. Discussions with other TDD folk convinced us that there was a real need to share and standardize the vocabulary around XUnit-based test automation. Naturally, patterns were the obvious choice for communicating this knowledge.

Since XP/Agile Universe 2003, we have been cataloging all the patterns we regularly use and the obvious alternatives that we have consciously chosen not to use. We have also been looking through the mostly JUnit-based books on test automation to see what others are advocating looking for techniques whether or not they are described as patterns. There is some obvious overlap between the various material but also seem to be considerable gaps and no comprehensive treatment of the topic. As of October 2003, we had identified over 120 patterns ranging from principles and strategies to coding-level idioms!

Organization of this website

I'll be keeping a brief summary of each pattern & smell on the web site once the book is out. The best place to start is with the Book Outline (or Book Outline Diagrams).

All the information on this website is organized into categories. The categories are accessible via hyperlinks in the "All Categories" box on the left side. Once a category is selected, the pages (patterns, narratives , etc.) within the category can be accessed from the "All categoryName" box that appears below the "All Categories" box.

The Narratives

All patterns, smells, etc. are first introduced in a series of narratives that provide an overview of a particular topic area. The narratives are accessible via the special category "Narratives" from the "All Categories" box or from the Book Outline.

The Patterns and Smells

The patterns and smells are organized into "categories" that each correspond to a single chapter in the book. The categories are accessible via hyperlinks in the "All Categories" box on the left side. Once a category is selected, the patterns within the category can be accessed from the "All categoryName" box that appears below the "All Categories" box.

The patterns and smells can also be accessed in the following ways:

via hyperlinks from within the narratives
via hyperlinks from other patterns or smells
from the "Category Contents" sidebar on many pages

The various categories can also be accessed from the Book Outline.

Page generated at Wed Feb 09 16:39:23 +1100 2011