Test Stub

The book has now been published and the content of this chapter has likely changed substanstially.
Please see page 529 of xUnit Test Patterns for the latest information.
Also known as: Stub

Variation of: Test Double

How can we verify logic independently when it depends on indirect inputs from other software components?

We replace a real object with a test-specific object that feeds the desired indirect inputs into the system under test.

Sketch Test Stub embedded from Test Stub.gif

In many circumstances, the environment or context in which the system under test (SUT) operates very much influences the behavior of the SUT. To get good enough control over the indirect inputs of the SUT, we may have to replace some of the context with something we can control, a Test Stub.

How It Works

First, we define a test-specific implementation of an interface on which the SUT depends. This implementation is configured to respond to calls from the SUT with the values (or exceptions) that will exercise the Untested Code (see Production Bugs on page X) within the SUT. Before exercising the SUT, we install the Test Stub so that the SUT uses it instead of the real implementation. When called by the SUT during test execution, the Test Stub returns the previously defined values. The test can then verify the expected outcome in the normal way.

When To Use It

A key indication for using a Test Stub is having Untested Code caused by the inability to control the indirect inputs of the SUT. We can use a Test Stub as a control point that allows us to the behavior of the SUT with various indirect inputs and we have no need to verify the indirect outputs. We can also use a Test Stub to inject values that allow us toget past a particular point in the software where it calls software that is unavailable in our test environment.

If we do need an observation point that allows us to verify the indirect outputs of the SUT, we should consider using a Mock Object (page X) or a Test Spy (page X). Of course, we must have a way of installing a Test Double (page X) into the SUT to be able to use any form of Test Double.

Variation: Responder

A Test Stub that is used to inject valid indirect inputs into the SUT so that it can go about its business is called a "Responder". They are commonly used in "happy path" testing when the real component is not yet available or is unusable in the development environment. The tests will invariably be Simple Success Tests (see Test Method on page X).

Variation: Saboteur

A Test Stub that is used to inject invalid indirect inputs into the SUT is often called a "Saboteur" because its purpose is to derail whatever the SUT is trying to do so we can see how the SUT copes with these circumstances. The "derailment" can be caused by returning unexpected values or objects, or it can be caused by raising an exception or causing a runtime error. Each test may either be a Simple Success Test or an Expected Exception Test (see Test Method) depending on how the SUT is expected to behave in response to the indirect input.

Variation: Temporary Test Stub

A Temporary Test Stub is a Test Stub that is used to stand in for a depended-on component (DOC) that is not yet available. It typically consists of an empty shell of a real class with hard-coded return statements. As soon as the real DOC is available, it is used instead. Test-driven development often causes us to create Temporary Test Stubs as we write code from the outside inwards; these shells evolve into the real classes as we add code to them. In the need-driven development we tend to use Mock Objects because we want to verify that the SUT calls the right methods on the Temporary Test Stub and we typically keep using the Mock Object even after the real DOC is available.

Variation: Procedural Test Stub

A Procedural Test Stub is a Test Stub written in a procedural programming language. This is particularly challenging in procedure programming languages that do not support procedure variables (a.k.a. function pointers.) In most cases this means that we must put if testing then hooks into the production code (a form of Test Logic in Production (page X))

Variation: Entity Chain Snipping

Entity Chain Snipping is a variation of Responder that is used to replace a complex network of objects with a single Test Stub that pretends to be the network of objects. This can make fixture setup must faster (especially when the objects would normally have to be persisted into a database) and can make the tests much easier to understand.

Implementation Notes

We must be careful when using Test Stubs because we are testing the SUT in a different configuration from that which will be used in production. We really should have at least one test that verifies it works without a Test Stub. A common mistake made by test automaters new to stubs is to replace a part of the SUT that they are trying to test. It is therefore important to be really clear about what is playing the role of SUT and what is playing the role of test fixture. Also, note that excessive use of Test Stubs can result in Overspecified Software (see Fragile Test on page X).

There are several different ways to build Test Stubs depending on one's needs and the tools we have on hand.

Variation: Hard-Coded Test Stub

A Hard-Coded Test Stub is a Test Stub that has it's responses hard-coded within its program logic. They tend to be purpose-built for a single test or a very small number of tests. See Hard-Coded Test Double (page X) for more information.

Variation: Configurable Test Stub

When we want to avoid building a different Hard-Coded Test Stub for each test we can use a Configurable Test Stub. A test configures the Configurable Test Stub as part of fixture setup. Many members of the xUnit family have tools available to generate Configurable Test Doubles (page X) including Configurable Test Stubs.

Motivating Example

The following test verifies the basic functionality of component that formats an HTML string containing the current time. Unfortunately, it depends on the real system clock so it rarely ever passes!

   public void testDisplayCurrentTime_AtMidnight() {
      // fixture setup
      TimeDisplay sut = new TimeDisplay();
      // exercise sut
      String result = sut.getCurrentTimeAsHtmlFragment();
      // verify direct output
      String expectedTimeString = "<span class=\"tinyBoldText\">Midnight</span>";
      assertEquals( expectedTimeString, result);
   }
Example NondeterministicTest embedded from java/com/clrstream/ex7/test/TimeDisplayTest.java

We could try to address this problem by making the test calculate the expected results based on the current system time as follows:

   public void testDisplayCurrentTime_whenever() {
      // fixture setup
      TimeDisplay sut = new TimeDisplay();
      // exercise sut
      String result = sut.getCurrentTimeAsHtmlFragment();
      // verify outcome
      Calendar time = new DefaultTimeProvider().getTime();  
      StringBuffer expectedTime = new StringBuffer();
      expectedTime.append("<span class=\"tinyBoldText\">");
      if ((time.get(Calendar.HOUR_OF_DAY) == 0)
         && (time.get(Calendar.MINUTE) <= 1)) {
         expectedTime.append( "Midnight");
      } else if ((time.get(Calendar.HOUR_OF_DAY) == 12)
                  && (time.get(Calendar.MINUTE) == 0)) { // noon
         expectedTime.append("Noon");
      } else  {
         SimpleDateFormat fr = new SimpleDateFormat("h:mm a");
         expectedTime.append(fr.format(time.getTime()));
      }
      expectedTime.append("</span>");
      assertEquals( expectedTime, result);
   }
Example FlexibleTest embedded from java/com/clrstream/ex7/test/TimeDisplayTest.java

This Flexible Test (see Conditional Test Logic on page X) introduces two problems. First, some test conditions are never exercised (do you want to come in at midnight to prove the software works at midnight?) Second, the test needs to duplicate much of the logic in the SUT to calculate the expected results. How do we prove it is correct?

Refactoring Notes

We can achieve proper verification of the indirect inputs by getting control of the time. To do this, we use the Replace Dependency with Test Double (page X) refactoring to replace the real system clock (represented here by TimeProvider) with a Virtual Clock[VCTP]. We then implement it as a Test Stub that is configured by the test with the time we want to use as the indirect input to the SUT.

Example: Responder (as Hand-Coded Test Stub)

   public void testDisplayCurrentTime_AtMidnight() throws Exception {
      // Fixture setup:
      //      Test Double configuration
      TimeProviderTestStub tpStub = new TimeProviderTestStub();
      tpStub.setHours(0);
      tpStub.setMinutes(0);
      //   Instantiate SUT:
      TimeDisplay sut = new TimeDisplay();
      //      Test Double installation
      sut.setTimeProvider(tpStub);
      // exercise sut
      String result = sut.getCurrentTimeAsHtmlFragment();
      // verify outcome
      String expectedTimeString = "<span class=\"tinyBoldText\">Midnight</span>";
      assertEquals("Midnight", expectedTimeString, result);
   }
Example TestStubUsage embedded from java/com/clrstream/ex7/test/TimeDisplayTestSolution.java

This test makes use of the following hand-coded Test Stub implementation:

   private Calendar myTime = new GregorianCalendar();
   /**
   * The complete constructor for the TimeProviderTestStub
   * @param hours specify the hours using a 24 hour clock
   *    (e.g. 10 = 10 AM, 12 = noon, 22 = 10 PM, 0 = midnight)
   * @param minutes specify the minutes after the hour
   *   (e.g. 0 = exactly on the hour, 1 = 1 min after the hour)
   */
   public TimeProviderTestStub(int hours, int minutes) {
      setTime(hours, minutes);
   }
  
   public void setTime(int hours, int minutes) {
      setHours(hours);
      setMinutes(minutes);
   }
  
   // Configuration Interface:
   public void setHours(int hours) {
      // 0 is midnight; 12 is noon
      myTime.set(Calendar.HOUR_OF_DAY, hours);
   }
  
   public void setMinutes(int minutes) {
      myTime.set(Calendar.MINUTE, minutes);
   }
   // Interface used by SUT
   public Calendar getTime() {
      // @return The last time that was set:
      return myTime;
   }
Example TestStubImplementation embedded from java/com/clrstream/ex7/test/TimeProviderTestStub.java

Example: Responder (Dynamically Generated)

Here's the same test coded using the JMock framework:

   public void testDisplayCurrentTime_AtMidnight_JM() throws Exception {
      // Fixture setup:
      TimeDisplay sut = new TimeDisplay();
      //  Test Double configuration
      Mock tpStub = mock(TimeProvider.class);
      Calendar midnight = makeTime(0,0);  
      tpStub.stubs().method("getTime").withNoArguments().will(returnValue(midnight));
      //  Test Double installation
      sut.setTimeProvider((TimeProvider) tpStub);
      // exercise sut
      String result = sut.getCurrentTimeAsHtmlFragment();
      // verify outcome
      String expectedTimeString = "<span class=\"tinyBoldText\">Midnight</span>";
      assertEquals("Midnight", expectedTimeString, result);
   }
Example JMockStubUsage embedded from java/com/clrstream/ex7/test/TimeDisplayTestSolution.java

There is no Test Stub implementation to look at for this test because the JMock framework implements the Test Stub using reflection. Because of this, we had to write a Test Utility Method (page X) called makeTime that contains the logic to construct the Calendar object to be return. In the hand-coded Test Stub, this logic was inside the getTime method.

Example: Saboteur (as Anonymous Inner Class)

   public void testDisplayCurrentTime_exception() throws Exception {
      // fixture setup
      //   Define and instantiate Test Stub
      TimeProvider testStub = new TimeProvider()
         { // anonymous inner Test Stub
            public Calendar getTime() throws TimeProviderEx {
               throw new TimeProviderEx("Sample");
         }        
      };
      //   Instantiate SUT:
      TimeDisplay sut = new TimeDisplay();
      sut.setTimeProvider(testStub);
      // exercise sut
      String result = sut.getCurrentTimeAsHtmlFragment();
      // verify direct output
      String expectedTimeString = "<span class=\"error\">Invalid Time</span>";
      assertEquals("Exception", expectedTimeString, result);
   }
Example ExceptionInputTest embedded from java/com/clrstream/ex7/test/TimeDisplayTestSolution.java

In this case, we chose to use an Inner Test Double (see Hard-Coded Test Double) to throw an exception that we expect the SUT to handle gracefully. One interesting thing to note about this test is that it uses the Simple Success Test method template rather than the Expected Exception Test template even though we are injecting an exception as the indirect input. This is because we are expecting the SUT to catch the exception and change the string formatting; we are not expecting the SUT to throw an exception.

Example: Temporary Test Stub

A Temporary Test Stub is a Test Stub that is used to stand in for a DOC that is not yet available. It typically consists of a empty shell possibly with hard-coded return statements. As soon as the real DOC is available, it is used instead. Test-driven development often causes us to create Temporary Test Stubs as we write code from the outside inwards. But in the need-driven development we tend to use Mock Objects because we want to verify that the SUT calls the right methods on the Temporary Test Stub and we typically keep using the Mock Object even after the real DOC is available.

Example: Entity Chain Snipping

Entity Chain Snipping is a variation of Responder that is used to replace a complex network of objects with a single Test Double that pretends to be the network of objects. This can make fixture setup must faster (especially when the objects are persisted into a database) and can make the tests much easier to understand.

In this example, we are testing the Invoice but we require a Customer to instantiate the invoice. The Customer requires an Address which requires a City so we find ourself creating all these additional objects just to set up the fixture. Suppose the behavior of the invoice depends on some attribute of the Customer that is calculated from the address.

   public void testInvoice_addLineItem_noECS() {
      final int QUANTITY = 1;
      Product product = new Product(getUniqueNumberAsString(), getUniqueNumber());
      State state = new State("West Dakota", "WD");
      City city = new City("Centreville", state);
      Address address = new Address("123 Blake St.", city, "12345");
      Customer customer= new Customer(getUniqueNumberAsString(), getUniqueNumberAsString(),
                                      address);
      Invoice inv = new Invoice(customer);
      // Exercise
      inv.addItemQuantity(product, QUANTITY);
      // Verify
      List lineItems = inv.getLineItems();
      assertEquals("number of items", lineItems.size(), 1);
      LineItem actual = (LineItem)lineItems.get(0);
      LineItem expItem = new LineItem(inv, product, QUANTITY);
      assertLineItemsEqual("",expItem, actual);
   }
Example EntityChain embedded from java/com/clrstream/camug/example/test/InvoiceTest.java

But in this test we only want to verify the behavior of the invoice logic, not the way this attribute is calculated from the invoice. So all the setup of the address, city, etc. is distracting the reader.

Here's the same test using a Test Stub instead of the Customer. Note how much simpler the fixture setup has become!

   public void testInvoice_addLineItem_ECS() {
      final int QUANTITY = 1;
      Product product = new Product(getUniqueNumberAsString(), getUniqueNumber());
      Mock customerStub = mock(ICustomer.class);
      customerStub.stubs().method("getZone").will(returnValue(ZONE_3));
      Invoice inv = new Invoice((ICustomer)customerStub.proxy());
      // Exercise
      inv.addItemQuantity(product, QUANTITY);
      // Verify
      List lineItems = inv.getLineItems();
      assertEquals("number of items", lineItems.size(), 1);
      LineItem actual = (LineItem)lineItems.get(0);
      LineItem expItem = new LineItem(inv, product, QUANTITY);
      assertLineItemsEqual("", expItem, actual);
   }
Example EntityChainSnipping embedded from java/com/clrstream/camug/example/test/InvoiceTest.java

We have used JMock to stub out the Customer with a customerStub that returns ZONE_3 when getZone is called. This is all we need to verify the Invoice behavior and we have managed to get rid of all that distracting extra object construction.