Generated Value

The book has now been published and the content of this chapter has likely changed substanstially.
Please see page 723 of xUnit Test Patterns for the latest information.
Also known as: Generated Value

How do we specify the values to be used in tests?

Generate a suitable value each time the test is run.

      BigDecimal uniqueCustomerNumber = getUniqueNumber();
Example DistinctValueGenerationSketch embedded from java/com/clrstream/camug/example/test/InvoiceTest.java

When initializing the objects in the test fixture, one of the issues is that most objects have various attributes (fields) that need to be supplied as parameters to the constructor. Sometimes the exact values to be used affect the outcome of the test, but more often than not it is only important that each object use a different value. When the values of these attributes are not important to the test, it is important not to have them visible within the test!

Generated Value is a technique used in conjunction with Creation Methods (page X) to help us remove this potentially distracting information from the test.

How It Works

Instead of deciding what values to use in our tests at test coding time, we generate the values when the tests are executed. This allows us to pick values to satisfy specific criteria such as "must be unique in the database" that can only be ensured as the test run unfolds.

When To Use It

We can use a Generated Value whenever we cannot or do not want to specify the test values until the test is executing. This can be because the value of an attribute is not expected to affect the outcome of the test and we don't want to be bothered to define Literal Values (page X) or it can be because we need to ensure some quality of the attribute that can only be determined at run time. In some cases, the system under test (SUT) requires the value of an attribute to be unique; using a Generated Value can ensure this and thus prevent Unrepeatable Tests (see Erratic Test on page X) and Test Run Wars (see Erratic Test) by reducing the likelihood of a test conflicting with its parallel incarnation in another test run. Optionally, we can use this distinct value for all attributes of the object; this makes object recognition very easy when inspecting the object in a debugger.

One thing to be wary of is that different values could expose different bugs. For example, a single digit number may be formatted correctly while a multi-digit number might not (or vice versa). Generated Values can result in Nondeterministic Tests (see Erratic Test); if we encounter non-determinism (sometimes the test passes and during the very next run it fails) we must check the SUT code to see whether differences in value could be the root cause.

In general, one shouldn't use a Generated Value unless the value must be unique because of the non-determinism this may introduce. The obvious alternative is to use a Literal Value. A less obvious alternative is to use a Derived Value (page X) especially when determining the expected results of a test.

Implementation Notes

We can generate values a number of different ways; the appropriateness of each one depends on the circumstance.

Variation: Distinct Generated Value

When we need to ensure that each test or object uses a different value, we can use a Distinct Generated Value. It is useful to create a set of utility functions that will return unique values of various types (such as integers, strings, floating point numbers, etc.). The various getUnique methods can all be built upon an integer sequence number generator. For numbers unique within the scope of a shared database, we can use database sequences or a sequence table. For numbers unique within the scope of a particular test run, we can use an in-memory sequence number generator (e.g., use a Java static variable that is incremented before usage). In-memory sequence numbers that start from the number one each time a test suite is run offer a useful quality: the values generated in each test are the same for each run and can simplify debugging.

Variation: Random Generated Value

One way to get good test coverage without spending a lot of time analyzing the behavior and generating test conditions is to use different values each time we run the tests. Using a Random Generated Value is one way to do this. While this may seem like a good idea, it makes the tests non-deterministic (Nondeterministic Tests) and can make debugging failed tests very difficult. Ideally, when a test fails, we want to be able to repeat that test failure on demand. This can be made possible by logging the Random Generated Value as the test is run and showing it as part of the test failure. We then need to find a way to force the test to use that value when we are troubleshooting the failed test. In most cases all this is too much effort for the potential benefit but when we need this technique, we really need it.

Variation: Related Generated Value

An optional enhancement is to combine Generated Value with Derived Value by using the same generated integer as the root for all the attributes of a single object. This can be accomplished by calling getUniqueInt once and then using that value to build unique strings, floats etc. This has the nice effect of having all fields of the object contain "related" data to make it easier to recognize when debugging. Another option is to separate the generation of the root from the generation of the values by calling generateNewUniqueRoot explicitly before calling getUniqueInt, getUnqueString, etc..

Another nice touch for strings is to pass a role-describing argument to the function that is to be combined with the unique integer key to make the code more intent revealing. We could also pass such arguments to the other functions but, of course, we wouldn't be able to build them into an Integer value.

Motivating Example

Here's a test that uses Literal Values for the arguments to a constructor.

   public void testProductPrice_HCV() {
      //    SetUp:
      Product product = new Product( 88,                       // id "Widget",                 // Name
                      new BigDecimal("19.99")); // price
      // Exercise:
      //   ...}
Example HardCodedValues embedded from java/com/xunitpatterns/values/GeneratedValue.java

Refactoring Notes

We can convert the test to use Distinct Generated Values by replacing the Literal Values with calls to the appropriate getUnique method. These methods simply increment a counter each time they are called and use that as the root for construction of an appropriately-typed value.

Example: Distinct Generated Value

Here is the same test using a Distinct Generated Value. For the getUniqueString method, we'll pass a string describing the role ("Widget Name").

   public void testProductPrice_DVG() {
      // SetUp:
      Product product = new Product( getUniqueInt(),              // id getUniqueString("Widget"), // Name
                      getUniqueBigDecimal());    // Price
      // Exercise:
      //   ...}
  
   static int counter = 0;
  
   int getUniqueInt() {
      counter++;
      return counter;  
   }
  
   BigDecimal getUniqueBigDecimal() {
      return new BigDecimal(getUniqueInt());
   }  
  
   String getUniqueString(String baseName) {
      return baseName.concat(String.valueOf( getUniqueInt()));  
   }
Example DistinctValueGeneration embedded from java/com/xunitpatterns/values/GeneratedValue.java

This test will use a different generated value for each argument of the constructor call. The numbers will be consecutive but the test reader will still need to look at a specific attribute when debugging to get a consistent view. We probably should not be generating the price value if the logic we were testing was related to price calculation because that would force our verification logic to accomodate different total costs.

Example: Related Generated Value

We can ensure that all the values used by the test are obviously related by separating the generation of the root value from the construction of the individual values. Here, we've moved the generation of the root to the setUp method so each test method gets a new value only once. The methods that retrieve the various values (getUniqueString, etc.) simply use the previously generated root when composed the Generated Values.

   public void testProductPrice_DRVG() {
      //    SetUp:
      Product product = new Product( getUniqueInt(),            // id getUniqueString("Widget"), // Name
                      getUniqueBigDecimal());    // Price
      // Exercise:
      //   ...}
  
   static int counter = 0;
  
   public void setUp() {
      counter++;
   }
  
   int getUniqueInt() {
      return counter;  
   }
  
   String getUniqueString(String baseName) {
      return baseName.concat(String.valueOf( getUniqueInt()));  
   }
  
   BigDecimal getUniqueBigDecimal() {
      return new BigDecimal(getUniqueInt());
   }
Example RelatedDistinctValueGeneration embedded from java/com/xunitpatterns/values/RelatedDistinctValue.java

If we were to look at this object in an object inspector or database or dump part of it to a log, it would be very easy to tell which object we are looking at regardless of which field we happened to see.

Page generated at Wed Feb 09 16:39:33 +1100 2011