Testing Without Mocks: A Pattern Language

gowebing

When programmers use test-driven development (TDD), the code they test interacts with other parts of the system that aren't being tested. To test those interactions, and to prevent the other code from interfering with their tests, programmers often use mock objects or other test doubles. However, this approach requires additional integration tests to ensure the system works as a whole, and it can make structural refactorings difficult.

This pattern language1 describes a way of testing object-oriented code without using mocks. It avoids the downsides of mock-based testing, but it has tradeoffs of its own.

(The structure of this article was inspired by Ward Cunningham's CHECKS Pattern Language of Information Integrity, which is a model of clarity and usefulness.)

For example code demonstrating these ideas, see my example on GitHub.

Goals

Tradeoffs

Architectural Patterns

Testing without mocks requires careful attention to the dependencies in your codebase. These patterns help establish the ground rules.

Overlapping Sociable Tests

Our goal is to create a test suite consisting entirely of "narrow" tests, with no need for "broad" end-to-end tests. But most narrow tests don't test that the system is wired together correctly. Therefore:

When testing the interactions between an object and its dependencies, inject real dependency instances (not test doubles) into the unit under test. Don't test the dependencies' behavior itself, but do test that the unit under test uses the dependencies correctly.

This will create a strong linked chain of tests. Each test will overlap with dependencies' tests and dependents' tests. The test suite as a whole should cover your entire application in a fine overlapping mesh, giving you the coverage of broad tests without the need to write them.

To avoid constructing the entire dependency chain, use Zero-Impact Instantiation and Parameterless Instantiation. To isolate your unit under test, use Collaborator-Based Isolation and Nullable Infrastructure. To test code that depends on infrastructure, use Configurable Responses, Send State, Send Events, and Behavior Simulation.

A-Frame Architecture

Because we aren't using mocks to isolate our dependencies, it's easiest to test code that doesn't depend on infrastructure (external systems such as databases, file systems, and services). However, a typical layered architecture puts infrastructure at the bottom of the dependency chain:

   Application/UI
         |
         V
       Logic
         |
         V
   Infrastructure

Therefore:

Structure your application so that infrastructure and logic are peers under the application layer, with no dependencies between Infrastructure and Logic. Coordinate between them at the application layer with a Logic Sandwich or Traffic Cop.

       Application/UI
       /            \            
      V              V
   Infrastructure   Logic

Build the bottom two layers using Infrastructure Patterns and Logic Patterns.

Although A-Frame Architecture is a nice way to simplify application dependencies, it's optional. You can test code that mixes infrastructure and logic using Infrastructure Wrappers and Nullable Infrastructure.

To build a new application using A-Frame Architecture, Grow Evolutionary Seeds. To convert an existing layered architecture, Climb the Ladder.

Logic Sandwich

When using an A-Frame Architecture, the infrastructure and logic layers can't communicate with each other. But the logic layer needs to read and write data controlled by the infrastructure layer. Therefore:

Implement the top-level code as a "logic sandwich," where data is read by the infrastructure layer, then processed by the logic layer, then written by the infrastructure layer. Repeat as needed. Each piece can then be tested independently.

let input = infrastructure.readData();
let output = logic.processInput(input);
infrastructure.writeData(output);

This simple algorithm can handle sophisticated needs if put into a loop with a stateful logic layer.

For applications with complicated infrastructure, use a Traffic Cop instead.

Traffic Cop

The Logic Sandwich boils infrastructure down into simple infrastructure.readData() and infrastructure.writeData()abstractions. Applications with complex infrastructure may not be a good fit for this approach. Therefore:

Instead of asking the infrastructure for its data, use the Observer pattern to send events from the infrastructure layer to the application layer. For each event, implement a Logic Sandwich. In some cases, your application code might need a bit of logic of its own.

infrastructure.on("login", (token) => {  // infrastructure layer
  let loginInfo = LoginInfo.createFromToken(token);  // logic layer
  if (loginInfo.isValid) {  // application logic
    let userData = subscriberService.lookUpUser(loginInfo.userId);  // infrastructure layer
    let user = new User(userData);  // logic layer
    infrastructure.createSession(user.sessionData);  // infrastructure layer
  }
});
infrastructure.on("event2", (data) => {
  let output = logic.processEvent2(data);
  infrastructure.writeData2(output);
});
//...etc...

Be careful not to let your Traffic Cop turn into a God Class. If it gets complicated, better infrastructure abstractions might help. Sometimes taking a less "pure" approach and moving some Logic code into the Infrastructure layer can simplify the overall design. In other cases, splitting the application layer into multiple classes, each with its own Logic Sandwich or simple Traffic Cop, can help.

Grow Evolutionary Seeds

One popular design technique is outside-in design, in which an application is programmed by starting with the externally-visible behavior of the application, then working your way into the details.

This is typically done by writing a broad integration test to describe the externally-visible behavior, then using narrow unit tests to define the details. But we want to avoid broad tests. Therefore:

Use evolutionary design to grow your application from a single file. Choose a simple end-to-end behavior as a starting point and test-drive a single class to implement a trivial version of that behavior. Hardcode one value that would normally come from the Infrastructure layer, don't implement any significant logic, and return the result to your tests rather than displaying in a UI. This class forms the seed of your Application layer.

// JavaScript example: simplest possible Application seed

// Test code
it("renders user name", function() {
  const app = new MyApplication();
  assert.equal("Hello, Sarah", app.render());
});

// Production code
class MyApplication {
  render() {
    return "Hello, Sarah";
  }
}

Next, implement a barebones Infrastructure Wrapper for the one infrastructure value you hardcoded. Code just enough infrastructure to provide one real result to your application layer class. Don't worry about making it robust or reliable yet. This Infrastructure Wrapper class forms the seed of your Infrastructure layer.

Before integrating your new Infrastructure class into your Application layer class, implement Nullable Infrastructure. Then modify your application layer class to use the infrastructure wrapper, injecting the Null version in your tests.

// JavaScript example: Application + read from infrastructure

// Test code
it("renders user name", function() {
  const usernameService = UsernameService.createNull({ username: "my_username" });
  const app = new MyApplication({ usernameService });
  assert.equal("Hello, my_username", app.render());
});

// Production code
class MyApplication {
  // usernameService parameter is optional
  constructor({ usernameService = UsernameService.create() } = {}) {
    this._usernameService = usernameService;
  }

  async render() {
    const username = await this._usernameService.getUsername();
    return `Hello, ${username}`;
  }
}

Next, do the same for your UI. Choose one simple output mechanism that your application will use (such as rendering to the console, the DOM, or responding to a network request) and implement a barebones Infrastructure Wrapper for it. Add support for Nullable Infrastructure and modify your application layer tests and code to use it.

// JavaScript example: Application + read/write to infrastructure

// Test code
it("renders user name", function() {
  const usernameService = UsernameService.createNull({ username: "my_username" });
  const uiService = UiService.createNull();
  const app = new MyApplication({ usernameService, uiService });

  app.render();
  assert.equal("Hello, my_username", uiService.getLastRender());
});

// Production code
class MyApplication {
  constructor({ 
    usernameService = UsernameService.create(),
    uiService = UiService.create(),
  } = {}) {
    this._usernameService = usernameService;
    this._uiService = uiService;
  }

  async render() {
    const username = await this._usernameService.getUsername();
    await uiService.render(`Hello, ${username}`);
  }
}

Now your application tests serve the same purpose as broad end-to-end tests: they document and test the externally-visible behavior of the application. Because they inject Null application dependencies, they're narrow tests, not broad tests, and they don't communicate with external systems. That makes them fast and reliable. They're also Overlapping Sociable Tests, so they provide the same safety net that broad tests do.

At this point, you have the beginnings of a walking skeleton: an application that works end-to-end, but is far from complete. You can evolve that skeleton to support more features. Choose some aspect of your code that's obviously incomplete and test-drive a slightly better solution. Repeat forever.

// JavaScript example: Application + read/write to infrastructure
// + respond to UI events

// Test code
it("renders user name", function() {
  const usernameService = UsernameService.createNull({ username: "my_username" });
  const uiService = UiService.createNull();
  const app = new MyApplication({ usernameService, uiService });
  app.start();

  uiService.simulateRequest("greeting");
  assert.equal("Hello, my_username", uiService.getLastRender());
});

// Production code
class MyApplication {
  constructor({ usernameService = UsernameService.create() }) {
    this._usernameService = usernameService;
    this._uiService = uiService;
  }

  async start() {
    this._uiService.on("greeting", () => {
      const username = await this._usernameService.getUsername();
      await uiService.render(`Hello, ${username}`);
    });
  }
}

At some point, probably fairly early, your Application layer class will start feeling messy. When it does, look for a concept that can be factored into its own class. This forms the seed of your Logic layer. As your application continues to grow, continue refactoring so that class collaborations are easy to understand and responsibilities are clearly defined.

To convert existing code to an A-Frame Architecture, Climb the Ladder instead.

Climb the Ladder

Most pre-existing code you encounter will be designed with a layered architecture, where Logic code has Infrastructure dependencies. Some of this code will be difficult to test or resist refactoring. Therefore:

Refactor problem code into a miniature A-Frame Architecture. Start at the lowest levels of your Logic layer and choose a single method that depends on one clearly-defined piece of infrastructure. If the Infrastructure code is intertwingled with the Logic code, disintertwingle it by factoring out an Infrastructure Wrapper.

When the Infrastructure code has been separated from the rest of the code, the method will act similarly to an Application layer class: it will have a mix of logic and calls to infrastructure. Make this code easier to refactor by rewriting its tests to use Nullable Infrastructure dependencies instead of mocks. Then factor all the logic code into methods with no infrastructure dependencies.

At this point, your original method will have nothing left but a small Logic Sandwich: a call or two to the infrastructure class and a call to the new logic method. Now eliminate the original method by inlining it to its callers. This will cause the logic sandwich to climb one step up your dependency chain.

Repeat until the class no longer has any dependencies on infrastructure. At that point, review its design and refactor as desired to better fit the Logic Patterns and your application's needs. Continue with the next class.

Climbing the Ladder takes a lot of time and effort, so do it gradually, as part of your normal work, rather than all at once. Focus your efforts on code where testing without mocks will have noticeable benefit. Don't waste time refactoring code that's already easy to maintain, regardless of whether it uses mocks.

When building a new system from scratch, Grow Evolutionary Seeds instead.

Zero-Impact Instantiation

Overlapping Sociable Tests instantiate their dependencies, which in turn instantiate their dependencies, and so forth. If instantiating this web of dependencies takes too long or causes side effects, the tests could be slow, difficult to set up, or fail unpredictably. Therefore:

Don't do significant work in constructors. Don't connect to external systems, start services, or perform long calculations. For code that needs to connect to an external system or start a service, provide a connect() or start()method. For code that needs to perform a long calculation, consider lazy initialization. (But even complex calculations aren't likely to be a problem, so profile before optimizing.)

Signature Shielding

As you refactor your application, method signatures will change. If your code is well-designed, this won't be a problem for production code, because most methods will only be used in a few places. But tests can have many duplicated method and constructor calls. When you change those methods or constructors, you'll have a lot of busywork to update the tests. Therefore:

If a file has a lot of tests that call a specific method, provide a proxy function for that method. Similarly, if it has a lot of tests that instantiate a class, provide a factory method. Program the proxies and factories so their parameters are all optional. That way you can add additional parameters in the future without breaking existing tests.

// JavaScript code with named, optional parameters

// Example test
it("uses hosted page for authentication", function() {
  const client = createClient({   // Use the factory function
    host: "my.host", 
    clientId: "my_client_id"
  });
  const url = getLoginUrl({   // Use the proxy function
    client, 
    callbackUrl: "my_callback_url"
  });
  assert.equal(url, "https://my.host/authorize?response_type=code&client_id=my_client_id&callback_url=my_callback_url");
});

// Example factory function
function createClient({
  host = "irrelevant_host",
  clientId = "irrelevant_id",
  clientSecret = "irrelevant_secret",
  connection = "irrelevant_connection"
} = {}) {
  return new LoginClient(host, clientId, clientSecret, connection);
}

// Example proxy function
function getLoginUrl({
  client,
  username = "irrelevant_username",
  callbackUrl = "irrelevant_url"
} = {}) {
  return client.getLoginUrl(username, callbackUrl);
}

Logic Patterns

When using A-Frame Architecture, the application's Logic layer has no infrastructure dependencies. It represents pure computation, so it's fast and deterministic. To qualify for the Logic layer, code can't talk to a database, communicate across a network, or touch the file system.2 Neither can its tests or dependencies. Any code that breaks these rules belongs in the Application layer or Infrastructure layer instead. Code that modifies global state can be put in the Logic layer, but it should be avoided, because then you can't parallelize your tests.

Pure computation is easy to test. The following patterns make it even easier.

This list inspired by Michael Feathers' unit testing rules.

Easily-Visible Behavior

Logic layer computation can only be tested if the results of the computation are visible to tests. Therefore:

Prefer pure functions where possible. Pure functions' return values are determined only by their input parameters.

When pure functions aren't possible, prefer immutable objects. The state of immutable objects is determined when the object is constructed, and never changes afterwards.

For methods that change object state, provide a way for the change in state to be observed, either with a getter method or an event.

In all cases, avoid writing code that explicitly depends on (or changes) the state of dependencies more than one level deep. That makes test setup difficult, and it's a sign of poor design anyway. Instead, design dependencies so they completely encapsulate their next-level-down dependencies.

Testable Libraries

Third-party code doesn't always have Easily-Visible Behavior. It also tends to introduce breaking API changes with new releases, or simply stop being maintained. Therefore:

Wrap third-party code in code that you control. Ensure your application's use of the third-party code is mediated through your wrapper. Write your wrapper's API to match the needs of your application, not the third-party code, and add methods as needed to provide Easily-Visible Behavior. (This will typically involve writing getter methods to expose deeply-buried state.) When the third-party code introduces a breaking change, or needs to be replaced, modify the wrapper so no other code is affected.

Frameworks and libraries with sprawling APIs are more difficult to wrap, so prefer libraries that have a narrowly-defined purpose and a simple API.

If the third-party code interfaces with an external system, use an Infrastructure Wrapper instead.

Parameterless Instantiation

Multi-level dependency chains are difficult to set up in tests. Dependency injection (DI) frameworks work around the problem, but we're avoiding magic like DI frameworks. Therefore:

Ensure that all Logic classes can be constructed without providing any parameters (and without using a DI framework). In practice, this means that most objects instantiate their dependencies in their constructor by default, although they may also accept them as optional parameters.

For some classes, a parameterless constructor won't make any sense. For example, an immutable "Address" class would be constructed with its street, city, and so forth. For these sorts of classes, provide a test-only factory method. The factory method should provide overridable defaults for mandatory parameters.

The factory method is easiest to maintain if it's located in the production code next to the real constructors. It should be marked as test-specific and should be simple enough to not need tests of its own.

// Example JavaScript code using named, optional parameters

class Address {
  // Production constructor
  constructor(street, city, state, country, postalCode) {
    this._street = street;
    this._city = city;
    //...etc...
  }

  // Test-specific factory
  static createTestInstance({
    street = "Address test street",
    city = "Address test city",
    state = State.createTestInstance(),
    country = Country.createTestInstance(),
    postalCode = PostalCode.createTestInstance()    
  } = {}) {
    return new Address(street, city, state, country, postalCode);
  }
}

Collaborator-Based Isolation

Overlapping Sociable Tests ensure that any changes to the semantics of a unit's dependencies will cause that unit's tests to break, no matter how far down the dependency chain they may be. On the one hand, this is nice, because we'll learn when we accidentally break something. On the other hand, this could make feature changes terribly expensive. We don't want a change in the rendering of addresses to break hundreds of unrelated reports' tests. Therefore:

Call dependencies' methods to help define test expectations. For example, if you're testing a InventoryReport that includes an address in its header, don't hardcode "123 Main St." as your expectation for the report header test. Instead, call Address.renderAsOneLine() as part of defining your test expectation.

// JavaScript example

// Example test
it("includes the address in the header when reporting on one address", function() {
  // Instantiate the unit under test and its dependency
  const address = Address.createTestInstance();
  const report = createReport({ addresses: [ address ] });

  // Define the expected result using the dependency
  const expected = "Inventory Report for " + address.renderAsOneLine();

  // Run the production code and make the assertion
  assert.equal(report.renderHeader(), expected);
});

// Example production code
class InventoryReport {
  constructor(inventory, addresses) {
    this._inventory = inventory;
    this._addresses = addresses;
  }

  renderHeader() {
    let result = "Inventory Report";
    if (this._addresses.length === 1) {
      result += " for " + this._address[0].renderAsOneLine();
    }
    return result;
  }  
}

This provides the best of both worlds: Overlapping Sociable Tests ensure that your application is wired together correctly and Collaborator-Based Isolation allows you to change features without modifying a lot of tests.

Infrastructure Patterns

The Infrastructure layer contains code for communicating with the outside world. Although it may contain some logic, that logic should be focused on making infrastructure easier to work with. Everything else belongs in the Application and Logic layers.

Infrastructure code is unreliable and difficult to test because of its dependencies on external systems. The following patterns work around those problems.

Infrastructure Wrappers

In the Logic layer, you can design your code to avoid complex, global state. In the Infrastructure layer, your code deals with nothing else. Testing infrastructure code that depends on other infrastructure code is particularly difficult. Therefore:

Keep your infrastructure dependencies simple and straightforward. For each external system--service, database, file system, or even environment variables--create one wrapper class that's responsible for interfacing with that system. Design your wrappers to provide a crisp, clean view of the messy outside world, in whatever format is most useful to the Logic and Application layers.

Avoid creating complex webs of dependencies. In some cases, high-level Infrastructure classes may depend on generic, low-level classes. For example, a LoginClient might depend on RestClient. In other cases, high-level infrastructure classes might unify multiple low-level classes, such as a DataStore class that depends on a RelationalDb class and a NoSqlDb class. Other than these sorts of simple one-way dependency chains, design your Infrastructure classes to stand alone.

Test your Infrastructure Wrappers with Focused Integration Tests and Paranoic Telemetry. Enable them to be used in other tests by creating Nullable Infrastructure.

Focused Integration Tests

Ultimately, Infrastructure code talks over a network, interacts with a file system, or involves some other communication with an external system. Its correctness depends on communicating properly. Therefore:

Test your external communication for real. For file system code, check that it reads and writes real files. For databases and services, access a real database or service. Make sure that your test systems use the same configuration as your production environment. Otherwise, your code will fail in production when it encounters subtle incompatibilities.

Run your focused integration tests against test systems that are reserved exclusively for one machine's use. It's best if they run locally on your development machine, and are started and stopped by your tests or build script. If you share test systems with other developers, you'll experience unpredictable test failures when multiple people run the tests at the same time.

You won't be able to get a local test system for every external system your application uses. When you can't, use a Spy Server instead.

Some high-level Infrastructure classes will use lower-level classes to do the real work, such as a LoginClient class that uses a RestClient class to make the network call. They can Fake It Once You Make It.

Spy Server

Some external systems are too unreliable, expensive, or difficult to use for Focused Integration Tests. It's one thing to run dozens of tests against your local file system every few minutes; quite another to do that to your credit card gateway. Therefore:

Create a test server that you can run locally. Program it to record requests and respond with pre-configured results. Make it very simple and generic. For example, all REST-based services should be tested by the same HTTPS Spy Server.

To test against the Spy Server, start by making a real call to your external system. Record the call and its results and paste them into your test code (or save them to a file). In your test, check that the expected request was made to the Spy Server and the response was processed correctly.

External systems can change out from under you. In the case of cloud-based services, it can happen with no warning. A Spy Server won't be able to detect those changes. To protect yourself, implement Paranoic Telemetry.

// Example Node.js LoginClient tests.

// Start, stop, and reset the Spy Server
const testServer = new HttpsTestServer();
before(async function() {
  await testServer.startAsync();
});
after(async function() {
  await testServer.stopAsync();
});
beforeEach(function() {
  testServer.reset();
});

// The test
it("gets user details", async function() {
  // Instantiate unit under test (uses Signature Shielding)
  const client = createNetworkedClient({
    managementApiToken: "my_management_api_token",
    connection: "my_auth0_connection",
  });

  // Set up Spy Server response
  testServer.setResponse({   
    status: 200,
    body: JSON.stringify([{
      user_id: "the_user_id",
      email_verified: false,
    }]),
  });

  // Call the production code
  const result = await client.getUserInfoAsync("a_user_email");

  // Assert that the correct HTTP request was made
  assert.deepEqual(testServer.getRequests(), [{
    method: "GET",
    url: "/api/v2/users?" +
      "fields=user_id%2Cemail_verified&" +
      "q=identities.connection%3A%22my_auth0_connection%22%20AND%20email%3A%22a_user_email%22&" +
      "search_engine=v2",
    body: "",
    headers: {
      host: testServer.host(),
      authorization: "Bearer my_management_api_token",
    },
  }], "request");

  // Assert that the response was processed properly
  assert.deepEqual(result, {
    userId: "the_user_id",
    emailVerified: false
  }, "result");
});

This is a complete example of a real-world Node.js HTTPS Spy Server. You can use this code in your own projects:

// Copyright 2018 Titanium I.T. LLC. All rights reserved. MIT License.
"use strict";

//** An HTTPS spy server for use by focused integration tests

const https = require("https");
const promisify = require("util").promisify;

const SELF_SIGNED_LOCALHOST_CERT_FOR_TESTING_ONLY =
  "-----BEGIN CERTIFICATE-----\n" +
  // TODO
  "-----END CERTIFICATE-----";

const CERT_PRIVATE_KEY_FOR_TESTING_ONLY =
  "-----BEGIN RSA PRIVATE KEY-----\n" +
  // TODO
  "-----END RSA PRIVATE KEY-----";

module.exports = class HttpsTestServer {
  constructor() {
    this._hostname = "localhost";
    this._port = 5030;
    this.reset();
  }

  reset() {
    this._forceRequestError = false;
    this._requests = [];
    this._responses = [];
  }

  hostname() { return this._hostname; }
  port() { return this._port; }
  host() { return this._hostname + ":" + this._port; }
  certificate() { return SELF_SIGNED_LOCALHOST_CERT_FOR_TESTING_ONLY; }

  getRequests() { return this._requests; }

  async startAsync() {
    const options = {
      cert: SELF_SIGNED_LOCALHOST_CERT_FOR_TESTING_ONLY,
      key: CERT_PRIVATE_KEY_FOR_TESTING_ONLY,
      secureProtocol: "TLSv1_method"
    };
    this._server = https.createServer(options);
    this._server.on("request", handleRequest.bind(null, this));

    await promisify(this._server.listen.bind(this._server))(this._port);
  }

  async stopAsync() {
    await promisify(this._server.close.bind(this._server))();
  }

  setResponses(responses) { this._responses = responses; }
  setResponse(response) { this._responses = [ response ]; }

  forceErrorDuringRequest() {
    this._forceRequestError = true;
  }
};

function handleRequest(self, request, response) {
  let responseInfo = self._responses.shift();
  if (responseInfo === undefined) responseInfo = { status: 503, body: "No response defined in HttpsTestServer" };

  const requestInfo = {
    method: request.method,
    url: request.url,
    headers: Object.assign({}, request.headers),
    body: ""
  };
  delete requestInfo.headers.connection;
  self._requests.push(requestInfo);

  if (self._forceRequestError) request.destroy();

  request.on("data", function(data) {
    requestInfo.body += data;
  });
  request.on("end", function() {
    response.statusCode = responseInfo.status;
    response.setHeader("Date", "harness_date_header");
    response.end(responseInfo.body);
  });
}

Paranoic Telemetry

External systems are unreliable. The only thing that's certain is their eventual failure. File systems lose data and become unwritable. Services return error codes, suddenly change their specifications, and refuse to terminate connections. Therefore:

Instrument the snot out of your infrastructure code. Assume that everything will break eventually. Test that every failure case either logs an error and sends an alert, or throws an exception that ultimately logs an error and sends an alert. Remember to test your code's ability to handle requests that hang, too.

All these failure cases are expensive to support and maintain. Whenever possible, use Testable Libraries rather than external services.

(An alternative to Paranoic Telemetry is Contract Tests, but they're not paranoid enough to catch changes that happen between test runs.)

Nullable Infrastructure

Focused Integration Tests are slow and difficult to set up. Although they're useful for ensuring that infrastructure code works in practice, they're overkill for code that depends on that infrastructure code. Therefore:

Program each infrastructure class with a factory method, such as "createNull()," that disables communication with the external system. Instances should behave normally in every other respect. This is similar to how Null Objects work. For example, calling LoginClient.createNull().getUserInfo("...") would return a default response without actually talking to the third-party login service.

The createNull() factory is production code and should be test-driven accordingly. Ensure that it doesn't have any mandatory parameters. (Nullable Infrastructure is the Infrastructure layer equivalent of Parameterless Instantiation.)

To implement Nullable Infrastructure cleanly, use an Embedded Stub. To test code that has infrastructure dependencies, use Configurable Responses, Send State, Send Events, and Behavior Simulation.

Embedded Stub

In order for Nullable Infrastructure to be useful to tests, null instances need to disable the external system while running everything else normally. The obvious approach is to use a flag and bunch of "if" statements, but that's a recipe for spaghetti. Therefore:

Stub out the third-party library that performs external communication rather than changing your infrastructure code. In the stub, implement the bare minimum needed to make your infrastructure code run. Ensure you don't overbuild the stub by test-driving it through your infrastructure code's public interface. Put the stub in the same file as your infrastructure code so it's easy to remember and update when your infrastructure code changes.

// Example Node.js wrapper for Socket.IO, a WebSockets library.
// Note how minimalistic the stub code is.

// Import real Socket.IO library
const io = require("socket.io");

// Infrastructure Wrapper
class RealTimeServer extends EventEmitter {

  // Instantiate normal wrapper
  static create() {
    return new RealTimeServer(io);
  }

  // Instantiate Null wrapper
  static createNull() {
    return new RealTimeServer(nullIo);
  }  

  // Shared initialization code
  constructor(io) {
    super();
    this._io = io;
    //...
  }

  // Normal infrastructure code goes here.
  // It's unaware of which version of Socket.IO is used.
}

// Null Socket.IO implementation is programmed here
function nullIo() {
  return new NullIoServer();
}

class NullIoServer {
  on() {}
  emit() {}
  close(done) { return done(); }
}

class NullSocket {
  constructor(id) { this.id = id; }
  get isNull() { return true; }
  emit() {}
  get broadcast() { return { emit() {} }; }
}

Fake It Once You Make It

Some high-level infrastructure classes depend on low-level infrastructure classes to talk to the outside world. For example, a LoginClient class might use a RestClient class to perform its network calls. The high-level code is typically more concerned with parsing and processing responses than the low-level communication details. However, there will still be some communication details that need to be tested. Therefore:

Use a mix of Focused Integration Tests and Nullable Infrastructure in your high-level infrastructure classes. For tests that check if external communication is done properly, use a Focused Integration Test (and possibly a Spy Server). For parsing and processing tests, use simpler and faster Nullable Infrastructure dependencies.

This Node.js JavaScript example demonstrates two tests of a LoginClient. The LoginClient depends on a RestClient. Note how the network request test uses a Spy Server and the error handling test uses a Null RestClient.

// Example Node.js tests for high-level LoginClient
// that depends on low-level RestClient.

describe("authentication", function() {
  // Network communication uses a Focused Integration Test and a Spy Server
  it("performs network request", async function() {
    // Instantiate the unit under test
    const client = createNetworkedClient({
      clientId: "my_auth0_id",
      clientSecret: "my_auth0_secret",
      managementApiToken: "my_management_api_token",
      connection: "my_auth0_connection",
    });

    // Set up Spy Server response
    testServer.setResponse({
      status: 200,
      body: JSON.stringify({
        id_token: createIdToken({
          email: "irrelevant_email_address",
          email_verified: false
        }),
      }),
    });

    // Call the production code
    await client.validateLoginAsync("login_code", "my_callback_url");

    // Assert that the correct HTTP request was made
    assert.deepEqual(testServer.getRequests(), [{
      method: "POST",
      url: "/oauth/token",
      body: JSON.stringify({
        client_id: "my_auth0_id",
        client_secret: "my_auth0_secret",
        code: "login_code",
        redirect_uri: "my_callback_url",
        grant_type: "authorization_code"
      }),
      headers: {
        host: testServer.host(),
        authorization: "Bearer my_management_api_token",
        "content-type": "application/json; charset=utf-8",
        "content-length": "148",
      },
    }]);
  });

  // Processing test uses Nullable Infrastructure RestClient dependency
  it("fails with error when HTTP status isn't 'okay'", async function() {
    // Instantiate unit under test with dependency configured to provide desired response
    const response = { status: 500, body: "auth0_response" };
    const client = createNulledClient(response);

    // Assert that the correct error was generated    
    await assert.exceptionAsync(
      () => validateLoginAsync(client),  // call production code
      expectedError(response, "Unexpected status code from Auth0.")  // expected error
    );
  });
});

// Factory for Focused Integration Tests
function createNetworkedClient({
  hostname = testServer.hostname(),
  port = testServer.port(),
  clientId = "irrelevant_id",
  clientSecret = "irrelevant_secret",
  managementApiToken = "irrelevant_token",
  connection = "irrelevant_connection",
} = {}) {
  if (port === null) port = undefined;
  return LoginClient.create({
    hostname,
    port,
    certificate: testServer.certificate(),
    clientId,
    clientSecret,
    managementApiToken,
    connection
  });
}

// Factory for Nullable Infrastructure tests
function LoginClient(responses) {
  return new Auth0Client({
    restClient: HttpsRestClient.createNull(responses),
    hostname: "irrelevant_hostname",
    clientId: "irrelevant_id",
    clientSecret: "irrelevant_secret",
    managementApiToken: "irrelevant_token",
    connection: "irrelevant_connection",
  });
}

Configurable Responses

Application and high-level infrastructure tests need a way of configuring the data returned by their infrastructure dependencies. Therefore:

Allow infrastructure methods' responses to be configured with an optional "responses" parameter to the Nullable Infrastructure's createNull() factory. Pass it through to the class's Embedded Stub and test-drive the stub accordingly.

When your infrastructure class has multiple methods that can return data, give each one its own createNull() parameter. Used named and optional parameters so they can be added and removed without breaking existing tests.

If a method needs to provide multiple different responses, pass them as an array. However, this may be a sign that your Infrastructure layer is too complicated.

// Example Node.js tests for Application layer code
// that reads data using LoginClient dependency

it("logs successful login", async function() {
  // Configure login client dependency
  const loginClient = LoginClient.createNull(
    validateLogin: {    // configure the validateLogin response
      email: "my_authenticated_email",
      emailVerified: true,
    }
  );
  const logCapture = LogService.createNull();

  // Run production code
  await performLogin({ loginClient, logCapture }));  // Signature Shielding

  // Check results
  assert.deepEqual(logCapture.logs, [ "Login: my_authenticated_email" ]);
});

To test code that uses infrastructure to send data, use Send State or Send Events. To test code that responds to infrastructure events, use Behavior Simulation.

Send State

Application and high-level infrastructure code use their infrastructure dependencies to send data to external systems. They need a way of checking that the data was sent. Therefore:

For infrastructure methods that send data, and provide no way to observe that the data was sent, store the last sent value in a variable. Make that data available via a method call.

// Example Send State implementation in JavaScript

class LoginClient {
  constructor() {
    //...
    this._lastSentVerificationEmail = null;
  }
  sendVerificationEmail(emailAddress) {
    //...
    this._lastSentVerificationEmail = emailAddress;
  }
  getLastSentVerificationEmail() {
    return this._lastSentVerificationEmail;
  }
  //...
}

// Example Node.js tests for Application layer code
// that sends data using LoginClient dependency

it("sends verification email", async function() {
  const loginClient = LoginClient.createNull();
  const emailPage = createPage({ loginClient });

  await emailPage.simulatePostAsync();
  assert.deepEqual(loginClient.getLastSentVerificationEmail(), "my_email");
});

If you need more than one send result, or you can't store the sent data, use Send Events instead. To test code that uses infrastructure to get data, use Configurable Responses. To test code that responds to infrastructure events, use Behavior Simulation.

Send Events

When you test code that uses infrastructure dependencies to send large blobs of data, or sends data multiple times in a row, Send State will consume too much memory. Therefore:

Rather than storing the sent data in a variable, use the Observer pattern to emit an event when your infrastructure code sends data. Include the data as part of the event payload. When tests need to make assertions about the data that was sent, they can listen for the events.

Send Events require complicated test setup. To make your tests easier to read, create a helper function in your tests that listens for send events and stores their data in an array. Doing this in production could cause a memory leak, but it's not a problem in your tests because the memory will be freed when the test ends.

This JavaScript example involves Application layer code for a real-time web application. When a browser connects, the server should send it all the messages the server had previously received. This test uses Send Events to check that the server sends those messages when a new browser connects.

// Example Node.js tests for Application layer code that sends
// multiple pieces of data using RealTimeServer dependency

it("replays all previous messages when client connects", function() {
  const network = createRealTimeServer();  // the infrastructure dependency
  const app = createApp({ network });  // the application code under test

  // Set up the test preconditions
  const message1 = new DrawMessage(1, 10, 100, 1000);
  const message2 = new DrawMessage(2, 20, 200, 2000);
  const message3 = new DrawMessage(3, 30, 300, 3000);
  network.connectNullBrowser(IRRELEVANT_ID);  // Behavior Simulation
  network.simulateBrowserMessage(IRRELEVANT_ID, message1);  // more Behavior Simulation
  network.simulateBrowserMessage(IRRELEVANT_ID, message2);
  network.simulateBrowserMessage(IRRELEVANT_ID, message3);

  // Listen for Send Events
  const sentMessages = trackSentMessages(network);

  // Run production code
  network.connectNullBrowser("connecting client");

  // Check that the correct messages were sent
  assert.deepEqual(sentMessages, [ message1, message2, message3 ]);
});

// Helper function for listening to Send Events
function trackSentMessages(network) {
  const sentMessages = [];
  network.on(RealTimeServer.EVENT.SEND_MESSAGE, (message) => {
    serverMessages.push(message);
  });
  return sentMessages;
}

To test code that uses infrastructure to get data, use Configurable Responses. To test code that responds to infrastructure events, use Behavior Simulation.

Behavior Simulation

Some external systems will push data to you rather than waiting for you to ask for it. Your application and high-level infrastructure code need a way to test what happens when their infrastructure dependencies generate those events. Therefore:

Add methods to your infrastructure code that simulate receiving an event from an external system. Share as much code as possible with the code that handles real external events, while remaining convenient for tests to use.

// Example Node.js Behavior Simulation implementation

class RealTimeNetwork {
  // Real Socket.IO event code
  _listenForBrowserMessages(socket) {
    socket.on("message", (payload) => {
      const message = Message.fromPayload(payload);
      _handleBrowserMessage(socket.id, message);
    }
  }

  // Simulated Socket.IO event code
  simulateBrowserMessage(clientId, message) {
    _handleBrowserMessage(clientId, message);
  }

  // Shared message-processing logic
  _handleBrowserMessage(clientId, message) {
    this.emit(RealTimeServer.EVENT.RECEIVE_MESSAGE, { clientId, message });
  }

  // Another example from the same class...

  // Real Socket.IO event code
  _listenForBrowserConnection(ioServer) {
    ioServer.on("connection", (socket) => {
      _connectBrowser(socket);
    }
  }

  // Simulated Socket.IO event code
  connectNullBrowser(browserId) {
    _connectBrowser(new NullSocket(browserId));
  }

  // Shared connection logic
  _connectBrowser(socket) {
    const id = socket.id;
    this._socketIoConnections[id] = socket;
    this.emit(RealTimeServer.EVENT.BROWSER_CONNECT, id);
  }

  //...
}

// Example Node.js tests for Application layer code
// that responds to events from RealTimeNetwork dependency

it("broadcasts messages from one browser to all others", function() {
  // Setup
  const network = createRealTimeNetwork();  // the infrastructure dependency
  const app = createApp({ network });  // the application code under test 
  const browserId = "browser id";
  const message = new PointerMessage(100, 200);
  network.connectNullBrowser(browserId);

  // Trigger event that runs code under test
  network.simulateBrowserMessage(browserId, clientMessage);

  // Check that code under test broadcasted the message
  assert.deepEqual(network.getLastSentMessage(), message);
});

To test code that uses infrastructure to get data, use Configurable Responses. To test code that uses infrastructure to send data, use Send Stream or Send Events.

Conclusion

These patterns are an effective way of writing code that can be tested without test doubles, DI frameworks, or end-to-end tests.

 

 

 

 

Top