Über Tests: fast, isolated integration tests for distributed apps

September 15, 2012 by Alexander Lang

TL;DR: when you split up your monolithic (web) app, you can either write integration tests spanning all the parts, which makes them slow and hard to set up, or you test each part in isolation and lose test coverage around the edges of your components. Über Tests try to solve the problem by testing those edges.

Suppose you have written a nice little Rails app (they all start out that innocent don’t they). Over time you add more and more features and suddenly you are looking at 15k LOC and a test suite (of course you have unit and integration tests for everything, don’t you) that takes 20 minutes to run.

You decide to go with the trend and split up your code base, say into one or more API apps and a frontend app. The backend(s) now expose a number of endpoints via HTTP and the frontend app uses those instead of directly talking to the database. This reduces coupling between the different modules and as your test suite gets split up with the app, you now only have to run the tests for the part of the app where you were making changes. Sweet.

There’s one problem though: what do you do with your integration tests? Keep them all in one place? Split them up?

After some headscratching you come up with three ideas:

1. Airbake - if it crashes you will get an email. Err, maybe not the best idea.
2. Write integration tests across the entire system. In addition to firing up your database, you simply run all your API services, too. The clear benefit here is that you don’t even have to change your existing tests. On the other hand those tests will run even slower now, with the added overhead of multiple processes and possibly databases. Didn’t we say we want fast tests? Also, setting those up will be a nightmare.
3. Test each piece of your app in isolation. When a feature you are testing requires a certain service/API you just stub it using for example WebMock. This approach allows you to split up your test suite, so you don’t have to run all the tests all the time. In addition they will run faster, too, as your stubbbed API will respond a lot quicker than a full stack. The downside here is that you are not testing the edges of your components anymore, e.g. how do you make sure your frontend app stays in sync with the backend API? If you change the API, will the frontend still work?

Here’s an example. Suppose you have a signup API endpoint and it returns the following JSON:

{"id": 1, "email": "joe@example.com", "access_token": "12345"}

You now want to change that API to return other data than just the user:

{
  "user": {"id": 1, "email": "joe@example.com", "access_token": "12345"},
  "links": [
    {"rel": "settings", "href": "https://api.example.com/account/settings"}
  ]
}

Which parts of your frontend do you have to change now? Are you not going to make any mistakes when stubbing out the new API? (yes you will).

Here’s where the idea of Über Tests™®© comes in: we take approach three, which gives us the advantage of fast and decoupled tests and add another suite of tests. These tests will make sure that the different components of our distributed system still work together. Here’s how it works:

First we write a small client for our API, something like this:

class SignupClient
  def initialize(http_client)
    @http_client = http_client # bear with me
  end

  def call(params)
    response = @http_client.post('/signups', {user: params})
    response.success?
  end
end

Step 2, we use that client for testing our backend (RSpec integration test example):

describe 'The signup API' do
  let(:signup_client) { SignupClient.new(http_client) }
  let(:http_client) {
    Faraday.new {|connection|
      connection.request :url_encoded
      connection.adapter :rack, app
    }
  }

  it 'signals success when sending valid user attributes' do
    expect(signup_client.call({email: 'joe@example.com'})).to be_success # let's not discuss RSpec syntax here
  end
end

As you can see we are using Faraday as our HTTP client. The cool thing about this is that it allows us to specify a Rack endpoint, and hence our integration test session (referenced by app) as the HTTP adapter. When calling the client in the test it will in turn call the post method of the integration test session. Nifty eh?

Okay, what do we have so far? We have a client for our API and since we are testing it along our backend we know those two are going to play along like BFFs. What’s left? We use that same client in the frontend:

class SignupsController
  def create
    if client.call(params[:user])
      redirect_to account_path, notice: 'Konichiwa!'
    else
      render 'new'
    end
  end

  def client
    SignupClient.new(Config.http_client)
  end
end

(By now we should probably put that client into its own gem so we can use the same code in the frontend app and for the backend tests.)

We set up the Config constant in environments/production.rb like so:

Config = OpenStruct.new http_client: Faraday.new {|f| f.adapter :net_http}

and for our test environment:

Config = OpenStruct.new

Lastly we add another test (using RSpec/Capybara) for our frontend:

describe 'Signing up' do
  let(:http_client) { stub(:http_client) }

  before(:each) do
    Config.stub(:http_client) { http_client }
    http_client.stub(:post) { {"user": {<...>}, "links": [...] }
  end

  it 'redirects to the account page on success' do
    visit signup_path
    fill_in 'Email', with: 'joe@example.com'
    click_button 'Sign up'

    expect(current_url).to eql('/account')
  end
end

In that test we use our API client but we stub the HTTP client, so that the test does not rely on the actual API endpoint to be running, but uses the code that we have already made sure works with it.

Now when we change our API there are two possible scenarios:

The change only affects the API Client. In this case we have to change the client’s code, which causes our frontend tests to break because the stubbed HTTP response don’t match the API client’s expectations anymore, so we have to fix only those.

The change causes the API client’s method signatures to change. In that case the same tests in the frontend app will fail where the frontend code calls the API client in an incorrect way.

In both cases our Über Tests catch the error and we can fix the code and tests. The rest of our integration tests reside in the different parts of our app: We have a bunch of backend tests that test the API endpoints and the stack behind it, plus a set of tests for the frontend app that only test that app using a stubbed API.

When you change something in the frontend you don’t need to run your backend tests again, as there’s no way you could have broken anything there. When you change your backend but nothing in the client you only have to run those tests. If you change the client run the client’s tests and the frontend tests, which are fast now because the run against a stubbed backend. Happy end.

This entry was posted on September 15, 2012 by Alexander Lang.