The more layers you add when testing your application the slower these tests become.

A unit test takes a fraction of a second to run. A simple integration test maybe up to a second. When JavaScript and a real browsers gets involved probably longer. But unnecessary long test runs hurt productivity. Up until a couple of weeks ago I had to make trade-offs either in speed, test-coverage or maintainability of my test suite when it came to real full stack integration/acceptance testing. Here is how I perceive these problems.

A standard Cucumber (a framework to write natural readable acceptance tests) setup uses Webrat which interacts with your app through a simple Rails integration session, so no JavaScript. The best what Webrat could do in this scenario, when it comes to JavaScript, is to recognize some common patterns in your html and send according requests – but still, most of the JavaScript code is not covered.

Webrat can also be used to drive Selenium, which interacts with your web app through a real browser. This will cover Javascript, CSS and possible browser issues. But compared to Webrat Selenium is snail mail slow.

With Culerity or Steam there are alternatives to Webrat which use HtmlUnit: HtmlUnit is a “GUI-Less browser for Java programs”. It models HTML documents and provides an API that allows you to load pages, fill out forms, click links, etc… just like you do in your “regular” browser. They are significantly faster than scripting a normal browser with Selenium at the cost that you don’t see how design works or if any browser issues cause trouble and it is still slower than Webrat without a special backend.

In order to run acceptance tests which require full JavaScript support separately from those who don’t to get shorter test runs I use two separate environments as they are generated out of the box either by Cucumber or Culerity. This leads to duplication of the setup code and step definitions, which means that there is more to maintain and more that could possible break.

Capybara addresses the shortcomings of a two environments solution and still allows you to be flexible in what backend you use for integration testing on a per scenario basis in Cucumber. It does that by using Cucumber’s tags feature.

Capybara “is inspired by and aims to replace Webrat as a DSL for interacting with a web application. It is agnostic about the driver running your tests and currently comes bundled with rack-test, Culerity and Selenium support built in”. In the future we certainly will see other backends. I saw that a backend for env-js (a pure JavaScript browser environment) is a already in the making.

Below is an example that runs one scenario with Selenium and the other with Culerity.

  @selenium
  Scenario: Searching a blog post using autocomplete
    Given a blog post titled "Autocomplete made easy" containing "Some example"
    When I go to the search page
      And I fill in "Search" with "Auto"
      And I follow "Autocomplete made easy"
    Then I should see "Some example"

  @culerity
  Scenario: delete found comments
    Given a comment by author "Thilo"
    Given a comment by author "Tilo"
  When I search for "ilo"
    And I check the element with xpath "//p[contains(@class,'delete')]/input"
    And I press "Submit"
  Then I should see "Are you really sure?"
  When I press "Yes"
  Then I should see "Comments were delete"
  And I should see "0 Comments"

You might have noticed the XPath expression in the second scenario. From time to time you will have to deal with XPath expressions when writing custom steps as XPath is used to locate elements within the DOM. The default steps can be scoped with CSS selectors or XPath, Capybara takes care of the conversion to XPath then.

Using Capybara with Cucumber is straightforward:
First get the gem:

gem install capybara

On OSX you may have to install libffi, you can install it via MacPorts with:

sudo port install libffi

Then go to your rails project and generate the cucumber files as followed:

script/generate cucumber --capybara

And you are ready to go to mix and match your acceptance with tags. For further informations like using capybara without cucumber, asynchronous JavaScript etc. checkout Capybara’s README.

Capybara definitely eases maintaining acceptance test and will give impulses to improve other tools, e.g. env-js.

Are you stuck in a maintenance nightmare or simply want to bring your testing skills to the next level? Get in touch with us.

For doing Red/Green TDD it is essential to have fast running tests. But no matter how fast your tests are, there are always these seconds when you and your pair are sitting in front of the screen waiting that the test run actually starts. That is the time when Rails, your code, gems and plugins are loaded. In a rather small project, which I use as an example here, this cumulates into a start up time of around 5s before EACH test run. This becomes even longer when your project grows.

…..

Finished in 0.127571 seconds

5 examples, 0 failures

real 0m6.456s
user 0m4.682s
sys 0m1.681s

Although 5s, as in my example, aren’t such a long time, it sums up to a significant amount when doing Red/Green TDD. So everything that helps you to cut down test startup times, increases your productivity. AutoSpec or Rspactor approach this on the user site by running your test as soon as you change something. But they won’t cut down the technical implied start up time. Spec-server tried to approach the load time issue first by loading the Rails stack including the libraries of your environment upfront in an extra process. It uses Drb to get your test code to the pre-loaded testing process. Although the idea behind spec-server is the right one, it never worked for me in day to day usage. I came in situations in which tests failed or code wasn’t reloaded properly at all. But recently I discovered spork thanx to Rany (@purzelrakete). Spork takes the idea from spec-server and learned from its mistakes. E.g. it has its own class loading mechanism that takes care of not preloading your models at startup, instead of relying on rails auto loading. Beside working more reliable, this is also an option for other frameworks but Rails. It just works! Running the same test code than before with spork (spec paht_to_spec --drb), cuts the startup time into half.

…..

Finished in 0.066628 seconds

5 examples, 0 failures

real 0m3.588s
user 0m0.469s
sys 0m0.108s

When doing 200 test startups a day, a number which is not unrealistic, consider doing it Red/Green, this a 10 min time win You probably won’t need more than 10 min to set up spork.

1. Install spork

gem install spork

2. Bootstrap your Rails app


cd /path/to/project/root
spork --bootstrap


This command adds some instructions and code to spec/spec_helper.rb. Basically two blocks, one to tell which code to preload and one to tell which to reload on every test run. Code outside these blocks is run at both times.

3. Modify spec/spec_helper.rb

4. Modify spec/spec.opts

Add --drb to spec/spec.opts so that rake spec uses drb to communicate with spork. Set TM_RSPEC_OPTS --drb in TextMate so that the rspec TextMate bundle uses drb as well.

5. Fire up spork for rspec

spork rspec

6. Kickstart your RSpec Tests

If you find out that some of your code wasn’t reloaded properly, spork -d shows you which classes are loaded and where from. There shouldn’t be any of your app/classes in it. If that’s the case, the reason for this can be either a eager plugin (e.g. older versions of thinking_sphinx) or you used your code in initializers or the environment. Put such code in spork blocks like these in your spec_helper for the testing environment, this will prevent wrong preloading.

Here is an example how the spec_helper could look like:

require 'rubygems'
require 'spork'

ENV["RAILS_ENV"] = "test"

Spork.prefork do
  require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
  require 'spec'
  require 'spec/rails'
  require 'machinist/active_record'
end

Spork.each_run do
  require 'spec/blueprints'
end

Oh and btw, you can use spork for Cucumber as well, just modify cucumbers env.rb (same as you did with your spec_helper) and then launch spork with spork cucumber

And one last thing, as we are talking about faster test cycles, take a look at parallel _specs, to run your specs on multiple cores.

Happy TDDing!

Ruby on Rails 2.3 is almost there and brings a bunch of pretty cool new features. One of them is called templates and it allows you to customize the bootstrapping of new applications in order to automate the initial setup of your new app and hence get up to speed faster.

I find the naming a bit confusing: since we already have view templates in Rails I will refer to this new feature as application templates from now on.

So what does an application template do? Whenever you start a new rails application you are already using the Rails default template by issuing the rails command. It creates the usual app, config etc. folders and generates a bunch of scripts and configuration files for you. Application templates simply extend this process by allowing you to add your own setup steps. This is done by writing your own template file (or grabbing someone else’s).

Pratik Naik has already written an excellent tutorial on this so I’m not going to repeat what’s already written. Basically Rails offers you a few convenience methods for adding for example routes, plugins, gems or files to your new application.

So this blog post is more about showing off my very own template that I wrote today. (For the impatient: here it is).

Of course I’m not the first and only one who has written such a template so before I started I found this collection of templates which I immediately started to steal borrow from. As you probably know every programmer has its own style, uses his own unique set of tools in his (or her) own way. Which is why I found all of the templates I saw didn’t really fit my very own needs. The ones I found were pretty basic (to me) so when I rolled my own I took what I needed from what was there and added a whole bunch of my own stuff:

• setup of configuration files for Rspec, Cucumber and Culerity (Culerity is my plugin for driving Celerity – think Selenium but withour having to use a real browser – with Cucumber)
• a working user registration/login process using authlogic, including all controllers and views
• an XHTML application layout with jQuery (+ a few plugins) and blueprint CSS set up
• a simple capistrano deployment script
• German localization for all the built-in helpers
• a Thinking Sphinx configuration file

I pretty much need all of the above for 99% of my Rails projects and it costs me at least a day every time to set this all up. Well, not anymore Now I can run rails my_new_app -m http://gist.github.com/75038 and immediately after that start working on the distinct features.

I’m already planning to add a few extra. First of all the user authentication needs cucumber features so I don’t break anything when extending that. I also want to add configuration and deployment for a staging server. And maybe make the whole script a bit more configurable, i.e. install thinking sphinx or not etc.

The script is already pretty large so I’m not sure how much more I would want to put into it. Since you can apply templates to exisitng apps it will probably make sense to split the whole thing up at some point, so I could put the authentication part into its own template.

If you want to get started with application templates now I suggest you simply start with mine and/or the others and grab whatever you need. Just make sure you re-publish what you added so we can all build upon everyone’s work. Thanks.