I just released Couch Potato 0.2.14 and amongst other things it has a new feature i think is pretty neat: you can unit test your (JavaScript) views using RSpec and Ruby.

You can declare a view in Couch Potato like this:

class Comment
  property :post_id
  view :by_post_id, :key => :post_id
end

This will generate a pair of map/reduce function and push them to Couch Potato. The map function looks something like this:

function(doc) {
  if(doc.ruby_class == 'Comment') {
    emit(doc.post_id, 1);
  }
}

And here’s a unit test for that function:

describe Comment, 'by_post_id' do
  it "should map to the post_id" do
    Comment.by_post_id.should map({:ruby_class => 'Comment', :post_id => 3}).to([3, null])
  end
end

As you can see all you have to do is pass a Ruby Hash that looks like your CouchDB document and the expected results. You can also pass in multiple results if you expect your map function to emit multiple key/value pairs.

Testing a reduce function works the same way:

describe Comment, 'by_post_id' do
  it "should reduce to the number of comments" do
    Comment.by_post_id.should reduce([], [1, 1, 1]).to(3)
  end
end

For testing re-reducing you simply call .should rereduce(...).to(...).

How it works

So how come you can test JavaScript functions in pure Ruby? Well, by stealing other people’s tricks. I recently contributed a few patches to mustache.js which is a new templating library ported to JavaScript by @janl. It has a test runner currently implemented in Ruby which generates JavaScript code on the fly, runs it using Spidermonkey and reads back the results. I have added a few steps to this process:

1. A custom RSpec matcher collects the map function, a Ruby Hash representing the input and the expected output
2. The input is converted to JavaScript using the JSON gem
3. JavaScript code is generated that runs the document through the map function and prints the resulting JSON.
4. The Ruby code runs spidermonkey, collects the output and parses it back to Ruby using again the JSON gem
5. The results are compared to the expected values.

You can see how it works by looking at the code for the RSpec matchers.

I think this addition lowers the barrier to test your views quite a bit. Although I’m usually not a big fan of “one language to rule them all” and love writing JavaScript, being able to write all the necessary tests in Ruby when working on a Ruby project makes things way easier.

For doing Red/Green TDD it is essential to have fast running tests. But no matter how fast your tests are, there are always these seconds when you and your pair are sitting in front of the screen waiting that the test run actually starts. That is the time when Rails, your code, gems and plugins are loaded. In a rather small project, which I use as an example here, this cumulates into a start up time of around 5s before EACH test run. This becomes even longer when your project grows.

…..

Finished in 0.127571 seconds

5 examples, 0 failures

real 0m6.456s
user 0m4.682s
sys 0m1.681s

Although 5s, as in my example, aren’t such a long time, it sums up to a significant amount when doing Red/Green TDD. So everything that helps you to cut down test startup times, increases your productivity. AutoSpec or Rspactor approach this on the user site by running your test as soon as you change something. But they won’t cut down the technical implied start up time. Spec-server tried to approach the load time issue first by loading the Rails stack including the libraries of your environment upfront in an extra process. It uses Drb to get your test code to the pre-loaded testing process. Although the idea behind spec-server is the right one, it never worked for me in day to day usage. I came in situations in which tests failed or code wasn’t reloaded properly at all. But recently I discovered spork thanx to Rany (@purzelrakete). Spork takes the idea from spec-server and learned from its mistakes. E.g. it has its own class loading mechanism that takes care of not preloading your models at startup, instead of relying on rails auto loading. Beside working more reliable, this is also an option for other frameworks but Rails. It just works! Running the same test code than before with spork (spec paht_to_spec --drb), cuts the startup time into half.

…..

Finished in 0.066628 seconds

5 examples, 0 failures

real 0m3.588s
user 0m0.469s
sys 0m0.108s

When doing 200 test startups a day, a number which is not unrealistic, consider doing it Red/Green, this a 10 min time win You probably won’t need more than 10 min to set up spork.

1. Install spork

gem install spork

2. Bootstrap your Rails app


cd /path/to/project/root
spork --bootstrap


This command adds some instructions and code to spec/spec_helper.rb. Basically two blocks, one to tell which code to preload and one to tell which to reload on every test run. Code outside these blocks is run at both times.

3. Modify spec/spec_helper.rb

4. Modify spec/spec.opts

Add --drb to spec/spec.opts so that rake spec uses drb to communicate with spork. Set TM_RSPEC_OPTS --drb in TextMate so that the rspec TextMate bundle uses drb as well.

5. Fire up spork for rspec

spork rspec

6. Kickstart your RSpec Tests

If you find out that some of your code wasn’t reloaded properly, spork -d shows you which classes are loaded and where from. There shouldn’t be any of your app/classes in it. If that’s the case, the reason for this can be either a eager plugin (e.g. older versions of thinking_sphinx) or you used your code in initializers or the environment. Put such code in spork blocks like these in your spec_helper for the testing environment, this will prevent wrong preloading.

Here is an example how the spec_helper could look like:

require 'rubygems'
require 'spork'

ENV["RAILS_ENV"] = "test"

Spork.prefork do
  require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
  require 'spec'
  require 'spec/rails'
  require 'machinist/active_record'
end

Spork.each_run do
  require 'spec/blueprints'
end

Oh and btw, you can use spork for Cucumber as well, just modify cucumbers env.rb (same as you did with your spec_helper) and then launch spork with spork cucumber

And one last thing, as we are talking about faster test cycles, take a look at parallel _specs, to run your specs on multiple cores.

Happy TDDing!

On a recent Rails project we have been working a lot with PDF documents. The application generates PDF invoices and account statements, places markers and comments into existing documents and also merges multiple single page PDFs into one. We do all this with a few PDF tools:

• the PDF::Writer library for Ruby to generate PDFs
• pdftk – to merge multiple PDFs into one or overlay them
• pdftotext part of the xpdf library, to extract texts from PDFs

On OSX all of these can be installed via MacPorts. Debian has packages as well.

While PDF::Writer is a Ruby library pdftk is a command line tool. We simple call it using Kernel.system and check the return code via the $? variable.

unless system (“pdftk #{source_pdf} … output #{target_pdf}”)
raise PdfError(“pdftk returned error code #{$?}”)
end

With all this PDF processing the need for testing the contents of the generated documents arose. We have factored out all the PDF processing into a bunch of extra classes which we simply unit test with RSpec: make sure the parameters are passed to the command line correctly, that the right exception is thrown for each return code etc.

In addition to unit testing we also write customer driven acceptance tests with Cucumber, where we assert on a high level the outcome of certain actions. With HTML pages we can simply use the built-in steps that in turn use Webrat to parse the HTML like this:

Given a purchase over 200 EUR
And an invoice
When I go to the start page
And I follow “Invoice”
Then I should see “200 EUR”

Now in our case the invoice link links to a PDF but we still want to know what’s inside the document. The solution we came up with looks like this:

Given a purchase over 200 EUR
And an invoice
When I go to the start page
And I follow the PDF link “Invoice”
Then I should see “200 EUR”

What this does in the background is follow the link as usual, write the response into a temporary file, convert that to text using pdftotext and write the result back into the response. This way we can make assertions about the contents of the PDF almost as if it were an HTML page (except for tags of course). Here is the implementation:

When ‘I follow the PDF link “$label”‘ do |label|
click_link(label)
temp_pdf = Tempfile.new(‘pdf’)
temp_pdf << response.body
temp_pdf.close
temp_txt = Tempfile.new(‘txt’)
temp_txt.close
`pdftotext -q #{temp_pdf.path} #{temp_txt.path}`
response.body = File.read temp_txt.path
end

On Thursday I gave a talk about Bug fighting with TDD and how it will help you to write better software at the Ruby User Group in Berlin. Sorry, I won’t put the slides online, although people asked about it. I think they are a pretty useless collection of keywords and pictures without the talking. Instead I like to provide a collection of links to dive into the whole testing matters.

First a collection of tools you can choose from for various testing scenarios. They all have their pros and cons, it just comes down to which will do the job best for you.

Tools for Unit Testing

• Rspec
• Test/Unit (part of Rails)
• shoulda

Tools for Mocking

• Flexmock
• Mocha
• RR

Tools for Acceptance Testing

• Webrat
• Cucumber
• HtmlUnit (with JRuby)
• Selenium

Tools to generate test objects

• Machinist
• FactoryGirl
• ObjectDaddy

Misc tools

• Autotest
• Tarantula
• Culerity

To save you the initial searching where to start, I collected some links where you find basic information about testing in general or with certain tools.

Getting started

• A Guide to Testing Rails Applications
• An introduction to RSpec
• Functional Testing awesomeness with Webrat
• How I Test Controllers, 2009 Remix
• Shoulda tutorial
• Mock objects explained and intro to Mocha (de)

For everyone who wants to get more into the subject, I drilled down my feed reader to collect some interesting blog posts on testing matters.

Opinions and In-sign

• The Great Test Framework Dance-off
• Thoughts on Developer Testing
• Spikes, test-after, and learning how to test
• A case against a case against mocking and stubbing
• A critical look at the current state of Ruby testing
• One Line Tests Without the Smells
• Mocks Aren’t Stubs
• Testing as Communication: Real-World Techniques Notes
• When duplication in tests informs design
• Writing Code That Doesn’t Suck (vid)

So much for my two cents to make the software world better.

—

Does your company need help with software testing? We can help: check out our new product Scene Investigation.

Since the day we started upstream testing was our obsession. When we began all we had was the Rails built-in unit, functional and integration tests. When we learned about RSpec the world became a brighter place. When the RSpec team released their StoryRunner we were thrilled. We started writing stories in English and were able to run them. Whoa. It soon turned out the first version had some serious issues with staying DRY. More versions were released, we rewrote our stories, step by step StoryRunner got better and we got better, too.

Then Cucumber came out and it was a huge leap ahead again. With the integration of Webrat we could write much shorter stories, but more importantly we could test that our links and forms were working as well. Everything was working perfectly and the world was a happy place – as long as we didn’t use any JavaScript/AJAX…

Welcome to Culerity. Culerity is a small gem that integrates Cucumber with Celerity, a wrapper around HtmlUnit which in turn is a Java library that parses HTML and runs the embedded JavaScript code. And unlike Watir or Selenium this all happens in headless mode: without hijacking your browser.

The problem with Celerity has been that it only works in a JRuby environment, which means you either had to run your application in JRuby as well (which might not even work with certain libraries and plugins) or somehow work your way around it by running your tests in JRuby and your application in whatever Ruby you wanted to use and somehow glue it together. Culerity now fills this gap. After installing it you run your Cucumber features as usual. Celerity now spawns a Java process in the background, sends all the Celerity calls to this process and evaluates the results back in the original Ruby environment — everything works (almost) as if you were running just in your single Ruby process. For an easier start Culerity comes with the same set of Cucumber step definitions that are provided by a default Cucumber/Webrat installation. This means you can reuse your step definitions written for Webrat.

To get started you can either clone the GitHub repository or simply install the gem (langalex-culerity). Then follow the instructions in the README. If you find something is not working for you or could be improved please feel free to fork the source code and show it some love. The codebase is really small right now and fully spec’d.

—

Does your company need help with software testing? We can help: check out our new product Scene Investigation.

yesterday while skimming through my Twitter timeline I stumbled across aslak hellesoy’s cucumber. cucumber is a rewrite of the rspec story runner, allowing you to write user stories in plain text and make them executable by adding a bunch of ruby magic.

the features of both – story runner and cucumber – are basically the same. cucumber just gets rid of all these little glitches that disturb your work flow when working with story runner and adds that little bit of polish that makes the difference. but see for yourself:
(more…)

Wir sind gerade mit einem neuen Projekt gestartet – alles richtig “nach Lehrbuch”: kurze Iterationen, Iteration Planning zusammen mit dem Kunden, Behavior Driven Design, User Stories, automatisierte acceptance Tests – all die schönen Dinge die man in so einem modernen agilen Softwareprojekt haben will.

Während der Planung der ersten Iteration haben wir zusammen mit dem Kunden User Stories geschrieben, um sie anschließend direkt in den RSpec Story Runner zu werfen, also As a User I want to upload my photo So that everyone can see my smiley face. Da unsere Kunden deutsch sprechen war das Meeting und dementsprechend auch die Stories auf deutsch, also Als Benutzer will ich mein Foto hochladen können damit alle mein tolles Grinsegesicht sehen können. Damit der Story Runner damit umgehen konnte mussten wir ihm eine neue Sprache beibringen. So geht’s:

(more…)

The latest update from the rspec trunk changed the version.rb file indicating a 1.1 Release Candidate 1 version. Will there finally be a stable RSpec version that has StoryRunner? That’d be a great present for christmas.

Last week was RailsConf 2007 in Berlin. Only a couple of days have passed and it already seems so far away. Time for some blogging before everybody forgets that it even took place and nobody’s going to read this So here’s the interesting part of the sessions I attended:

A Half-day of Behavior-driven Development on Rails (rspec)

This was the first session on tutorial day and for me one of the most interesting. It basically gave a looong introduction to behavior driven design, how it evolved from things like TDD and who realized what while enganged in which project back in the good old times. One of the interesting parts for me was when they talked about the whole story writing/specification process. I had heard most of it before but it was a good refreshment. The central statement was to write “Software that matters”, and to achieve this, you’d have to get the specs right – as we XPers know this should be done by collecting user stories. The suggested format for such a story was this:

(more…)