Ruby on Rails 2.3 is almost there and brings a bunch of pretty cool new features. One of them is called templates and it allows you to customize the bootstrapping of new applications in order to automate the initial setup of your new app and hence get up to speed faster.

I find the naming a bit confusing: since we already have view templates in Rails I will refer to this new feature as application templates from now on.

So what does an application template do? Whenever you start a new rails application you are already using the Rails default template by issuing the rails command. It creates the usual app, config etc. folders and generates a bunch of scripts and configuration files for you. Application templates simply extend this process by allowing you to add your own setup steps. This is done by writing your own template file (or grabbing someone else’s).

Pratik Naik has already written an excellent tutorial on this so I’m not going to repeat what’s already written. Basically Rails offers you a few convenience methods for adding for example routes, plugins, gems or files to your new application.

So this blog post is more about showing off my very own template that I wrote today. (For the impatient: here it is).

Of course I’m not the first and only one who has written such a template so before I started I found this collection of templates which I immediately started to steal borrow from. As you probably know every programmer has its own style, uses his own unique set of tools in his (or her) own way. Which is why I found all of the templates I saw didn’t really fit my very own needs. The ones I found were pretty basic (to me) so when I rolled my own I took what I needed from what was there and added a whole bunch of my own stuff:

• setup of configuration files for Rspec, Cucumber and Culerity (Culerity is my plugin for driving Celerity – think Selenium but withour having to use a real browser – with Cucumber)
• a working user registration/login process using authlogic, including all controllers and views
• an XHTML application layout with jQuery (+ a few plugins) and blueprint CSS set up
• a simple capistrano deployment script
• German localization for all the built-in helpers
• a Thinking Sphinx configuration file

I pretty much need all of the above for 99% of my Rails projects and it costs me at least a day every time to set this all up. Well, not anymore Now I can run rails my_new_app -m http://gist.github.com/75038 and immediately after that start working on the distinct features.

I’m already planning to add a few extra. First of all the user authentication needs cucumber features so I don’t break anything when extending that. I also want to add configuration and deployment for a staging server. And maybe make the whole script a bit more configurable, i.e. install thinking sphinx or not etc.

The script is already pretty large so I’m not sure how much more I would want to put into it. Since you can apply templates to exisitng apps it will probably make sense to split the whole thing up at some point, so I could put the authentication part into its own template.

If you want to get started with application templates now I suggest you simply start with mine and/or the others and grab whatever you need. Just make sure you re-publish what you added so we can all build upon everyone’s work. Thanks.

On a recent Rails project we have been working a lot with PDF documents. The application generates PDF invoices and account statements, places markers and comments into existing documents and also merges multiple single page PDFs into one. We do all this with a few PDF tools:

• the PDF::Writer library for Ruby to generate PDFs
• pdftk – to merge multiple PDFs into one or overlay them
• pdftotext part of the xpdf library, to extract texts from PDFs

On OSX all of these can be installed via MacPorts. Debian has packages as well.

While PDF::Writer is a Ruby library pdftk is a command line tool. We simple call it using Kernel.system and check the return code via the $? variable.

unless system (“pdftk #{source_pdf} … output #{target_pdf}”)
raise PdfError(“pdftk returned error code #{$?}”)
end

With all this PDF processing the need for testing the contents of the generated documents arose. We have factored out all the PDF processing into a bunch of extra classes which we simply unit test with RSpec: make sure the parameters are passed to the command line correctly, that the right exception is thrown for each return code etc.

In addition to unit testing we also write customer driven acceptance tests with Cucumber, where we assert on a high level the outcome of certain actions. With HTML pages we can simply use the built-in steps that in turn use Webrat to parse the HTML like this:

Given a purchase over 200 EUR
And an invoice
When I go to the start page
And I follow “Invoice”
Then I should see “200 EUR”

Now in our case the invoice link links to a PDF but we still want to know what’s inside the document. The solution we came up with looks like this:

Given a purchase over 200 EUR
And an invoice
When I go to the start page
And I follow the PDF link “Invoice”
Then I should see “200 EUR”

What this does in the background is follow the link as usual, write the response into a temporary file, convert that to text using pdftotext and write the result back into the response. This way we can make assertions about the contents of the PDF almost as if it were an HTML page (except for tags of course). Here is the implementation:

When ‘I follow the PDF link “$label”‘ do |label|
click_link(label)
temp_pdf = Tempfile.new(‘pdf’)
temp_pdf << response.body
temp_pdf.close
temp_txt = Tempfile.new(‘txt’)
temp_txt.close
`pdftotext -q #{temp_pdf.path} #{temp_txt.path}`
response.body = File.read temp_txt.path
end

Since the day we started upstream testing was our obsession. When we began all we had was the Rails built-in unit, functional and integration tests. When we learned about RSpec the world became a brighter place. When the RSpec team released their StoryRunner we were thrilled. We started writing stories in English and were able to run them. Whoa. It soon turned out the first version had some serious issues with staying DRY. More versions were released, we rewrote our stories, step by step StoryRunner got better and we got better, too.

Then Cucumber came out and it was a huge leap ahead again. With the integration of Webrat we could write much shorter stories, but more importantly we could test that our links and forms were working as well. Everything was working perfectly and the world was a happy place – as long as we didn’t use any JavaScript/AJAX…

Welcome to Culerity. Culerity is a small gem that integrates Cucumber with Celerity, a wrapper around HtmlUnit which in turn is a Java library that parses HTML and runs the embedded JavaScript code. And unlike Watir or Selenium this all happens in headless mode: without hijacking your browser.

The problem with Celerity has been that it only works in a JRuby environment, which means you either had to run your application in JRuby as well (which might not even work with certain libraries and plugins) or somehow work your way around it by running your tests in JRuby and your application in whatever Ruby you wanted to use and somehow glue it together. Culerity now fills this gap. After installing it you run your Cucumber features as usual. Celerity now spawns a Java process in the background, sends all the Celerity calls to this process and evaluates the results back in the original Ruby environment — everything works (almost) as if you were running just in your single Ruby process. For an easier start Culerity comes with the same set of Cucumber step definitions that are provided by a default Cucumber/Webrat installation. This means you can reuse your step definitions written for Webrat.

To get started you can either clone the GitHub repository or simply install the gem (langalex-culerity). Then follow the instructions in the README. If you find something is not working for you or could be improved please feel free to fork the source code and show it some love. The codebase is really small right now and fully spec’d.

—

Does your company need help with software testing? We can help: check out our new product Scene Investigation.

Scotland on Rails will take place On March 26-28 2009 in Edinburgh. The organizers have just published the list of speakers. Pretty nice lineup including Jim Weirich, Yehuda Katz, Pat Maddox, Chad Fowler and more. And the best thing: I will be speaking about using CouchDB with Ruby/Rails, starring live coding and couch potato – my own little persistence layer for CouchDB.

When Hoptoad came out we were really excited. No more hundreds or even thousands of emails in case of some server failure or *gasp* even a bug deployed to the production server. Instead a nice looking website with a clean list. Nice.

Within days we had switched all projects from exception_notifier to Hoptoad – and then that was it. Occasionally Hoptoad would send an email with an exception but most of the time it would be some crawler requesting an invalid URL so we read and forgot about it. The problem was that with Hoptoad to forget about those errors had become be really easy – too easy. So last week a client told me the signup form on our project was showing an error page and nobody had been able to sign up for days.

Wait. We are monitoring the app and get notified about every exception. Of course we did but over the months we had accumulated more than 50 exceptions in Hoptoad and nobody was really paying attention to that big pile of false alarms caused by some crawler anymore. So yes, we did miss that positive alarm.

As a consequence we spent the same day going through all our exceptions in all projects and killed them all.

A big part of the exception came from crawlers requesting non-existent actions and stuff like that. With Rails 2.2 this is relatively easy to fix. In our routes.rb we have declared most of our routes as restful resources:

map.resources :posts

Now the PostsController does not implement the index action but we do get requests on /posts from time to time which result in an Unknown Action exception. The solution is to remove the index route:

map.resources :posts, :except => :index

In other cases we had to improve our guards on parameters a little. Passing an empty string for the page parameter to a paginate call seems to be a favorite pastime for the google crawler.

So, lessons learned: keep your exception inbox clean. Even if it’s just a bunch of bots.

If you write a custom template handler, e.g. for pdf templates you might want to use the rails resource route methods e.g. new_users_path. Here is the way how to get this working for a imaginary PDF template handler.

class PdfRender

attr_reader :controller
#delegate the url_for call to the controller
delegate :url_for, :to => :controller

def initialize(action_view)
@action_view = action_view
@controller = action_view.controller
end
… # more handler code
end
#install routes for the handler
ActionController::Routing::Routes.named_routes.install PdfRenderer

That couch was our intro picture at the workshop.
(picture from gizmodo.com)

Monday was the first event at our new upstream office, starring @janl and me presenting an introduction to CouchDB – including a new hands on examples part – and afterwards an overview of what can be done with CouchDB in Ruby so far. The talks were followed by a discussion that gave (hopefully not only) me a couple of new insights I want to share here – after summarizing the evening for the people who couldn’t make it. There will also be a video recording with synchronized slides of both talks be available in the next days (thanks @klimpong).

What is CouchDB

CouchDB ist the new cool kid on the block. It’s a document oriented database that has replication built in, can scale massively and uses an HTTP REST interface to query it. Documents are stored as JSON constructs and can be queried with views that are built using Map-Reduce (a smaller company called google has had a bit of success with that recently). Oh and it’s written in Erlang. Jan has given a number of talks on numerous events already, so there are already a couple of videos and slides available – not from the hands on part though For that you should watch his blog I guess.
(more…)

I find it rather hard to choose the right size for text boxes. They are either to small so that you feel like caged in while writing into them or they are so big that you feel lost in that great white emptiness. These times are past since someone came up with auto expanding text areas (e.g. facebook uses them).

For one of our rails apps we use John R. Wulff’s ‘text_area_with_auto_resize‘ plugin. The plugin will give you auto growing text boxes for all text_area calls. I wrote a little extension for prototype’s (1.6.x) in-place editing text area that makes these text areas auto growing as well. So we can have auto growing text areas all over the place.

If you ever meet a bunch of people sitting with their laptops in front of a tent in the middle of nowhere you might have stumbled across a rails camp. The rails camp is a new unconference where ruby hackers meet to exchange ideas and have fun in an open and inspiring environment. And what could be more open and inspiring when being torn out of your everyday internet/office/city routine by camping. Being cut off from the internet will bring up new approaches and insights to overcome programming challenges as well as bring the members of the ruby community more in touch with each other. Of course it’s also a hell of fun when you’re off from your daily routine with a lot to laugh, drink and play.

That’s why we will attend the rails camp in Denmark to meet our fellow rubyists (eg. from Oslo or the last Euruko) and might code some crazy ideas. The Rails Camp Denmark will likely take place at the end of October – the final date isn’t set yet - 24th October just outside Svendborg on the Danish island of Funen. If you want to join the rails camp you should invest 75 Euro and be prepared for camping over a weekend. More details and discussions can be found in the google group.

Hold your thumbs that the temperatures at the End of October don’t drop below zero so we won’t have to keep our mac books running to heat our tent.

I recently started using Webrat in one of our project’s test suites and fell in love with it immediately. Webrat is a functional testing library for Ruby on Rails which you can use in your rails integration tests (or better: the RSpec Story Runner) to let your tests walk through your application. The big difference between Webrat and a “standard” Rails Integration Test is that with Webrat you click on actual links and submit actual forms, instead of just sending requests to you application.

This closes an important gap to testing in a real browser (like with Selenium) and now gives us much more confidence in our tests – we now not only know that our models and controllers integrate, but we also know that they work with our views. That had been an unpleasant problem for a while. All our tests were passing but we still had way too many problems with invalid links and forms submiting to the wrong action. No more.

Webrat provides a sort of DSL (everything is a DSL nowaday isn’t it? ) for interacting with all the HTML elements you would find in a HTML document like links, checkboxes, radio buttons, text fields, selects and submit buttons. A test case might look like this:

visits ‘/’
clicks_link ‘Sign up’
fills_in ‘username’, :with => ‘joedoe’
selects ‘Male’
checks ‘Terms of Service’
clicks_button

Webrat now fetches the start page of the application, parses it and follows the first link labelled with “Sign Up”, gets the corresponding page, parses the form, fills in the values and submits it. Whenever a response comes back with anything else than a 200 (or 302, in which case that redirection is followed automatically) or an element on the page can’t be found, Webrat complains with an error and you know there’s probably something wrong in your view.

In Order to get this working with Story Runner I have wrapped most calls into When calls, for example:

When ‘I click on $link’ do |link|
clicks_link link
end
When ‘I submit the form’ do
clicks_button
end
When ‘I select $option’ do |option|
selects option
end

And now my Story looks like this:

Scenario: successful signup
When I go to the start page
And I click on Sign up
And I select Male
And I check the Terms of Service
And I submit the form
Then the page should show Thank you for signing up

I’m not sure yet if it makes sense to release those wrappers as a plugin, we’ll see. After all it’s just a few lines of trivial code.

By the way, Webrat is hosted on awesome github, so after using it for 30 minutes (seriously) I forked my own version and added support for the Rails date helpers (selects_date Date.today, :from => 'signup_date' instead of selects 'December', :from => 'signup_date_2i'; selects '2008', :from => 'signup_date_1i'; selects '01', :from => 'signup_date_3i') and Emails (e.g. clicking the activation link in a signup email). (Bryan Helmkamp, if you read this: Why haven’t you pulled this (and in fact any other fork) into the main line yet?)

All in all the experience has been fantastic. Webrat is on all my Rails projects now. The only – conceptual – problem so far is that I can’t use it to test AJAX. But that’s the topic of another post. And there’s a fork on github that experiments with celerity, which can do AJAX and everything.