Posts Tagged ‘couchpotato’

Unit Testing CouchDB Views with Couch Potato

Friday, October 30th, 2009 by Alexander Lang

I just released Couch Potato 0.2.14 and amongst other things it has a new feature i think is pretty neat: you can unit test your (JavaScript) views using RSpec and Ruby.

You can declare a view in Couch Potato like this:

class Comment
  property :post_id
  view :by_post_id, :key => :post_id
end

This will generate a pair of map/reduce function and push them to Couch Potato. The map function looks something like this:

function(doc) {
  if(doc.ruby_class == 'Comment') {
    emit(doc.post_id, 1);
  }
}

And here’s a unit test for that function:

describe Comment, 'by_post_id' do
  it "should map to the post_id" do
    Comment.by_post_id.should map({:ruby_class => 'Comment', :post_id => 3}).to([3, null])
  end
end

As you can see all you have to do is pass a Ruby Hash that looks like your CouchDB document and the expected results. You can also pass in multiple results if you expect your map function to emit multiple key/value pairs.

Testing a reduce function works the same way:

describe Comment, 'by_post_id' do
  it "should reduce to the number of comments" do
    Comment.by_post_id.should reduce([], [1, 1, 1]).to(3)
  end
end

For testing re-reducing you simply call .should rereduce(...).to(...).

How it works

So how come you can test JavaScript functions in pure Ruby? Well, by stealing other people’s tricks. I recently contributed a few patches to mustache.js which is a new templating library ported to JavaScript by @janl. It has a test runner currently implemented in Ruby which generates JavaScript code on the fly, runs it using Spidermonkey and reads back the results. I have added a few steps to this process:

  1. A custom RSpec matcher collects the map function, a Ruby Hash representing the input and the expected output
  2. The input is converted to JavaScript using the JSON gem
  3. JavaScript code is generated that runs the document through the map function and prints the resulting JSON.
  4. The Ruby code runs spidermonkey, collects the output and parses it back to Ruby using again the JSON gem
  5. The results are compared to the expected values.

You can see how it works by looking at the code for the RSpec matchers.

I think this addition lowers the barrier to test your views quite a bit. Although I’m usually not a big fan of “one language to rule them all” and love writing JavaScript, being able to write all the necessary tests in Ruby when working on a Ruby project makes things way easier.

New Couch Potato: simple, testable, opinionated.

Sunday, May 17th, 2009 by Alexander Lang

After my talk about Ruby CouchDB frameworks at Scotland on Rails where I dismissed a few of of the libraries available (including my own Couhch Potato) as not fitting the CouchDB way of doing things, I have been hacking away the past few weeks working on a complete overhaul of Couch Potato.

As a first result I have just released version 0.2 of the framework. Its new goals are simplicity, embracing the CouchDB semantics and testability. In order to achieve this I had to introduce some major changes:

I disconnected models from the database – there are no more save/get/find methods in the models. Instead you can hand the models to a database object that will save/load documents for you – this allows for unit tests that are completely disconnected from the database

I have dropped associations and thrown away all the ActiveRecord like view creation/querying, replacing it with a new, more CouchDB like system. That new system is lighter, simpler and easier to extend.

The following paragraphs will show you how to work with the new Couch Potato.

Saving / loading models

As I said I have decoupled the models from the database, a model doesn’t have permanent access to the database anymore. Instead you instantiate a database object yourself and tell it to load or save a model object. This change isn’t so much about CouchDB as it is about testability. Having the database separated means you can now have true unit test of your models without talking to the database. Here is how you save/load models:

class Book
include CouchPotato::Persistence
property :title
end

CouchPotato::Config.database_name = ‘my_db’ # in Rails this is done for you
db = CouchPotato.database

book = Book.new :title => ‘The Passionate Programmer’ # good book
db.save! book # saves the book or raises an exception

db.load book._id # the original book

The database is responsible for running the model’s validations and lifecycle callbacks, saving the document to the database and afterwards setting the _id and _rev on the model. This way a lot of the persistence related logic is removed from the models making them more lightweight and most importantly easier to test (see below).

New Views

Overhauling of the views had two main goals:

  • provide a simple and extensible way for saving/querying views that works the way CouchDB works
  • since models don’t have access to the database anymore, find a new way to query them

Here is how you create and query a simple view:

class Book
view :by_title, :key => :title
property :title
end

db = CouchPotato.database
db.view Book.by_title # no parametters
db.view Book.by_title(:key => ‘The Passionate Programmer’, :descending => true) # just use any of the parameters CouchDB accepts

The way views work is now essentially reversed (inversion of control, rings a bell?). Instead of the view method calling the database the new view method creates a specification for a CouchDB view. The view is passed to the actual database in order to query CouchDB. Again this makes testing easier (see next section) but it also gives Couch Potato a clean interface in order to make creating views easier through abstractions. There is now a hierarchy of view specification classes that you can use to more easily create and query views. The above example used the CouchPotato::View::ModelViewSpec which is the default. This spec creates a view whose map function emits one (or an array of) attribute of the model and returns full documents.

Other view specs let you create more customized views, for example the RawViewSpec, which lets you define your own map/reduce functions and return the raw CouchDB results hash.

class Book
view :title_length, :type => :raw, :map => “function(doc) emit(doc.title.length, null)}”
property :title
end

db.view Book.title_length # returns something like {:rows => [{:key => 25}]}

For more examples see the Documentation in the CouchPotato::View::*ViewSpec classes.

Testing

As I have mentioned repeatedly decoupling the database from the models makes testing easier. With the new Couch Potato you can unit test your models without hitting the database once. First of all that makes your test lightning fast, and secondly it allows you to more easily test for example your lifecycle callbacks because you can call them directly passing them a stub or mock database.

Here’s an example: (with RSpec)

class Book
property :title
property :slug
end

describe Book, ‘create’ do
it ’should generate a slug’ do
book = Book.new :title => ‘The Passionate Programmer’
book.run_callbacks :before_create
book.slug.should == ‘the-passionate-programmer’
end
end

If you don’t like calling the run_callbacks method you can also use the actual database object but without a real connection to CouchDB. Couch Potato is based on the excellent CouchRest – a more low level CouchDB framework. The CouchPotato::Database constructor expects an instance of a CouchRest database which we can replace with a stub:

describe Book, ‘create’ do
it ’should generate a slug’ do
book = Book.new :title => ‘The Passionate Programmer’
db = CouchPotato::Database.new stub(‘couchrest database’, :save_doc => {})
db.save book
book.slug.should == ‘the-passionate-programmer’
end
end

For more details and examples please see the README, the RDocs and watch this blog. If you want to know all about Couch Potato I encourage you to read through its source code and specs – it’s not that much code actually. Note that all this is still work in progress and time will show how well all of this works. I would be happy to hear your feedback.

Couch Potato unleashed – a couchdb persistence layer in ruby (updated)

Monday, October 27th, 2008 by Alexander Lang

Update: the gem is now available, see the installation instructions below.

After several weeks of incubating on my computer it’s finally time to get real: I have just open sourced Couch Potato under the MIT license. You can get Couch Potato on github now. For an introduction to CouchDB and ruby please read my previous blog post A CouchDB primer for an ActiveRecord mindset. The following is a very short introduction into using Couch Potato. If you want to know more you can start with the README.

The goal of Couch Potato is to create a migration path for users of ActiveRecord and other object relational mappers to port their applications to CouchDB. It therefore offers a basic set of the functionality provided by most ORMs and adds functionality unique to CouchDB on top.

Installation

Couch Potato is available as a gem from http://gems.github.com, so you can just do

$ sudo gem source –add http://gems.github.com # if you haven’t alread
$ sudo gem install langalex-couch_potato

$ irb
>> require ‘rubygems’
>> gem ‘couch_potato’
>> require ‘couch_potato’
>> CouchPotato::Config.database_name = ‘name of the db’

Alternatively you can download the sources from github. If you are using rails just copy the files into vendor/plugins, create a RAILS_ROOT/config/couchdb.yml file (see the README for the format) and you are ready to go. For other applications you will have to require the lib/couch_potato.rb file and then set the database name by calling CouchPotato::Config.database_name = 'name of the db'.

As Couch Potato is still very young you can expect its feature set to grow quite a bit in the near future. What you can download now is the very core together with a few features giving you a glimpse of what is about to come:

Persistence

Create a new class and make its instances persistable by including the Persistence module. As there is no schema in a CouchDB you have to declare the properties you want to persist:

class User
include CouchPotato::Persistence

property :name
end

Now you can save your objects:

user = User.new :name => ‘joe’
user.save # or save!

Properties:

user.name # => ‘joe’
user.name = {:first => ['joe', 'joey'], :last => ‘doe’, :middle => ‘J’} # you can set any ruby object that responds_to :to_json
user._id # => “02097f33a0046123f1ebc0ebb6937269″
user.created_at # => Fri Oct 24 19:05:54 +0200 2008

You can of course also retrieve your instance:

User.get “02097f33a0046123f1ebc0ebb6937269″ # => < #User 0x3075>

Associations

As of now has_many and belongs_to are supported. By default the associated objects are stored in separate documents linked via foreign keys just like in relational databases.

class User
has_many :addresses, :dependent => :destroy
end

class Address
belongs_to :user
property :street
end

user = User.new
user.addresses.build :street => ‘potato way’
user.addresses.first # => < #Address 0x987>
user.addresses.create! # raises an exception as street is blank
user.addresses.first.user == user # => true

When saving an object all associated objects are automatically saved as well. All these save operations are sent to CouchDB in one operation which means the whole process is atomic across all objects saved, plus only one database roundtrip is required making it much faster.

As CouchDB can not only store flat structures you also store associations inline:

class User
has_many :addresses, :stored => :inline
end

This will store the addresses of the user as an array within your CouchDB document.

(more…)