Rails Fragment Caching - Testing and time based expiry

March 23, 2007 by alex

in the last days i started implementing caching for autoki.com. my first stop was this excellent rails caching tutorial over at railsenvy.com.

basically, rails offers 3 ways of caching page content:

• page caching: an entire page gets stored on the hard disk and can then be served by apache instead of rails - very fast but almost useless for us, as every page has some dynamic element in this. we still consider it for some ajax calls.
• action caching: also caches the entire page, but it's still served by rails, which means before_filters for stuff like authentication still work - still not for us, see above
• fragment caching: as the name implies caches fragments of a page - yay, sounds good

fragment caching

the basics are really easy. first, you surround the parts of the page that you want to cache with a <% cache do %> ... <% end %>. this writes the fragment to a file in /tmp/caches and from now on, rails serves this part of the page from the cache.

this alone doesn’t get you any performance improvements yet, because your controller is still loading all the data from the database before rendering the view. to avoid this, you wrap your data loading code into unless read_fragment 'name' ... end blocks - now you page should be running lightning fast already.

the third problem to tackle is to clean the cache at the right time. this can either be done in the controllers by calling expire_fragment 'name' or by using so called sweepers. they are basically observe your models and clean the right fragments on events like after_create - so when you add a new user to the system, you can clean the list of users from the cache.

time based expiry

sometimes, it’s not very efficient to clean the cache every time the underlying data changes. we have some site wide statistics for example, that require a load of processing power to calculate, and it’s enough if they get recalculated, say, every hour. rails doesn’t come with time bases cache expiry, but luckily, the timed_fragment_cache plugin comes to the rescue. it allows you to add an expiry time to your cache blocks in the erb templates and adds another method when_fragment_expired to you controllers, which allows you to test if a fragment has expired before running your data loading code.

testing it

we have some fairly complex pages which results into a multitude of cache blocks and sweeper calls on many many models, so i wanted to make sure to get it all right - how else could i do this than by using test first? rails doesn’t offer any support for testing its caching functionality so i had to use another plugin: the page cache test plugin. it basically offers two assertions: assert_caches_fragments makes sure that i have inserted a cache block in my template and that the fragment gets saved into the cache. assert_expire_fragments makes sure that my fragment gets removed from the cache when i hit a certain action. that’s all nice and good but a few things were missing: first, i wanted to test that my cache logic in the controller was working (i.e. i wasn’t loading data when i didn’t have to) and second, i wanted to test my time based expiring fragments as well.

testing caching in the controllers

my controller test does two things: first it hits the page and checks if the action has loaded a certain @variable - then it hits the same page again and checks that this time, the variable has not been loaded, i.e. the cache was hit instead. my custom assertion for that looks like this:

def assert_caches_fragments(*fragments, &block)
  yield
  fragments.each do |fragment|
    assert_not_nil(assigns(fragment[1]), "Fragment '#{fragment[0]}': the variable @#{fragment[1]} doesn't get set at all")
  end
  yield
  fragments.each do |fragment|
    assert_nil(assigns(fragment[1]), "Fragment '#{fragment[0]}' is not cached")
  end
end

now i can do this:

assert_caches_fragments ['/home/index/best_rated_auto', :best_rated_auto],
  ['/home/index/best_game_auto', :best_game_auto] do
    get '/'
  end

this calls the url ‘/’ and checks that the variable @best_rated_auto is filled with data. then it calls it again and makes sure rails now uses the ‘/home/index/best_rated_auto’ fragment instead. same for the second pair.

testing time based cache expiry

# time based fragment expiry cache test
# asserts that the given fragments are enclosed in when_fragment_expired calls
# fragments - 2 element arrays of [fragment_name, variable_name]
# variable_name - the name of @variable that gets set when the cache has expired
# &block - the call that hits the action, e.g. get '/'

def assert_expires_by_time(*fragments, &block)
  yield
  fragments.each do |fragment|
    assert_not_nil(assigns(fragment[1]), "Fragment '#{fragment[0]}': the variable @#{fragment[1]} doesn't get set at all")
  end
  yield
  fragments.each do |fragment|
    assert_nil(assigns(fragment[1]), "Fragment '#{fragment[0]}' is not cached at all")
    set_fragment_date fragment[0], 1.minute.ago
  end
  yield
  fragments.each do |fragment|
    assert_not_nil assigns(fragment[1]), "Fragment '#{fragment[0]}' is not time cached"
  end
end

def set_fragment_date(name, time)
  ActionController::Base.fragment_cache_store.write(name+'_meta', YAML.dump(time))
end

this code adds a bit more logic to the assertion in the last paragraph. after the second hit to the url it sets the fragment’s time so it is expired and hits the page a third time to check if the data comes from the database again.

plugin hacking

one last thing. in order not to fill my functional tests even more i decided to use integration tests for the cache testing. the fragment cache plugin has one limitation: in integration tests, it expects the fragment name to be a has, and that hash to contain a :controller => 'my_controller' pair. to work around this, i hacked the plugin the following way: in fragment_cache_test.rb i changed the first line of the check_options_has_controller method to this: if option = options.detect { |option| option[:controller].nil? } && !@controller - now i could keep my fragment names and still use integration tests. all i had to do was to define @controller = MyController.new in my test.

welcome to fully tested rails caching :) now comes all the boring stuff - implementing it everywhere.

This entry was posted on March 23, 2007 by alex.