It’s time to share some code from Astrid again!

We had a problem. In our database, tasks belong to users, and so when I load & read a list of 100 tasks (from cache and database), our system will load all 100 users as well. For many of these users, we’ll hit the cache instead of the database. It’s better than solely querying the database, but what tends to happen is those 100 tasks belong to the same user, and so we asking memcached for the same key over and over.

The solution? A per-request cache built into the Rails cache. Basically, what we’re going to do is create a lightweight in-memory cache, and add to the Rack middleware to clear the cache after every request. The hardest part of this project was probably getting Rails to recognize our new cache as a legitimate cache store in our environment .rb files.

The first thing we’ll do is to create a PerRequestCache. It gets inserted into the Rack middleware so that on every request, a new cache is created and then destroyed (for garbage collection purposes).

Hopefully, this code is really straightforward:

# THIS IS TOTALLY NOT THREAD SAFE!!!!!!!
# from https://gist.github.com/177780
class PerRequestCache 
 
  class < < self
    def open_the_cache
      @cache = {}
    end
 
    def clear_the_cache
      @cache = nil
    end
 
    def fetch(key, &block)
      return yield if @cache.nil?
      return @cache[key] if @cache.include? key
      @cache[key] = yield
    end
 
    def read(key)
      return nil if @cache.nil?
      @cache[key]
    end
 
    def write(key, value)
      return if @cache.nil?
      @cache[key] = value
    end
 
    def exist?(key)
      return false if @cache.nil?
      @cache.include? key
    end
 
    def delete(key)
      return if @cache.nil?
      @cache.delete key
    end
 
    def clear
      @cache = {}
    end
  end
 
  def initialize(app)
    @app = app
  end
 
  def call(env)
    self.class.open_the_cache
    @app.call(env)
  ensure
    self.class.clear_the_cache
  end
end

This plugs into the Rack middleware in environment.rb:

require 'per_request_cache'
 
Your::Application.configure do
  config.middleware.use PerRequestCache
end

From here, we created a new Store that subclasses ActiveSupport::Cache::Store. In the constructor, we read parameters for the underlying cache (e.g. memcached), and for each of our cache members, we try the PerRequestCache before calling through to the underlying implementation. For example,

def fetch(key, options = nil, &block) # :nodoc:
  PerRequestCache.fetch(key) do
    @underlying.fetch(key, options, &block)
  end
end

Then, when we're configuring our cache store, we use the following configuration to pass an underlying cache implementation (p.s. Dalli is an awesome replacement memcached client):

config.cache_store = :astrid_store, [ :dalli_store, '127.0.0.1:11211' ]

Oh, and the trick to getting :astrid_store to be recognized in environment files? Putting our ruby file in lib/active_support/cache.

It's a simple and elegant way to increase the efficiency of our cache by storing the results of our memcached queries in memory without having to deal with staleness. Tested with ruby-1.9.2 and rails-3.1.