Feedtools Database Cache

The ruby gem FeedTools has a built-in caching mechanism. Unfortunately, it doesn’t quite work out of the box, and the documentation doesn’t explain it very well. Despite what it says, you have to tell it to explicitly use the cache in your code:

#!/usr/bin/env ruby

require 'rubygems'
require 'feed_tools'

FeedTools.configurations[:feed_cache] = "DatabaseFeedCache"

puts "Getting feed 1st time..."
f = FeedTools::Feed.open('http://www.slashdot.org/index.rss')
puts "1. live? #{f.live?}"
puts "Getting feed 2nd time..."
f = FeedTools::Feed.open('http://www.slashdot.org/index.rss')
puts "2. live? #{f.live?}"

Assuming FeedTools can find a database.yml file and the database has the cached_feeds table described in the FeedTools documentation, the above script will cache the feed after the first time and f.live? will return false the second time you run it.

I also added an index on the href column in cached_feeds. All the lookups are done on that column and the migration that comes with FeedTools doesn’t have any indices. Here’s my migration for cached_feeds:

class AddFeedToolsTables < ActiveRecord::Migration
  def self.up
    create_table :cached_feeds do |t|
      t.column :href, :string
      t.column :title, :string
      t.column :link, :string
      t.column :feed_data, :text
      t.column :feed_data_type, :string
      t.column :http_headers, :text
      t.column :last_retrieved, :datetime
      t.column :time_to_live, :integer
      t.column :serialized, :text
    end
    add_index :cached_feeds, :href
  end

  def self.down
    drop_table :cached_feeds
  end
end