Go wrapper for Tokyo Dystopia

To go along with the Tokyo Tyrant Go wrapper I wrote, I also wrote a Tokyo Dystopia Go wrapper.

I’m using it for the fast online foreign language dictionary on Langalot. So far, it’s blazing fast.

Go wrapper for Tokyo Tyrant

Just released a little open source Go wrapper for Tokyo Tyrant:
gotyrant.

It’s far from complete…it only does what I’ve needed so far for a side project, but it could be useful for somebody.

SPF Records with Rails and ActionMailer

An SPF record is a DNS record that works like a reverse MX record. MX records tell the internet what machines can receive mail for a domain, SPF records specify which machines can send mail for a domain. They aren’t required by any means, but they are a good idea to help mail pass spam filters.

There is no such thing as an SPF record in the DNS spec, they are just TXT records. There are several online tools to help you create the right TXT record for your specifications.
I used openspf.org. For example, for one of my servers, it generated the following SPF record:

"v=spf1 a ~all"

This basically means that only the A record for this domain is allowed to send mail. The ~all means to “soft” deny mail from all other hosts. Changing it to -all would make it a “hard” deny.

If you also use google apps for mail for a domain, you can also add include:aspmx.googlemail.com, so it will look like:

"v=spf1 a include:aspmx.googlemail.com ~all"

This means the A record server(s) can send mail for this domain as well as all the google servers.

If you use rails, you might have to do some other work. Say you have a domain mycoolapp.com and have setup up your SPF record correctly. A mail recipient will most likely look at the Return-Path header when validating the SPF record. I always use sendmail as the ActionMailer delivery method:

ActionMailer::Base.delivery_method = :sendmail 

and by default, ActionMailer will send it as the user running the script or application.
So if a cron job is running under the runner account and the default hostname for your server is mistral.example.com (or mail.mycoolapp.com, but not mycoolapp.com), the Return-Path will end up being runner@mistral.example.com. Thus, even if with your correct SPF record, the mail recipient will be looking for an SPF record for mistral.example.com.

This is easy to fix by setting a header in the ActionMailer. For example:

def verify(subscription)
  subject "Verify your subscription"
  recipients subscription.email
  from "Some Robot <nobody@mycoolapp.com>"
  body :subscription => subscription
  headers 'return-path' => 'nobody@mycoolapp.com'
end

This will cause ActionMailer to add -f nobody@mycoolapp.com when it calls sendmail and the return path will be set correctly.

To verify everything is working well, send an email to a gmail account and view the original message. If you see something like:

Received-SPF: pass (google.com: domain of nobody@mycoolapp.com designates 99.99.99.99 as permitted sender) client-ip=99.99.99.99;
Authentication-Results: mx.google.com; spf=pass (google.com: domain of nobody@mycoolapp.com designates 99.99.99.99 as permitted sender) smtp.mail=nobody@mycoolapp.com

Then everything is set up perfectly!

Autotest with Rails Tip -- Ignore Files for Less CPU Usage

By default, autotest polls every file in your rails project for changes. By telling it to ignore certain files, you can drastically reduce its CPU usage. It looks for a config file named .autotest either in the working directory or your home directory. For a rails project, my .autotest file looks like this:

Autotest.add_hook :initialize do |autotest|
  %w{.git .DS_Store ._* vendor}.each do |exception|
    autotest.add_exception(exception)
  end
end

You can also use autotest-fsevent (more info here), which eliminates the polling altogether.

CIFilter on iPhone? Nope.

If you look in CALayer.h in the iPhone SDK, you’ll see these properties:

/* An array of filters that will be applied to the contents of the
 * layer and its sublayers. Defaults to nil. Animatable. */

@property(copy) NSArray *filters;

/* An array of filters that are applied to the background of the layer.
 * The root layer ignores this property. Animatable. */

@property(copy) NSArray *backgroundFilters;

While the reference manual is (sort of) clear that the CIFilter class is unavailable:

filters

An array of CoreImage filters that are applied to the contents of the receiver and its sublayers. Animatable.

Special Considerations

While the CALayer class exposes this property, Core Image is not available in iPhone OS. Currently the filters available for this property are undefined.

It gets confusing because their example code (in the iPhone doc set) uses a CIFilter:

// The selection layer will pulse continuously.
// This is accomplished by setting a bloom filter on the layer
 
// create the filter and set its default values
CIFilter *filter = [CIFilter filterWithName:@"CIBloom"];
[filter setDefaults];
[filter setValue:[NSNumber numberWithFloat:5.0] forKey:@"inputRadius"];
 
// name the filter so we can use the keypath to animate the inputIntensity
// attribute of the filter
[filter setName:@"pulseFilter"];
 
// set the filter to the selection layer's filters
[selectionLayer setFilters:[NSArray arrayWithObject:filter]];

But before you get your hopes up, don’t bother trying this. It won’t compile…

Create new git repository with gitosis

Assuming gitosis already set up and working…

Edit gitosis-admin/gitosis.conf and add the new repository.

[group company]
members = jackblack
writable = examproj

Commit this and push it.

git commit -a -m "added examproj"
git push

In a local repository (you’ve already called git init and done your initial commit), push the repository to the server.

git remote add origin git@code.example.com:examproj.git
git push origin master:refs/heads/master

Then clone it to where you want it.

cd ~/git
git clone git@code.example.com:examproj.git
git tags

Some notes on how to use tags in git:

You should always use the -a switch (or sign it). Explanation here.

To tag and the push to remote:

git tag -a v1.0.0 -m "version 1.0!!!"
git push --tags

Then, in the future, to get the code at the tag point and do stuff with it:

git checkout -b fromtag1.0 v1.0.0
<do some stuff>
git commit ...

And then back in the master, you can merge that branch back in:

git checkout master
git merge fromtag1.0
git push

That’s enough for now…

Max OS X Animated GIF Editor Champion -- Pixen

I needed to edit an animated gif on Mac OS X today…tried Acorn, but it doesn’t seem to support them. Tried Gimp, but it was unusable…could be user error, but every tool I selected would not stay selected and I couldn’t paste between documents.

Then I downloaded Pixen and despite it crashing the first time I tried saving the GIF, it worked great. And it’s free.

One note: if you use multiple layers in the frames, you need to merge them into one layer for the output to be correct.

Authlogic, Passenger, and cookie_store Sessions Don't Mix

There seems to be a problem with Authlogic, Passenger, and cookie_store sessions. You can read some details on this github commit.

A lot of the commenters keep saying that it gets fixed for them with newer versions of rails/passenger, but I’m having trouble with the rails 2.3.4, passenger 2.2.5, and authlogic 2.1.2 (all the latest versions as of this post). It was working fine at some point…

The fix? Use the old :active_record_store.

Add the following to config/environment.rb:

config.action_controller.session_store = :active_record_store

Then run

rake db:sessions:create

For PostgreSQL, I changed the migration it generates to the following:

class CreateSessions < ActiveRecord::Migration
  def self.up
    create_table :sessions do |t|
      t.text :session_id, :null => false
      t.text :data
      t.timestamps
    end

    add_index :sessions, :session_id
    add_index :sessions, :updated_at
  end

  def self.down
    drop_table :sessions
  end
end   

(column type for :session_id changed from string to text).

I imagine that :mem_cache_store would fix it as well…

Rails to_xml with procs

Ever wanted to add something to the xml representation of an ActiveRecord object that wasn’t a method or an include? You can with procs.

respond_to do |format|
  format.xml do
    proc = Proc.new { |options| options[:builder].tag!('saved', current_user.saved?(@page)) }
    render :xml => @page.to_xml(:include => [:user], :procs => [proc])
  end
end

And now this will spit out:

<?xml version="1.0" encoding="UTF-8"?>
<page>
  <content>So here's line 1</content>
  <created-at type="datetime">2009-10-15T16:50:57Z</created-at>
  <dictionary-id type="integer">1</dictionary-id>
  <saved>false</saved>
</page>

In this case, the model doesn’t know anything about the user and I wanted to keep it that way, but the XML api call needs to include the saved state.

Vim Objective-C Colon Indentation

Vim’s objective-c mode has an annoying indentation issue. It tries to line colons up on subsequent lines, like so:

[self performSelectorOnMainThread:@selector(finishLoad) 
                       withObject:nil 
                    waitUntilDone:NO];

That’s nice, but it has an annoying bug that makes it try to align the colons with a previous line that is unrelated:

if ([someVariable isEqualToString:@"blah"]) {
[self performSelectorOnMainThread:@selector(finishLoad) 
                       withObject:nil 
                    waitUntilDone:NO];
}

This all takes place in indent/objc.vim (full path on a MacPorts installation is /opt/local/share/vim/vim72/indent/objc.vim, for MacVim it is /Applications/MacVim.app/Contents/Resources/vim/runtime/indent/objc.vim).

There is probably a way to fix this in a .vimrc file, but I just edited objc.vim. At the bottom of the file is a function GetObjCIndent. Change the second if statement to match the following:

function GetObjCIndent()
    let theIndent = cindent(v:lnum)

    let prev_line = getline(v:lnum - 1)
    let cur_line = getline(v:lnum)
        
    if prev_line !~# ":" || cur_line !~# ":"
        return theIndent
    endif
     
    if prev_line !~# ";" && prev_line !~# "{"
        let prev_colon_pos = s:GetWidth(prev_line, ":")
        let delta = s:GetWidth(cur_line, ":") - s:LeadingWhiteSpace(cur_line)
        let theIndent = prev_colon_pos - delta
    endif
        
    return theIndent
endfunction

There are some refinements that could be made to fix it for more cases, but this one fixes the one that was driving me crazy.

Vim ack plugin with MacVim and MacPorts

The vim ack plugin doesn’t work if you use MacVim and MacPorts. The problem is the path (/opt/local/bin/ack). MacVim doesn’t get its path from your login shell (despite what the preferences page says) and /opt/local/bin isn’t a default path.

The way to fix it is to add a file to /etc/paths.d with one line it:

/opt/local/bin

You can call the file whatever you want. Restart MacVim and :Ack will work.

Safari Keyboard Shortcuts To Switch Tabs

zebras Photo by wwarby

I am so used to the cmd-1, cmd-2, cmd-3, cmd-9 keyboard shortcuts in Firefox to switch to the first, second, third, and last tabs in a window. Whenever I try to use Safari and I instinctually use these shortcuts, it goes to bookmarks (1, 2, 3, and 9), which is incredibly annoying. Apple provides no way to change this.

With Leopard, I was using Safari Commander, but it stopped working with Snow Leopard. There are hints on twitter that a new version is coming out soon that will work, but it requires Safari to run in 32-bit mode. And Firefox seems to be slower than usual under Snow Leopard.

So the solution? Fastscripts. Thanks to this blog post by Justin Blanton, I now have scripts to switch to tabs one through five and the last tab. I modified his scripts to be for Safari instead of WebKit. They are each one line long, but in case you’re too lazy to type them in, here they are for you:

Put them in ~/Library/Scripts/Applications/Safari then assign them to the correct keys in the Script Shortcuts preferences pane in FastScripts.

Jekyll Script

Speaking of jekyll, there’s a way to get git to publish using post-commit hooks, but I like running a little script to do it. It just runs jekyll to build the site, then calls rsync to make it live (it connects to the server over ssh).

#!/bin/sh

echo "running jekyll"
jekyll --no-auto
echo "rsyncing site with blog server"
cd _site && rsync -Cavz --delete . example.com:/var/www/blog.example.com
echo "done"

That’s it…

Jekyll Exclude Files

By default, jekyll will copy everything in the source directory into _site, but there’s an undocumented feature that lets you exclude or ignore files. A configuration variable named exclude exists. Any files or directories listed will be ignored. For example, the following is in my _config.yml file:

exclude: makelive.sh, ideas.txt, newpost.rb

so that some scripts and a text file of ideas aren’t published on the live site.

Google Analytics API with Ruby

As the old adage goes, “You can’t manage what you don’t measure.” While I’ve always used Google Analytics on all my sites, I find it a pain to log in to the site to just see some basic information like page views. It allows you to set up reports that get mailed to you every day, but these are PDFs and basically screen shots of the site. Better, but I just want to see some basic stats for my sites in an email message every morning.

Thankfully, there’s an API for Google Analytics. And if there’s an API there are more than one Ruby libraries for accessing it: garb and gattica. I chose gattica because it looked simpler.

In a few minutes, I had the perfect little script written. Here’s the meat of it:

metrics = ['pageviews']
ga = Gattica.new({:email => 'bbbbbb@gmail.com', :password => 'xxxxxxxx'})
ga.accounts.each do |account|
  ga.profile_id = account.profile_id
  daydata = ga.get({:start_date => yesterday, :end_date => today, :metrics => metrics})
  weekdata = ga.get({:start_date => week, :end_date => today, :metrics => metrics})
  monthdata = ga.get({:start_date => month, :end_date => today, :metrics => metrics})
  data[account.title] = { :day => daydata.points.first.metrics.first[:pageviews],
                          :week => weekdata.points.first.metrics.first[:pageviews],
                          :month => monthdata.points.first.metrics.first[:pageviews] }
end

rows = ""
data.keys.sort.each do |site|
  puts "#{site}:\t#{data[site][:day]}, #{data[site][:week]}, #{data[site][:month]}"
  rows << "<tr><td><b>#{site}</b></td><td>#{data[site][:day]}</td><td>#{data[site][:week]}</td><td>#{data[site][:month]}</td></tr>\n"
end

message = <<MESSAGE_END
From: Robot <info@xxxxxxxx.com>
To: Somebody <somebody@yyyyyyyyy.com>
MIME-Version: 1.0
Content-type: text/html
Subject: Analytics

Site analytics

<table cellpadding="10" cellspacing="0" border="1">
<tr><th>site</th><th>day</th><th>week</th><th>month</th></tr>
#{rows}
</table>
MESSAGE_END

pipe = open("|/usr/sbin/sendmail -t", "w")
pipe.write(message)
pipe.close

I added a cron job and now every morning I get the report I wanted.

My 5 Must-have vim Plugins

vim charity in Uganda Photo of vim charity: Kibaale Childrens Centre

  1. camelcasemotion

    This thing is amazing. With it, all the word-level editing you can do in vim you can do with CamelCase words or variables_with_dashes_in_them. For example, if the cursor was on the ‘h’ in ‘dashes’ in the example, doing c,w will delete ‘hes_’ and go into insert mode. It’s fantastically useful for editing code.

  2. rails

    I’ve written about this one before. Very useful for navigating rails projects. :Rmodel post will take you to app/models/post.rb from anywhere. :Rmigration will take you to the most recent migration file, :Rmigration 0 will take you to the schema. There are a ton of others…

  3. a.vim

    Another file navigation plugin, but this is for “alternate files”. If you’re editing a C file, it allows you to switch to the corresponding header file with a simple :A command, and vice-versa. Works for many languages, including objective-c.

  4. bufexplorer

    I needed this coming from emacs. It provides the equivalent of emacs’ buffer list (C-x C-b). The default keystroke to activate the list is \be. Once the list is up, you can change the sort, select a file to edit, etc.

  5. TabIndent

    Another one that mirrors default emacs behavior. If you’re in insert mode and press tab, it will indent the line under the cursor. Sounds simple and perhaps useless, but my brain needs this plugin.

Jekyll Blog Engine

Trying to resurrect my blog…it’s now being powered by Jekyll. Pretty easy to get all my old posts incorporated, but they sure are old! Maybe they’ll be useful to someone else…

Write Facebook Apps Using Google Appengine

facebook friends Photo by dantaylor

I hacked up minifb in order to get it to work with Google AppEngine. The result is gminifb. It uses Google’s urlfetch library instead of urllib2.

To get it to work, download gminifb and place it in your application’s directory. It requires simplejson, so put that in your app’s directory as well.

Here’s a sample webapp.py action that uses the library:

class FacebookAction(FbHandler):
  def get(self):
    arguments = gminifb.validate(FB_SECRET_KEY, self.request)
    session_key = arguments["session_key"]
    uid = arguments["user"]

    usersInfo = gminifb.call("facebook.users.getInfo",
        FB_API_KEY, FB_SECRET_KEY, session_key=session_key,
        call_id=True, fields="name,pic_square",
        uids=uid) # uids can be comma separated list
    name = usersInfo[0]["name"]
    photo = usersInfo[0]["pic_square"]

    self.render('facebook.html', {'name': name, 'photo': photo})

Thanks to Peter Shinners for writing the original minifb library. My changes were very minimal…

Firefox Search Keyboard Shortcut Saves Years Of Your Life

Cmd-k (or ctrl-k) takes you to the search box, but opens the results in the current tab. Go to about:config and change browser.search.openintab to true and now your search results go in a new tab.

Firefox 3 del.icio.us + firebug

Firefox 3 is now usable because there are versions of the Firebug and del.icio.us extensions that work:

Firebug: http://www.getfirebug.com/releases/firebug/1.2/

del.icio.us: http://tech.groups.yahoo.com/group/delicious-firefox-extension (Join the group, go to the Files section)

Vim Indent Block Of Code

Summary: Vjjjjjj=

Enter visual line mode: V

Select a block of code: jjjjjjj (or whatever)

Indent it: =

Heaven…

rails.vim tip :RSunittest

Using rails.vim, there are a bunch of ways to navigate around your project using :R commands: :Rview, :Rcontroller, :Rmodel, :Runittest, etc.

You can also do :RS with any of these and they will open up in a split window. So if you are in a model, you can do :RSunittest and vim will open your unit test in a split window.

Open A New Terminal Window Using Quicksilver

Create the following AppleScript file:

tell application "Terminal"
  do script ""
end tell

I named mine newterm.scpt and put it in ~/bin, where I put all my little shell scripts.

Once Quicksilver reindexes its catalog (you can force this), you can activate this script anywhere. I’ve trained Quicksilver to use Ctrl-space nt.

Automatic Production Log Performance Reports Rawk Logrotate

rails Photo by alphaducentaure

Killing two birds with one stone here…rotating your Rails production.log (in the default scenario, it just grows forever) and monitoring the performance of your application.

First, get rawk.rb. This little script analyzes your production log and produces nice reports showing which requests are taking the most time. Put it somewhere like /usr/local/bin.

Next, you want to create a logrotate config file for your app’s production.log. On a debian system, they live in /etc/logrotate.d. Here’s an example in a file named rails-example:

/var/www/example.com/shared/log/production.log {
	daily
	missingok
	rotate 14
	compress
	delaycompress
	notifempty
	copytruncate
    prerotate
      /usr/local/bin/rawk.rb < /var/www/example.com/shared/log/production.log | mail -s "[example.com] rawk report" somebody@example.com
    endscript
}

This tells logrotate to rotate the production log every day, keep 14 days worth of copies, and before it rotates the log file, it runs rawk on it and mails the result to somebody.

Postgresql Primary Key Sequence Out Of Sync

old computer room Photo by cote

I’m not sure how this happened, but I kept getting

PGError: ERROR:  duplicate key violates unique constraint...

messages while doing a simple INSERT using ActiveRecord with a PostgreSQL database, and the key was the primary key for the table. My guess was that the primary key sequence was out of sync.

SELECT MAX(id) FROM posts;

returned 4, but

SELECT nextval('posts_id_seq');

returned 4 as well.

I found two ways to fix it. The first odd way is to run the INSERT statement manually in psql. I don’t know why it works in psql and not through ActiveRecord, but I don’t know why there was a problem in the first place.

The second is to update the sequence manually:

SELECT setval('posts_id_seq', (SELECT MAX(id) FROM posts)+1);

I never tried the second method as the problem hasn’t returned.

Feedtools Database Cache

The ruby gem FeedTools has a built-in caching mechanism. Unfortunately, it doesn’t quite work out of the box, and the documentation doesn’t explain it very well. Despite what it says, you have to tell it to explicitly use the cache in your code:

#!/usr/bin/env ruby

require 'rubygems'
require 'feed_tools'

FeedTools.configurations[:feed_cache] = "DatabaseFeedCache"

puts "Getting feed 1st time..."
f = FeedTools::Feed.open('http://www.slashdot.org/index.rss')
puts "1. live? #{f.live?}"
puts "Getting feed 2nd time..."
f = FeedTools::Feed.open('http://www.slashdot.org/index.rss')
puts "2. live? #{f.live?}"

Assuming FeedTools can find a database.yml file and the database has the cached_feeds table described in the FeedTools documentation, the above script will cache the feed after the first time and f.live? will return false the second time you run it.

I also added an index on the href column in cached_feeds. All the lookups are done on that column and the migration that comes with FeedTools doesn’t have any indices. Here’s my migration for cached_feeds:

class AddFeedToolsTables < ActiveRecord::Migration
  def self.up
    create_table :cached_feeds do |t|
      t.column :href, :string
      t.column :title, :string
      t.column :link, :string
      t.column :feed_data, :text
      t.column :feed_data_type, :string
      t.column :http_headers, :text
      t.column :last_retrieved, :datetime
      t.column :time_to_live, :integer
      t.column :serialized, :text
    end
    add_index :cached_feeds, :href
  end

  def self.down
    drop_table :cached_feeds
  end
end
Apache 2 Virtual Host Redirect

For one of my sites, I own .org, .net, and .com for the domain and I want all the requests to go to the .org domain. I switched to Apache 2 and this is the configuration I used:

<VirtualHost *:80>
  ServerName india.bigpatents.com
  ServerAlias india.bigpatents.com 

  RedirectMatch (.*) http://india.bigpatents.org$1

</VirtualHost>
Postgresql Slow Query Log

Set the following variable in postgresql.conf to log all queries that take longer than 100 milliseconds:

log_min_duration_statement = 100

They will show up in the main postgresql.log file.

pgFouine looks like an intriguing postgresql log file analyzer, except it’s written in PHP??!?

Ping Google Blog Search Using Ruby

radar dish Photo by leedsyorkshire

With the following class, you can ping Google’s blog search whenever you update your blog:

require 'xmlrpc/client'

class BlogPing
  def self.ping(site_name, site_url, page_url, feed_url)
    begin
      server = XMLRPC::Client.new2("http://blogsearch.google.com/ping/RPC2")
      server.call2('weblogUpdates.extendedPing', site_name, site_url, page_url, feed_url)
      return true
    rescue => detail
      puts "ping failed (#{detail})"
      return false
    end
  end
end

With any luck, it just worked on this post!

gettext 1.10.0 and rails 2.0

The ruby gem gettext 1.10.0 doesn’t work with html.erb files. It’s simple to fix. In the file .../gems/gettext-1.10.0/lib/gettext/parser/erb.rb change the following:

module GetText
  module ErbParser
    @config = {
      :extnames => ['.rhtml']
    }

to

module GetText
  module ErbParser
    @config = {
      :extnames => ['.rhtml', '.erb']
    }

With this change, the gettext system will parse all your erb files.

How To Fix Unsupervised Daemontools Log Scripts

I am using daemontools and was having trouble with the log files. The log script was not being supervised, nothing I did seemed to make it work. svstat /service/test/log returned unable to open supervise/ok: file does not exist.

The problem was that svscan had picked up on my service before the log directory existed. And no matter what I did (including restarting the service), it wasn’t noticing the log directory. I couldn’t get it to notice the filesystem change because I made real directories in /service for each service, not symlinks.

One solution would be to reboot, but rebooting a unix box is equivalent to admitting failure. I stopped my service, moved the service files to /var/local/services/test, then symlinked the service directory into /service.

Daemontools Scripts For Sphinx

Here are scripts to use daemontools to manage sphinx.

The run script (/service/sphinx/run):

#!/bin/sh
echo starting
exec 2>&1
exec /usr/local/bin/searchd --console --config /var/www/example.com/current/config/sphinx/production.conf

the --console flag is key: it keeps searchd running in the foreground.

The log/run script is standard (/service/sphinx/log/run):

#!/bin/sh
exec multilog t ./main

And finally a script to reindex everything:

#!/bin/sh

/usr/local/bin/svstat /service/sphinx | awk -F ')| ' '{print $4}' > /var/www/example.com/current/tmp/pids/searchd.pid
/usr/local/bin/indexer --config /var/www/example.com/current/config/sphinx/production.conf --all --rotate

The searchd.pid file is referenced in the sphinx production.conf file and is used by indexer. It would be nice to just call indexer --all and then svc -h /service/sphinx, but indexer requires the --rotate flag to reindex without taking searchd offline, so you have to do the awk junk to get the pid first.

git fatal empty ident not allowed

I use git like I used to use svn: one central repository, clients push/pull to it. I recently migrated the repository from a crappy Virtuozzo VPS (horrible, horrible disk access times…had to wait many seconds for ls to run) to a Xen one.

Everything seemed fine until today when I did my first git push, which returned the error:

fatal: empty ident <git@example.com> not allowed

It took me a while to figure out as .git/config had my user information set, and I even had a ~/.gitconfig file with default user information set. The git faq has an entry for this problem, but doesn’t mention server set-up.

The problem was the git user on my new server didn’t have any name information in /etc/passwd. It didn’t have this on the old server either, but for whatever reason it needed it on the new one (the old server is too slow to bother investigating). I changed /etc/passwd on the server, giving the git user the full name of ‘Git Git’ and everything worked fine. It’s weird that it cares…in the log, this user never shows up. Must be something to do with merging? Anyway, it works…

Rails Fragment Caching With Multiple Accounts

In Episode 90: Fragment Caching railscast, there is a great, simple explanation of how to do fragment caching.

I like using strings as the identifiers for the cached portion of a view, but if your site has accounts with users signed in and you want to cache fragments that change based on the user, you need to do something more. If the user id is exposed in the url, you can use the url to generate the cache identifier. But if the user id is hidden, then you need to do something more.

First, it should be noted that if you put slashes in the cache identifier, it will treat those as directories. So

<% cache 'some/deep/fragment' do %>
Hello
<% end %>

will put the fragment in RAILS_ROOT/tmp/cache/some/deep/fragment.cache. This is good because if you have a lot of users, you won’t want all their cache files in the same directory as that would slow down the filesystem dramatically.

To be clear, if you did

<% cache 'recent_contacts' do %>
<%= render :partial => "/shared/contact", :collection => @contacts %>
<% end %>

this would not work for multiple accounts as once it is cached, every account will see the same contact list.

In my project, there are many accounts. Each account has many users, but the fragments can be cached for all the users in an account, so putting the account id in the cache identifier is enough to make them unique.

So you could do

<% cache "recent_contacts_#{@account.id}" do %>
<%= render :partial => "/shared/contact", :collection => @contacts %>
<% end %>

This is fine, but once you get a lot of accounts, you are going to have a lot of files in the cache directory, one for every account. Besides slowing down the filesystem, this could even stop working if you go above the number of allowed files in a directory.

My approach was to add a cache_dir method to the Account model that splits the accounts into many subdirectories. There are many ways to do this that balance the directories better (hash the id, then split it up into directories to randomize where it goes), but this way is simple and makes for an easy explanation:

class Account < ActiveRecord::Base

  # ...
  
  def cache_dir
    unless @cache_dir
      x = sprintf("%08d", self.id)
      @cache_dir = x[0..1] + "/" + x[2..3] + "/" + x[4..5] + "/" + x[6..7]
    end
    @cache_dir
  end

end

Then the cache statements in the views look like:

<% cache "#{@account.cache_dir}/contacts/recent" do %>
<%= render :partial => "/shared/contact", :collection => @contacts %>
<% end %>

So the recent contacts for account id 1 would go into tmp/cache/00/00/00/01/contacts/recent. It keeps it fairly clean and keeps all the cache files separated by account ids. Once I hit 10,000,000 accounts, I’ll need to adjust this, but that’s a good problem to have.

Rails 2 Custom Source Annotations

Rails 2.0 has the new rake notes:... tasks built in. If you annotate your code with OPTIMIZE, FIXME, or TODO comments, then rake notes will show all of them, rake notes:fixme will show just the FIXME comments.

What if you want annotations of other comments? I’ve been using XXX for a few years now, so I wrote the following in lib/tasks/notes.rake:

require 'source_annotation_extractor'

namespace :notes do
  desc "Enumerate all XXX annotations"
  task :xxx do
    SourceAnnotationExtractor.enumerate "XXX"
  end
end

Now I can run rake notes:xxx and see all my XXX comments in the project. They don’t show up in the global rake notes task, but…

Getting Rails 2 Release Candidate To Work

To install:

sudo gem install rails --source http://gems.rubyonrails.org

Then change environment.rb like so:

RAILS_GEM_VERSION = '>= 1.99' unless defined? RAILS_GEM_VERSION

if you get an error when running script/server that is something like:

Missing the Rails 1.99.0 gem. Please `gem install -v=1.99.0 rails` 

then update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.

You should follow me on Twitter: @patrickxb