Vanity is an Experience Driven-Development framework for Rails.

Requires Ruby 1.9 and Redis 1.0 or later.

Add Vanity to your Rails app:

class ApplicationController < ActionController::Base
  use_vanity :current_user
end

Define an A/B test. This test compares three pricing options:

experiment "Price options" do
  description "Mirror, mirror on the wall, who's the better price of them all?"
  alternatives 19, 25, 29
end

Present different options to the user:

<h2>Get started for only $<%= ab_test :pricing_options %> a month!</h2>

Measure conversion:

class SignupController < ApplicationController
  def signup
    @account = Account.new(params[:account])
    if @account.save
      ab_goal! :pricing_options # <- conversion
      redirect_to @acccount
    else
      render action: :offer
    end
  end
end

Check the report:

vanity

Each A/B experiment represents several (two or more) alternatives. Use the ab_test method to choose an alternative. Call ab_test without a block to return the value of the chosen alternative. Call ab_test with a block to yield with the value.

Here are some examples:

def index
  if ab_test(:new_page) # classic true/false test
    render action: "new_page"
  else
    render action: "index"
  end
end

def index
  # alternatives are names of templates 
  render template: ab_test(:new_page)
end

<%= if ab_test(:banner) %>100% less complexity!<% end %>

<%= ab_test(:greeting) %> <%= current_user.name %>

<% ab_test :features do |count| %>
  <%= count %> features to choose from!
<% end %>

To measure conversion, call ab_goal! with the experiment’s name. Typically, you would do that from a controller action, for example:

def create
  ab_goal! :new_page
  ...
end

To measure conversion, simply call ab_goal! with the experiment name. From the Vanity identity set by the filter we know which alternative was presented by ab_test, and can correlate conversions to alternative. It’s that simple!

For effective A/B tests, you want to:

• Randomly show different alternatives to different people

• Consistently show the same alternatives to the same person

• Know which alternative caused a conversion

• When running multiple tests at once, keep them independent

If you don’t use any other mechanism, Vanity will assign a random value to a persistent cookie and use it to track the same visitor on subsequent visits. Cookie tracking is enabled by use_vanity.

If you keep track of users, you would want to use the user’s identity instead. Using user identity is more reliable than a cookie tied to a single Web browser.

To do that, call use_vanity with the name of a method which returns an object with the desired id attribute. Alternatively, you can use a proc. These two examples are equivalent:

use_vanity :current_user
use_vanity { |controller| controller.current_user.id }

There are times when you would want to use a different identity to distinguish test alternatives. For example, your application may have groups and you may want to A/B test an option that will be available (or not) to all people in the same group.

You can tell Vanity to use a different identity on a particular controller using use_vanity. Alternatively, you can configure the experiment to extract the identity. The following example will apply to all controllers that have a project attribute (without affecting other experiments):

example "New feature" do
  description "New feature only available to some groups"
  identify { |controller| controller.project.id }
end

Vanity will work out of the box on a default configuration. Assuming you’re using Redis on localhost, post 6379, there’s nothing special to do.

If you run a different setup, use the playground object to configure Vanity. For example:

Vanity.playground.host = "redis.local"
Vanity.playground.password = "supersecret"

EDD was all Nathaniel Talbott’s idea, I had experience tests to finish for Apartly, there was coffee involved and out came the idea for Vanity.

First experiment, A/B tests, heavily influenced by Patrick McKenzie’s awesome A/Bingo (www.bingocardcreator.com/abingo)

Pain points courtesy of Google Analytics’s stylish graphs and too-many-clicks goal tracking process.

Vanity, copyright © 2009 Assaf Arkin, released under the “Use for good, not evil” license (www.json.org/license.html)