Posted by: rusteddev | March 20, 2011

Check successful Heroku deployment with Cucumber & Selenium

Cucumber is a great tool to describe and test your Rails app behavior. In the project I’m working on, we do use it for acceptance testing and views testing (we use Rspec for Unit and Controller tests). After 4 months, we have nearly 400 rspec tests & 220 cucumber scenarios. But still it is not enough…..

Indeed, last week we deployed to our Heroku hosted production-ready environment and noticed a few hours later that we forgot to execute migrations…. Luckily the app is not public yet, but it will be soon…..So now how to be sure the app is running in production environment ?

I can’t execute my existing cucumber steps on production because they require test data (users, groups, etc…) and I can’t delete my database for that…. But I like the way cucumber is self explanatory and could help me in testing a “live” app. Below is a solution to running cucumber tests on heroku-hosted production app :

First, you need new testing scenarios. These will be simpler and won’t cover a feature in detail. They purpose is to check that the feature is available and running in production. Keep your detailed and large test for earlier stages. I also suggest to use selenium webdriver to run these test (instead of pure capybara) as it is the closest to what the “user” actually see.

@deploy @selenium
Feature: Smoke Test - Successfull deployment on production
  In order to test the application is succesfully deployed
  As a site admin
  I want to check a few features of the web site
    Given the checked environment is determined
  Scenario: Acces web site
    When I go to the home page
    Then I should see "Welcome to my website"

Note the @deploy tag used to identify these “post depoloyment” or “smoke” tests.
The custom step Given the checked environment is determined holds the logic that configures cucumber to test the “real” web site. This is the code of the step :

Given /^the checked environment is determined/ do
  Capybara.current_driver = :selenium
  Capybara.app_host = CHECKED_APP.nil? ? "http://localhost:3000" : "http://#{CHECKED_APP}"

The CHECKED_APP variable is set up by the rake task that runs the feature (see below). It contains the name of the heroku app and thus the url of the web_site. This can be useful if you have others env on heroku that you want to smoke test.

Then you need a rake task to execute these steps. Here is the code of the deploy.rake task :

require 'rubygems'
require 'cucumber'
require 'cucumber/rake/task'

task :set_environment, :env do |t, args|
  CHECKED_ENV = args[:env]
  heroku_config = YAML.load_file(File.join(Rails.root, 'config', 'heroku.yml'))
  CHECKED_APP = heroku_config[CHECKED_ENV].nil? ? nil : heroku_config[CHECKED_ENV]['app']

task :deploy_and_check, :env, :needs => :set_environment do |t, args|
    sh "heroku maintenance:on --app #{CHECKED_APP}"
    #Do git push & migration here   
    sh "heroku maintenance:off --app #{CHECKED_APP}"
    puts "------------Error while deploying to #{CHECKED_ENV}"
end do |t|
  t.fork = false # You may get faster startup if you set this to false
  t.cucumber_opts = "--tags @deploy --format pretty"

There are 3 tasks :

  • set_environment: sets up the CHECK_APP variable reading the heroku.yml file. This file is coming from heroku_san gem. It matches an environment with an app_name here is an example
  • cuke_it: executes the cucumber features with @deploy tag
  • deploy_and_check: to deploy the app and execute the cuke steps. It could also include the logic to rollback migration and deployment if something happening.

Now, just run rake deploy_and_check[“production”] : it will deploy your app and execute the @deploy cucumber scenario on your “production” environment.



  1. Nice work, thanks for this! I modified your approach to more closely apply to my local environment using capybara-webkit:

    …if its useful.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s


%d bloggers like this: