Running Rails tests on a CI server

Posted by Michael Reinsch on 2013-02-24

Running automated tests on a continuos integration (CI) server for your Rails projects usually requires some tweaking. I had already forgotten about this, but at the last 東京Ruby会議10, Keith (hey man!) was complaining about his issues. So I wanted to share how we are running the tests for our private projects.

For any public projects we simply use the awesome Travis CI. Unfortunately setting up Travis CI to use it for internal projects isn't easy. So I went with bigtuna, mainly because it's also a Rails project and easy to setup.

For the build process on bigtuna one simply supplies a script. Some other CI servers probably require less scripting, but in case you need some ideas, here is the basic script I am using for bigtuna:

rvm 1.9.3 do bundle install --path=%project_dir%/bundle --deployment
rvm 1.9.3 do bundle exec rake test:ci

You might have noticed that I'm using a non-default rake task "test:ci". This is because the default "test" or "spec" tasks will require you to have a development database setup. Rails tries to be smart here and make sure you aren't running tests against an old version of the database schema (i.e. with outstanding migrations). In this scenario however we always want to load the latest schema and not bother with migrations at all. Thus this special task which we simply add to the project's Rakefile:

namespace :test do
  task :ci do
    Rake::Task["db:test:prepare"].clear_prerequisites
    Rake::Task["db:test:load"].invoke
    Rake::Task["spec"].invoke
  end
end

The above example assumes you are using rspec. In that case, you should also be using the fabulous Fuubar - an instafailing RSpec formatter - for development. So you'll have "--format Fuubar" in your .rspec - which is probably also checked into git. For the CI server however, the default rsepc output is better. So I'm using the following additional line in the build script to reset the .rspec before running tests:

echo '--format p' > %build_dir%/.rspec

If you're using the standard minitest framework on the other hand, you can use the following:

namespace :test do
  Rake::TestTask.new("ci_test") do |t|
    t.libs << "test"
    t.test_files = Dir.glob("test/**/*_test.rb")
    t.verbose = true
    t.warning = false
  end
  task :ci do
    Rake::Task["db:test:prepare"].clear_prerequisites
    Rake::Task["db:test:load"].invoke
    Rake::Task["test:ci_test"].invoke
  end
end

That snippet will also run all your tests in one go, and not first run "test:units", then "test:functionals", and so on, saving you the additional bootup time required for spinning up Rails each time.

Finally, you might want to use a different database on the CI server - ideally the same type as on production. For local development I prefer to use SQLite - more on that in a future post. Anyway, we can again extend the build script with a simple line to overwrite that for the CI server. I also have the following in the config/database.yml:

default_mysql: &default_mysql
  adapter: mysql2
  host: localhost
  encoding: utf8
  pool: 5
  timeout: 5000

production:
  <<: *default_mysql
  database: the_production_db_name
  username: secret
  password: very secret

This is already nice because it makes it easy to add a staging environment config. And for the CI setup, something like the following is then good enough:

echo "test:\\n  <<: *default_mysql\\n  database: ci_db_name\\n  username: root\\n  password:\\n" >> %build_dir%/config/database.yml

And that's it. The usual CI setup for a project now doesn't take more than a couple minutes, and I don't need any additional databases or worry about migrations.