Thursday, July 28, 2011

Selenium WebDriver + Linux (Headless) + Ruby + Jenkins == Awesome

Continuous Integration is a big part of our development process in OpenShift.  In this post, I'll talk about our setup to allow us to run automated Selenium tests for our website on a headless Linux machine, integrated with Jenkins.

For our web testing, we use the brand new Selenium WebDriver, integrated into Jenkins with our tests in Ruby.  The trick is that we are are running Jenkins and the Selenium tests on headless (i.e. no graphical desktop) machines but we still need to launch a real browser like Firefox, do the tests and capture screenshots if anything goes wrong.  The following setup is working like a charm for us, so I figured I would share:

Step 1 - Install Prerequisites
These prereqs are for a Fedora or RHEL 6.x machine (this was tested on RHEL 6, 6.1 and Fedora 14).  For RHEL, make sure you are subscribed to the Optional channel if you are using RHN to get the ruby dependencies like rubygem.  For Fedora 14 & 15, I had some issues building the ffi rubygem, so make sure to install the latest ffi rubygem from Koji.

Fedora and RHEL Prereqs:


sudo yum -y install make rubygems ruby-devel xorg-x11-font* wget

Fedora 14 only:

32 bit:
sudo yum install --nogpgcheck -y http://kojipkgs.fedoraproject.org/packages/rubygem-ffi/1.0.9/2.fc14/i686/rubygem-ffi-1.0.9-2.fc14.i686.rpm

64 bit:
sudo yum install --nogpgcheck -y http://kojipkgs.fedoraproject.org/packages/rubygem-ffi/1.0.9/2.fc14/x86_64/rubygem-ffi-1.0.9-2.fc14.x86_64.rpm

Fedora 15 only:

32 bit:
sudo yum install --nogpgcheck -y http://kojipkgs.fedoraproject.org/packages/rubygem-ffi/1.0.9/2.fc15/i686/rubygem-ffi-1.0.9-2.fc15.i686.rpm


64 bit:
sudo yum install --nogpgcheck -y http://kojipkgs.fedoraproject.org/packages/rubygem-ffi/1.0.9/2.fc15/x86_64/rubygem-ffi-1.0.9-2.fc15.x86_64.rpm


Step 2 - Install Xvfb and Firefox
This essentially allows a graphical display to run on your headless node.  It's lightweight though, so you're not having to install Gnome or a full desktop environment.  This will also install firefox so you have a browser to launch.  I prefer this to the Xvnc based environments.

sudo yum -y install xorg-x11-server-Xvfb firefox


Step 3 - Install Selenium WebDriver
These rubygems package up the Ruby bindings to the WebDriver libraries and let's you write your Selenium tests in Ruby without running SeleniumRC.  SeleniumRC has treated us well for many years, but WebDriver is going to be that much better.

sudo gem install selenium-webdriver

Step 4 - Install Headless
This allows your tests to directly spawn a headless display automatically and tear it down at the end of the tests.  With this gem, you don't need any special Jenkins plugins to manage the display on each test run.

sudo gem install headless

Step 5 - Write Your First Test
Put this in a file called openshift_test:

#!/usr/bin/env ruby


require 'rubygems'
require 'headless'
require 'selenium-webdriver'


# Create a headless display
headless = Headless.new
headless.start


# Create a firefox Selenium driver
driver = Selenium::WebDriver.for :firefox


# Navigate to OpenShift
puts "Navigating to OpenShift"
driver.navigate.to 'http://openshift.redhat.com'


# Find an element on the page
puts "Finding an element on the page by id"
element = driver.find_element(:id, 'app_promos')
puts element


# Now, save a screenshot
driver.save_screenshot("openshift.png")


# Clean up the display
headless.destroy

Step 6 - Run Your Test
Assuming the file is saved in the current directory and named openshift_test, run:

chmod 755 openshift_test
./openshift_test

You should see some output and a png image of the screenshot in the current directory as well.

Step 7 - Install Jenkins
Now, let's install Jenkins and fire it up.  We'll install Jenkins from their yum repo:
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
sudo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
sudo yum -y install java-1.6.0-openjdk jenkins
sudo service jenkins start
sudo chkconfig jenkins on

Jenkins will now be running on your hostname, port 8080.  For example: http://localhost:8080.  If you are accessing remotely make sure 8080 is allowed on your firewall.

Step 8 - Setup Jenkins
Now, let's make Jenkins run this thing.  You'll just need to create a new Jenkins job that runs that file you just created and archives the screenshots.  Since lots of screenshots can each up space, we usually use the Advanced settings and only keep archived screenshots for 10 days or so.

Conclusion
Hope this helps some people out.  Here are some references to the projects I've referenced above and for more info on the Ruby WebDriver project:

Projects:
  1. Selenium WebDriver - http://code.google.com/p/selenium
  2. WebDriver RubyGem - http://rubygems.org/gems/selenium-webdriver
  3. Headless RubyGem - http://github.com/leonid-shevtsov/headless
  4. Jenkins - http://jenkins-ci.org

Reference Info:
  1. WebDriver Getting Started - http://code.google.com/p/selenium/wiki/GettingStarted
  2. WebDriver Ruby Bindings - http://code.google.com/p/selenium/wiki/RubyBindings