Pain-free Continuous Integration for Python Using tox+Hudson

At ActiveState, we use Hudson for continuous integration of our various Python projects such as PyPM.
Why do we use Hudson and not Buildbot? The short answer is that Hudson is simply much easier to configure and run than buildbot.
Most of our Python projects – internal and external – are meant to work on at least three platforms: Linux, Windows and MacOSX. In addition, they must also work on a variety of Python versions, say: 2.5, 2.6, 2.7, 3.1. Each of these projects have a reasonable coverage of tests; yet every commit to the trunk leaves the risk of breaking a specific functionality on a particular platform and particular Python version, that the developer who introduced the commit may not be using. This is where continuous integration, and Hudson in particular, comes to help.
We first install Hudson in a master machine and then configure the various “slave” machines each corresponding to a platform. We normally use the following seven slaves/platforms for our Python projects:

1. win32-x86
2. win64-x64
3. macosx-10.4
4. macosx-10.5
5. macosx-10.6
6. linux-x86
7. linux-x86_64

Configuring the slave is magically easier with Hudson. On Windows machine, thanks to Java Web Start, assuming Java is already installed, we only have to invoke a particular URL from browser. On other machines, this is a matter of running a particular Java command line.
And that veers our attention to the Hudson web interface for the rest of the configuration. Titus’ PyCon talk referred to above walks through the necessary steps for setting up a project for continuous integration in Hudson, so I won’t go into the details. Suffice it to say that you will have to, at minimum, provide the a) source control URL for your project, and b) the command-line (or Python script) to run your project’s tests.
Where does tox come into this picture? tox makes it much easier to test across the various Python versions. If you have been watching Titus’ talk, it must have been clear that setting up testing across multiple platforms is as simple as checking across each of the Hudson slave you want to test. With tox, you just need to write a file named “tox.ini“ in your project root directory. The platinfo project has one:

$ cat tox.in
[tox]
envlist = py24, py25, py26, py27, py31
[testenv]
commands = python test/test.py
$

If you now run “tox” in the command line, tox will automagically:

1. Find the various Python versions installed on your system, without you telling it explicitly
2. Create virtualenv for each of the configured Python versions
3. Create a sdist of your project (setup.py) and then install it on the virtualenv along with the dependencies.
4. Run the specified test command using that virtualenv.

tox integrates with Hudson by way of a command line – consult the tox documentation for more details.
For the fine details, I suggest you to watch this excellent PyCon 2010 talk by Titus Brown.
See also:

1. tox home page
2. Michael Foord’s blog post on how he uses tox with the mock project
3. tox bootstrap script, so you do not have to install tox on each of the slaves
4. Continuous Integration (Wikipedia)