This page presents a bullet point comparison of the capabilities of various continuous integration systems. The products compared (in alphabetical order) are:

CAVEAT This comparison is far from complete. If you find errors in it please mail the lists. Similarly if you want extra comparison points listed, let us know - we're happy to add points, even if Test-AutoBuild doesn't get a tick in the box! Finally, this comparison was written by the primary developer of Test-AutoBuild and while the aim was to be objective, there is undoubtably some bias present - if only in the list comparison points chosen. Corrections, clarifications, suggestions are all welcome.

Build engine

Engine AntHill AntHill Pro Continuum CruiseControl Luntbuild Test-AutoBuild
Runtime Java 1.4 Java 1.4 Java 1.4 Java 1.3 Java 1.4 Perl 5.x
Run mode Daemon Daemon Daemon Daemon Daemon Single shot
Schedular Built-in Built-in Quartz Built-in Built-in Cron
Run on demand Yes Yes Yes Yes Yes Yes
Scheduled Yes Yes Yes Yes Yes Yes
Continuous Yes Yes Yes Yes Yes Yes
License GPL Commercial Apache v2 BSD BSD GPL
Configuration Web based Web based XML / Web based XML Web Structured text

Source Control Integration

This feature is assessed on three points, representing increasing level of functionality.

  1. Supports checkout of sources
  2. Detects whether changes have been made since last run
  3. Extracts changelogs since last run

The goal of Test-AutoBuild is to have complete coverage for all open source version control systems. Proprietry SCMs will only be supported if it is possible to download a copy of the tools under a license which allows license free use in a unit test scenario.

Repository AntHill AntHill Pro Continuum CruiseControl Luntbuild Test-AutoBuild
0 3 0 3 2 0
0 0 3 0 0 3
CM Synergy 0 0 0 2 0 0
3 3 3 3 2 2
0 0 0 0 0 3
File System 0 0 1 1 1 1
GIT 0 0 0 0 0 3
Mercurial 0 0 0 0 0 3
MKS 3 3 0 3 0 0
0 0 0 0 0 0
Perforce 3 3 3 3 0 3
PVCS 3 3 0 3 0 0
Star Team 3 3 3 3 2 0
0 0 0 0 0 2
3 3 3 3 2 3
GNU Arch 0 0 0 0 0 3
3 3 0 3 2 0

Module builds

Feature AntHill AntHill Pro Continuum CruiseControl Luntbuild Test-AutoBuild
Virtual install root Yes Yes No No No Yes
Module dependancies Yes Yes Yes No No Yes
Automatic build ordering Yes Yes Yes No No Yes
Build caching Yes Yes Yes No Yes Yes
Dependancy skip Yes Yes No No No Yes
Log capture Yes Yes Yes Yes Yes Yes
Test capture Yes Yes Yes Yes Yes Yes
Package detection No No No No No Yes
Artifact detection Yes Yes No Yes No Yes

Post-build actions

Point AntHill AntHill Pro Continuum CruiseControl Luntbuild Test-AutoBuild
Generate status pages Yes Yes Yes Yes Yes Yes
Customize status No No No No No Yes
Publish logs Yes Yes Yes Yes Yes Yes
External links No No No Yes Yes Yes
Publish artifacts Yes Yes Yes Yes No Yes
Changelogs in status pages Yes Yes Yes Yes No Yes
Publish packages No No No No No Yes
YUM/URPMI repository index No No No No No Yes
APT repository index No No No No No Yes
Generate CD ISO No No No No No Yes
Hard linking No No No No No Yes

Post-build notifications

Point AntHill AntHill Pro Continuum CruiseControl Luntbuild Test-AutoBuild
E-Mail alerts Yes Yes Yes Yes Yes Yes
RSS feed No No No No No Yes
IRC alerts No No Yes No No No
MSN alerts No No Yes No No No
Jabber alerts No No Yes No No No

Extension points

Extension AntHill AntHill Pro Continuum CruiseControl Luntbuild Test-AutoBuild
Workflow No No No No No Yes
Source control Yes Yes Yes Yes Yes Yes
Archive No No No No No Yes
Builder No No No Yes No No
Counter No No No No No Yes
Monitor No No No No No Yes

Topics