UPDATED July 28, 2014: Interested in downloading a full report on Jenkins? Since writing this incredibly popular post, we’ve published a full report called Jenkins CI – The Origins of Butlers, Build Masters and Bowties. You can download a beautiful PDF version and take it with you!
While planning my trip to San Francisco for JavaOne (catch me there for a beer), I saw that there is a Jenkins User Conference coming up October 2nd, 2011, just prior to JavaOne. As I was signing up for the event, I started thinking that we’ve been using Hudson/Jenkins for years now. Through the years, certain Jenkins features/plugins have really paid off and been life savers for getting higher-quality software out quicker. So I thought, why not share this with you? Here is a list of my own and the team’s top 10 must-have Jenkins features/plugins.
- Easy Installation
- Easy Upgrades
- View Filters
- Amazon EC2
- HTML Publisher
- Throttle Builds
- Green Balls
Jenkins Pipeline is an incredibly useful addition that came to life with the Jenkins 2.0 release. It allows you to script your build pipeline consisting of one or more build jobs into a single workflow. It makes the visualization of the pipeline much easier, the monitoring of which parts have been run and which jobs are still in the queue a non-issue. The best part is that it’s integrated with the SCM, so you can develop your pipeline definition script using proper tools and see the historical changes, audit, do the code reviews, etc. The build pipeline can become the essential part of your mental model about the CI.
And with the pipeline plugin it is not easier to achieve than before. The image above is taken from the CloudBees blog, where you can read more about the best practices of using the pipeline plugin!
Jenkins comes as a WAR file (there are also a wide variety of Linux packages available and a Windows installer) that you can drop into your favourite JEE container (Tomcat, Jetty, Glassfish etc.) or you can start the WAR file directly with
java -jar jenkins.war. That’s it! Now that it is up and running, you can start configuring jobs and installing plugins. It will use a folder in your
$HOME to persist the configuration and store the actual jobs. No database, no special installation, no special registration. Clean and easy. You can even easily chain the job executions and visualize the whole pipeline of multiple jobs.
Jenkins has very quick release cycles, and statistics shows that there is a release for every 6.59 days. The release process page will state that an RC freeze is schedules for Monday, and then late Friday the RC is released. Also, there is a Long-term Support version available that is put together every 3 months and is suitable for more stable installations.
The upgrade process itself is super easy. It does not matter if you are using Linux, Mac OS or Windows; there is a single, easy way for upgrades built into the product itself.
Jenkins provides different means to communicate with it. There is a web-based GUI that everyone is accustomed to, the command line interface and also a Rest API for Python, XML and JSON. You can feel that this is a tool from developers for developers. For example, to iterate through all jobs available in a Jenkins installation, all you need in Python is 3 lines of code.
import urllib obj = eval(urllib.urlopen("http://ci.jenkins-ci.org/api/python").read()); for job in obj['jobs']: print job['name']
To generate a graph relationship of all your jobs in your installation, you don’t need much more code (mind the configurability and usability sections).
Once you get a large number of jobs in your installation (we have about 200 defined), you will want to organize them so that the right people can get a quick grasp of the current status of the builds in which they are interested. This is where the little plus sign (+) comes to work. You are able to construct views for your branches, projects and persons that will list only a subset of your jobs. The subset can be defined either by a regular expression or explicit project selection.
One of the little-mentioned features, which I think is totally taken for granted in Jenkins, is its ability to scale out to a large number of nodes and distribute the workload between them. If your own infrastructure is not large enough, then this plugin lets you tap into the resources of the Amazon EC2 infrastructure. You can specify the AMIs to use, label them, provide connection credentials and test the connectivity.
We use EC2 to run our JEE container tests. This means that to test WebSphere 7.0.1 we launch the specific AMI and start up WebSphere with our agent (XRebel or JRebel) and go through the test suite. This is orchestrated by Jenkins using this plugin and then copying back the results.
This plugin allows you to
download the internets and take advantage of Maven’s features. Jenkins reads your project’s dependencies from your POM and, if those are also built on Jenkins, sets up triggers in such a way that a new build in one of those dependencies will automatically start a new build in this project. Jenkins understands all kinds of dependencies in POM, namely:
- parent POM
- <dependencies> section of your project
- <extensions> section of your project
- <reporting> section of your project
This process takes versions into account, so you can have multiple versions/branches in your project on the same Jenkins instance and it will correctly determine dependencies. Of course, there are over 20 plugins listed under the Maven section of the Jenkins Plugins Page.
This plugin is handy with built-in copy to master functionality. Imagine that you have some files generated through your builds that are not actual artifacts. Let’s say logs, debug info, obfuscation maps whatnot. This plugin lets you to copy them to a master and provide an easy HTML file that will have links to these special files of yours for really quick access.
If you are not using a dependency management system, your build jobs become isolated as they do not know anything about each other. This plugin helps to solve this problem by allowing you to copy needed files from one job to another. So it may be handy to copy a resulting artifact from one build job into another job that will run tests against it. It does sound a bit hackish, but it is way better than referencing your artifacts from one job folder to another.
I am pretty sure that you have some jobs running in Jenkins that should not interfere with each other. This plugin allows you to group jobs, and allow only some defined number of jobs run concurrently (in most cases, it is one) in the same environment. Take test jobs that use some container instances preinstalled on your master/slaves, for example: this plugin allows you to configure your jobs so that they will run on different slaves or wait for another job to finish before starting.
This plugin allows a job to be run after all the immediate downstream jobs have completed. In this way, the execution can branch out and perform many steps in parallel, and then run a final aggregation step just once after all the parallel work is finished. This plugin is useful for creating a diamond shape project dependency. This means there is a single parent job that starts several downstream jobs. Once those jobs are finished, a single aggregation job runs. Unfortunately, more complex interactions are not possible with this plugin.
Historically Jenkins has used the blue color to denote successful builds. So, all status messages throughout the GUI use blue as the color of success. I don’t know the history of this, but closest thing that comes to mind is the The Matrix, where the blue pill was designated as the safe choice. If you are like me and feel that green is the way to go (Green Hornet, Green Lantern, Gumby, traffic lights, etc) then you can change your preference to green with the Green Balls plugin.
This was our list, a list that is suitable to deliver Java-based software that needs to be tested on a large matrix of Java versions, vendors and JEE containers. The whole list is not universal but the 4 features mentioned in the beginning will always stand out. I’d really like to know which other plugins are awesome – what are your favorite plugins and why are they useful in your line of software development? Let me know in the comments section below.