Imagine a bacon-wrapped Ferrari. Still not better than our free technical reports.
See all our reports

Using Jenkins to Build Pipelines, Chain and Visualize Jobs

Job chaining in Jenkins is the process of automatically starting other job(s) after the execution of a job. This approach lets you build multi-step build pipelines or trigger the rebuild of a project if one of its dependencies is updated. In this article, we will look at a couple of plugins for Jenkins job chaining and see how to use them to build and visualize these Jenkins pipelines. If you’re interested in the best Jenkins plugins, we have an excellent post about that!

Out of the box solution – Build Other Projects

Jenkins has a built-in feature to build other projects. It is in the Post-build Actions section. You can specify the projects that you want to build after this project is built (you can trigger more than one). So whenever project A is built you will trigger the building of project B. You can also specify the conditions when the other jobs are built. Most often you are interested in continuing with the build pipeline only if the job is successful but your mileage might vary.

Jenkins Build Pipeline: Build other projects solution

One thing to remember here is that this feature has two configuration location. You can configure project A and specify a post action as in the previous screenshot. Another option is to configure this from project B and say “build this project B only after project A is built”. You don’t have to fill out both, just change one and the other is updated. See the next screenshot for the second option.

Jenkins Build Pipeline: Build after others solution

Jenkins Build Pipeline Plugin

Jenkins Build Pipeline Plugin is one interesting plugin. The main features of this plugin is to provide visualization of the build pipeline and also to provide manual trigger for continuous delivery purposes. The configuration is a separate Post Build action where you can configure which projects should be built after project A. By default the triggering is actually done manually by the end user! If you want certain steps of the Jenkins pipeline to be automatic then you have to use the built-in job chaining (see the Out of the Box Solution for more details).

Jenkins Build Pipeline Configuration example

The Jenkins Build Pipeline plugin offers a very good visualization of the build pipeline. By configuring a new Jenkins view and choosing which job is the first job in the Jenkins pipeline you can get a visualization of the whole build pipeline. In the screenshot, be sure to note that one of those steps is manual and the result are automatic. The manual one can be triggered from the very same view.

Jenkins Build Pipeline example

Jenkins Parameterized Trigger Plugin

The Parameterized Trigger Plugin is another triggering plugin but with a twist: this plugin lets you configure more aspects of the triggering logic. It covers the basic Out of the Box Solution features and adds many more. The most important one is the option to trigger the next build with parameters. For example, by defining SOURCE_BUILD_NUMBER=${BUILD_NUMBER} you are able to use the variable $SOURCE_BUILD_NUMBER in project B. This way you can, for example, use the artifact built in the previous job to be fetched from your central artifact repository using the ${BUILD_NUMBER}.

Jenkins Trigger Parameterized Build



Downstream Buildview Plugin

The Downstream Buildview Plugin plugin that does not do job chaining itself, but provides a means to visualize the Jenkins build pipeline. It is similar to the Build Pipeline View but more dynamic. You can click on any item in the build history and have its build pipeline visualized.

Jenkins Downstream Build Pipeline Visualization

Conclusion

The main feature that makes Jenkins so good is that there is always an app plugin for what you need. Of course, the same fact also highlights its biggest weakness. It is rather difficult to choose the correct plugin and very often you need a couple of plugins to achieve your goal. The same is true for job chaining and visualization.

The job chaining features that we covered in this post all provide the minimum functionality – triggering other jobs. The Parameterized Trigger plugin is the most feature-rich, but lacks the manual triggering. The Jenkins Build Pipeline plugin only offers manual triggering and you need to figure out automatic triggering yourself (using the built-in feature for example).

From the visualization side, the Jenkins Build Pipeline plugin is definitely the best looking. At the same time, the plugin does not support passing parameters (the latest alpha build is a bit better) and once the build pipeline gets long it gets a bit ugly. We do like the part of defining a separate view and then always being on top of your build pipeline. The Downstream Build View plugin gives you great flexibility and insight to job chaining, but does not enforce any kind of process.

So, there are the Jenkins plugins that we use at ZeroTurnaround for job chaining and visualization. Do you use the same tools? If not, can you recommend any others? Which are your favorites? Please leave comments below!

  • Billy Foss

    We use the pipeline view to show the full cycle from branching to build to deploying on test systems.  It gives a nice summary view of a full branch/build process.  I will be trying the Downstream Buildview Plugin to get a more dynamic view of other jobs that chain.  
    Some may want to consider using the Promote step to allow review of build results that can be approved and then trigger the following steps in a chain.  For example, the Promote can allow certain users to approve a build, which can trigger copying that build to a QA server for further testing.  

  • I did not know about the promote plugin but will check it out. Sounds like a great addition.

  • Grégory Boissinot

    A build chain in Jenkins can also be achieve with the XTrigger Jenkins plugin (https://wiki.jenkins-ci.org/display/JENKINS/XTrigger+Plugin)

  • The pipeline plugin looks really interesting. We are going to checkout that.  Thanks for the article Toomas,

  • The comments mangled your URL a little (404 because of the extra bracket). Here is a shortened link http://bit.ly/JJlhqO I guess now I know 4 ways for chaining :)

  • irfan

    i have around 10 jobs in my view and i want to run the remaining 9 jobs after the build of first job. Instead of mentioning the names of the remaining 9 jobs as comma separated is there any other way to give lije some * function or some thing.

  • I think not.

  • irfan

    Thanks for the reply Toomas. There is one more thing i would like to know, the email notification thing only works under the negative scenarios and a successful build after a fail build so is there any way to get the e-mail notification for success scenarios..

  • I think you cannot do it with vanilla installation but check out the Email-ext plugin, https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

  • Hi. How can I use Build Pipeline Plugin with parameters? I’d like to use the same BUILD_NUMBER on job trigged by Build Pipeline Plugin

  • Satish

    Hi team,
    while building the application it is taking more time, so I want to stop the running build instance, how to do that. Can you please help me on this.

  • Gorovdude

    Another nice plugin is the MultiJob Plugin https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin

  • jack

    Which one is the best for you ?

  • Greg

    The pipeline plugin was amazing, until someone wanted to save a little money. Imagine:

    build-pipeline =

    execute step 1
    execute step 2, parameters x, y, z
    execute step 3a
    execute step 4
    execute step 5

    Now someone wants:

    execute step 1
    execute step 2, parameters x, y’, z”
    execute step 3b
    execute step 4

    Good luck with that; I learned the hard way this weekend. If you’re going to chain together jobs, SKIP the point-and-click and install the workflow: aggregator plugin, instead, then learn enough groovy to do what you need to do. I just built pre-defined jobs, and ripped out the post-build actions / conditional steps on all of my jobs. If you examine each job’s xml file, I still have a lot of code duplication, but I’ll rip that out eventually, replacing it with the job dsl plugin (NOT the workflow dsl).

  • Hello there, there is another interesting plugin https://github.com/jenkinsci/coordinator-plugin

    Which is no code required to define jobs running sequence.Much easier to get the whole status of one build

    And I found a walk through demo here
    http://www.tothenew.com/blog/jenkins-coordinator-plugin/

    Hope you will enjoy it

  • Good article. Thank you!

  • Azim

    Hi,

    Is it possible to run a post node step just like a post build step. Example:

    node (“${label}”) {
    // build steps

    // post build steps. Should run irrespective of failure in build steps. Example if I want to update GitHub commit status here.
    }

    Thanks

  • Ganesh

    There is a completely another CI system which has been supporting these pipeline concept for quite some time. But, not much community support and cohesion is there for this but it is also nice CI tool. https://www.go.cd/