Imagine a bacon-wrapped Ferrari. Still not better than our free technical reports.
See all our reports

Code Quality Tools Review for 2013: Sonar, Findbugs, PMD and Checkstyle

Catching up with Code Quality Tools from 2012

When we released our Developer Productivity Report last year, it was the first time we asked our respondents about Code Quality Tools.

Code quality tools fulfill a growing need, as our code bases become larger and more complex, and it’s important to try to automate your code checks as much as possible. They are pretty versatile and customizable, and typically they are integrated into your build process, but can also be run manually in a one-off fashion.

What’s interesting is that the entire code quality tools ecosystem is kind of meant to be complementary, and not really competing. This is indicated by the fact that there really isn’t a clear leader in the code quality tools space, although Sonar, which combines all three other tools into a single package, seems to be the most well suited for advanced users & enterprise teams.

What IS good to see is that a significant portion of our respondents use some kind of code quality tool, we believe this to be because of how well Java lends itself to static analysis. Thus, many of the advancements in code quality tools have been tied to integration with other technologies and improved rule sets that increase the accuracy with which future bugs can be identified.

What if you no longer had to redeploy your Java code to see changes? The choice is yours. In just a few clicks you can Say Goodbye to Java Redeploys forever.


License: Open Source
Current version: 5.0.2 (released 03.02.2013)
From the horse’s mouth: “PMD is a source code analyzer. It finds unused variables, empty catch blocks, unnecessary object creation, and so forth.”

PMD is used for detecting bad practices in code, which is intended decrease the number of bugs in your code. The theory is that conforming to good practices in coding leads to better code, which we definitely agree with. With tagline that states “Don’t shoot the messenger”, it’s clear that PMD might be responsible for alerting sloppy devs of bad practices more often that they’d like…


License: Open Source
Current version: 2.0.2 (released 10.12.2012)
From the horse’s mouth: “This is the web page for FindBugs, a program which uses static analysis to look for bugs in Java code. FindBugs has been downloaded more than a million times.”

FindBugs was the most-used tool in the Code Quality space according to our survey results. Created by the University of Maryland, it actually scans your code for bugs, breaking down the list of bugs in your code into a ranked list on a 20-point scale. The lower the number the scarier the bug. While FindBugs also does some checking of best practices, PMD is better suited and is a common combination.


License: Open Source
Current version: 5.6 (released 18.09.2012)
From the horse’s mouth: “Checkstyle is a development tool to help programmers write Java code that adheres to a coding standard. It automates the process of checking Java code to spare humans of this boring (but important) task. This makes it ideal for projects that want to enforce a coding standard.”

Checkstyle is incredibly useful in a team environment: whenever you introduce individual developers’ IDE preferences for viewing and editing, you without doubt have gotten into the old “tabs vs. spaces” flamewar. Checkstyle helps you maintain your code easier because it’s more readable as a result of adhering to code standards that Checkstyle introduces, such as complaining about poor formatting in your source code.


License: Open Source
Current version: 3.4.1 (released 08.01.2013)
From the horse’s mouth: “Put your technical debt under control. Productivity is falling? Confess your source code to clean it up!”

Sonar is the most polished tool in the code quality space, and literally includes PMD, FindBugs and Checkstyle all in a single package, plus some excellent support for tools in the Java ecosystem. You can find plugins readily available for all sorts of useful tools: Checkstyle, Clover, Cobertura, Emma, FindBugs, JaCoCo, PMD.

Sonar is used as more of an orchestration engine, letting you call on all of the previous tools and aggregating their results. It is very convenient to use, especially if you’re already using Maven, and certainly helps you “get your technical debt” under control. In a large, enterprise environment, Sonar and some combination of the previous tools is essential.

Conclusion: Complementary, not competitive, ecosystem

Putting it simply, code quality tools are pretty cool and can lead to reducing stress during development and, in the end, less manic debugging later.

The level of confidence you have in your code is never higher than when all of your tests are passing and Sonar, for example, gives you a thumbs up. Unlike most of the items covered in our productivity report, the code quality tools section is not competitive, it’s very complimentary. Which makes sense, why not test your code for everything you can think of. In the future, we hope to include in our report all the tools  Checkstyle, Clover, Cobertura, Emma, FindBugs, JaCoCo, PMD.

If you use any other tools, or think you’re too much of an awesome ninja code for quality analysis, let us know in the comments section or @Rebel_Labs on Twitter :)

Your time is too valuable to sit around waiting for your code to redeploy. Seriously, imagine how much more you could accomplish if you choose to Eliminate Redeploys from your daily routine!


  • balder

    Complementary and not competitive, that’s true, but use Sonar and you have all of them in one package.

  • pit

    A few points:

    * PMD and Checkstyle share a lot of rules. I wouldn’t call them complementary, even though the had different “starting points”.

    * Findbugs has the advantage that it operates on the binary level. It’s the only tool finding some bugs that are not obvious when looking at source code. I wouldn’t call these rules complementory to PMD / Checkstyle. The are rather “more in-depth” or “beyond” PMD (to the point where they can – in some negative cases – produce more false-positives). Findbugs however comes with the cost of longer times for analysis (and higher memory consumption).

    * Sonar has the highest initial cost since it needs a centralized infrastructure. And it is the most difficult to bring into a given build environment (e. g. if you don’t use maven, Sonar is still a pain to integrated). Since it is build partly on other tools, it is also the slowest in moving forward (e. g. Sonar does not support PMD 5 yet). But it might be an investment well worth. Furthermore Sonar brings history, overviews over different projects and comparison views to the table.