Worked when compiled, runtime’s problem now
In today’s Java build automation landscape, dependency management has become a part of what constitutes as a major role for a build automation tool, but this wasn’t always so.
Back in the good old days, your project’s dependency management was basically dumping a jar file in a lib-folder and then adding it to your version control system (VCS). Or, for the more adventurous, you could create a script that would download the library in the desired version from an external source, and pray to whichever deity would listen that the external URL didn’t change.
Doing all of this manually could be a daunting and cumbersome process, especially considering transitive dependencies (dependencies of dependencies), which might only show up if certain branches of the library’s code is executed. This would then cause unexpected stuff to happen during runtime, even though this is a problem that could have been fixed at compile time had the transitive dependencies been known and resolved.
Luckily, in the current crop of build automation tools this scenario is no longer predominant.
With the continuously gaining popularity of modularity, the need for inter-project as well as external dependencies has grown, and as a result all the most commonly used build automation tools have stepped up to that challenge and have dependency management support; either out of the box, or through plugins. And to make things even simpler for the developer, they all use similar syntax for defining dependencies, as well as are all able to pull dependencies from the same public artifact repositories (e.g. Maven Central).
But as convenient as public artifact repositories are, they also introduce a new set of complications into the mix; conflicts between the transitive dependencies, and the added reliance on a remote repository. In this article, we go over all sorts of stuff regarding dependency management in preparation for our upcoming RebelLabs report on Java Build Tools, so stay tuned!
How to define dependencies
The basic syntax most commonly used for defining dependencies is adding a tuple of the group-id, artifact-id and requested version to the dependencies section of the build script. The build tool then tries to resolve these dependencies, by searching for them in its local and remote-defined repositories.
In the examples below, we will show you how to add a dependency for the open-source library Guava from Google, which is available from Maven Central. To start, you need to know its group-id (com.google.guava), its artifact-id (guava), and which version of the library you’re interested in (as of this writing, the latest version is “15.0”). With that information at hand, you can add the dependency to the dependencies section of your build script. Here is how to do that with Maven, Gradle and Ant+Ivy:
<dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>15.0</version> </dependency>
compile group: 'com.google.guava', name: 'guava', version: '15.0'
Ant+Ivy (ivy.xml, or in build.xml)
<dependency org="com.google.guava" name="guava" rev="15.0" /> <ivy:cachepath organisation="org.com.guava" module="guava" revision="15.0" pathid="lib.path.id" inline="true" />
Now you can see the previous parameters used in all three examples, with Maven being a bit more verbose about it. The property names vary a bit from build tool to build tool, but the general gist is the same.
How to use version ranges
What if you don’t want to depend on a specific version of a library, but know that any version in a specific range will do? Or if you know that any version except ‘X’ will work? This is where version ranges can come in handy.
Instead of just a single version number, you can also specify a range, using a notation very similar to how you would define an interval in mathematics: parenthesis ( ) work as exclusive markers and brackets [ ] as inclusive markers. Using a range allows the tool to pick a suitable version for the build. Continuing the example with Guava, if you know that any version between 12.0 and up to 15.0 (excluded) would work, you could define it as “[12.0,15.0)”. You can also leave it open-ended like “[12.0,)”, which adds a requirement for any version from 12.0 and up, similarly “(,12.0]” would be any version up to and including 12.0. Or if you want anything higher than 12.0, except for version 13.0, you could specify “[12.0,13.0),(13.0,)” as the version.
But why use ranges at all? It’s an easy and convenient way to get a newer version of the library without having to change your build script; however, it also sets you up for potential trouble, should the author of the library opt to change functionality or the API that you’re relying on! Another caveat about using ranges is that if the version numbering is inconsistent or doesn’t follow some standard, things might not go as expected. Using ranges on artifacts with qualifiers in the version string (like SNAPSHOT, ALPHA, BETA etc) also doesn’t always go as expected, as the range definition only supports numerical intervals and the build tool might pick a beta version because it has a higher number than the release version.
Besides ranges and specific versions, dependencies can also be resolved using dynamic versions by using the keywords ‘LATEST’ or ‘RELEASE’ instead of the version number. Using those will make the build tool inquire the artifact repository about which version is the latest (or latest release) and use that version. The same caveats apply here as with version ranges though–any changes to the API or functionality might break the world.
Transitive dependencies and dependency conflicts
Let’s go back in time to the “good old days” again. Here, you had full control and an overview over which libraries were used, since it was immediately visible which libraries were present in the lib-folder. But with declarative dependencies remotely available, and the transitive dependencies that are automatically included as well, this easy overview of which libraries are in use have become somewhat obscured. Luckily, most build tools have a plugin or an option to list the entire dependency tree:
mvn dependency:tree -Dverbose
gradle -q dependencies
<report conf="compile" />
Ivy’s report option allows you to generate the reports in various different formats, the default being an HTML and graphml report; so not as straight-forward as the console output you get from Maven and Gradle.
But what happens if there are conflicts in the dependencies? What if Library A and Library B both depend on Library C, but require different versions of it? This is where things start to get a bit tricky, and where build tools’ dependency management implementations diverge.
Assuming we have a project dependency structure that looks something like this:
- Module A
- Module B
Trying to build the above project with Maven will result in an error because the dependencies could not be resolved, since no version of Guava can satisfy both ranges. But if you use an equivalent Gradle build script and build it with Gradle, it will pick the highest version of Guava available in either of the ranges; which in this case means version ’15.0’. Changing the dependency of Module B, so its range is ‘[12.0,)’, Maven will now pick version ‘12.0.1’, which satisfies both ranges; Gradle still picks version ’15.0’.
Ivy and Gradle acts very similar in these scenarios, which isn’t that surprising considering Gradle originally used Ivy as their underlying dependency management implementation until they implemented their own dependency resolution engine.
The usage of ranges isn’t that widely-used though, and the more common use case is to just have the simple version number listed in the dependency. Even in this simple case, Maven, Gradle and Ivy again act vastly different when resolving dependencies!
Maven utilizes a “nearest definition” strategy, in which the closest version to the root is the version it’ll use throughout the entire build! In the case of the structure above, Module A and Module B both depend on Guava, and they are both found at the same depth; but since Module A is listed in the project before Module B, the dependency for Guava used there will be the version used for the entire build, even if the latter relies on a higher version!
Due to this, a common approach for having better control over which version of conflicting dependencies is used is to add a dependency to the wanted version in the parent pom file. Since this is the first pom to be parsed, the dependencies listed there will always be nearest, thus Maven should use that version and ignore every other version mentioned in the dependency graph.
As opposed to Maven, both Gradle and Ivy by default resolve dependency conflicts with a simple strategy: they just select the highest version :) If this strategy doesn’t suit your needs, you can select a different strategy: for instance, force the build to fail should a dependency conflict arise, or forcing a specific version of the dependency to be used, overriding any version otherwise defined as part of the dependency graph.
While the above was just a short introduction to some of the great and not so great things about dependency management, these are some of the things to keep in mind when dealing with it, and a good excuse to start reading up on the documentation for your build tool to see exactly what’s happening behind the screen.
- Maven: Introduction to the Dependency Mechanism is a good place to start: http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
Gradle: a good starting point is Chapter 50 of its user guide: http://www.gradle.org/docs/current/userguide/dependency_management.html
Ant + Ivy: A slightly obfuscated path to the documentation can be found here: http://ant.apache.org/ivy/history/latest-milestone/ivyfile/dependency.html
Even though keeping binary dependencies in your VCS is somewhat frowned upon today, the idea of having complete control over which libraries you include is still a good idea. No matter which build tools you prefer (or are required to use), it’s always a good idea to know how they handle your dependencies; an old version of a library might introduce bugs, strangeness or, in the worst case, security risks to your production system! Keeping a handle on this can potentially save you a lot of headaches down the road.
Stay tuned for more on Build Tools from RebelLabs, and please leave comments below or tweet @RebelLabs.