5 reasons it’s silly to use awesome provisioning tools like Chef, Puppet and Ansible to release applications
We love the concept of “infrastructure as code” and the tools that go with it. Chef, Puppet, Ansible, CFEngine and others have transformed the way we manage tens, hundreds or even thousands of nodes today. Provisioning tools have really been a great value-add to the world of software, and this widespread adoption of the paradigm illustrates the key strength of those tools: simplicity.
Unfortunately, when we have a tool close at hand that we love, we tend to overuse it for problems it wasn’t really designed to solve.
Releasing your applications using Provisioning tools (instead of Release Management tools) is like moisturizing your face with bacon drippings instead of fancy lotion; you might love bacon nectar, but it’s simply not appropriate for use in all situations.
As I wrote in the previous post, at ZeroTurnaround we use Chef and Ansible extensibly, but complemented by LiveRebel (yep, we eat our own dogfood to make sure our users continue to be impressed) when it comes to releasing applications.
So in this post I’ll take a look at some of the reasons why the simplicity of provisioning tools tends to be not enough, and why release management tools are the way to go if you’re serious about getting the job done well. For you lazy/efficient readers out there, it comes down to:
- Provisioning tools don’t rock at orchestrating releases
- Provisioning tools aren’t awesome at rolling back changes (failures)
- Provisioning tools aren’t great at tracking artifacts
- Provisioning tools aren’t meant for tracking changes
- Provisioning tools aren’t designed for developing applications
Reason #1: Provisioning tools don’t rock at orchestrating releases
Provisioning tools are focused on the end state. You describe the desired state and then the nodes are brought to that state individually. This is what we actually value the most in Ansible for example. It is extremely declarative, and that works fine for provisioning.
But when you release applications, you also need to pay attention to all that stuff happening in the middle. You need orchestration at every level of your stack to make it work:
- Dependencies between application components: Which versions and in which order have to be released?
- Dependencies between layers: Can you update front end, back end or database nodes at the same time?
- Dependencies on middleware: At which point do you set up the cache, database and queues?
- Dependencies between nodes: After the first node passes acceptance tests (if you do that, which you should!) can you then roll out to all the rest of your nodes? Do you need to mitigate downtime by rolling out to nodes gradually?
The point is that doctors don’t use a meat cleaver for brain surgery. In the same way, you definitely don’t want just describe your final state to some system and watch it operate your delicate production environment like a dorm kitchen.
You can definitely hack it and teach it to be sequential and efficient, and smart, but then it seems like you are building a release management system yourself. Maybe it’s better to just use one!
Reason #2: Provisioning tools aren’t awesome at rolling back changes (failures)
We engineers make promises all the time, but then we often break them: “I won’t sacrifice code quality for development speed. I won’t deploy on Friday evenings. I won’t…” We do these things anyway and find changes soon after to regret it, since deploying new code is hard and often produces unexpected side-effects.
You’re in a better shape if you deliver value–meaning, new features or bug fixes–in short cycles, which means releasing changes frequently. But every change you make comes with the risk of breaking the application in production. When that happens, you really want to be back to the previous good state as soon as possible.
Let’s say, in the best case response to a failure, that you could restore the previous Chef recipe or Ansible playbook and reapply it. This takes time and is prone to human errors, plus when you combine this with the orchestration requirements discussed in my previous point, things get even worse. What you really need is a tool that can detect failures at any point along the rollout and immediately roll back everything.
Release management tools are made to embrace the dynamic nature of application releases, so in case of a failure, everything can automatically be rolled back–during or after a release.
Reason #3: Provisioning tools aren’t great at tracking artifacts
Unless something terrible happens, we’re really not that interested in the version of MySQL we have in production, both 5.1.1 and 5.1.3 are fine. This is why Ansible and Chef don’t come with a nice dashboard showing all your artifacts and versions in different environments. However, we are very interested in which build of the application has been tested and is ready for a production release, or alternatively which version just caused a major issue in production and calls for an immediate response.
Applications, versions and environments are the key concepts in release management. As I’ve mentioned previously, you can probably pull and visualize that information yourself, but then you’re already building your own release management software.
Reason #4: Provisioning tools aren’t meant for tracking changes
Tracking changes is a key benefit of the “infrastructure as code” mantra, but as soon as your application grows from being trivial into something larger, it’s important to consider what kind of changes are we talking about. Chef, for example, will tell us which packages and versions are pushed out, but not the actual changes in the application, such as:
- database migrations
- new configuration options
- dependencies between application components
None of these are pleasant to identify when the service is down and you need a fix out ASAP. So unless you’re making changes to mature middleware or libraries, you’ll want a tool that tracks actual changes, not just the metadata.
Reason #5: Provisioning tools aren’t designed for developing applications
You know, I really want my new app version to launch smoothly – what about you?
That means “battle-testing” the application and its roll-out process long before the time when an error or failure starts to negatively affect things. Battle-testing these days is carried out commonly enough: by Operations, after deadlines, while trying to figure out why the application doesn’t even start up.
Instead, it makes way more sense to test the functionality of the changes, as well as releasing them, very early on – right when they are introduced in the development process in fact. This means that developers have to step up when it comes to releasing their own changes.
Provisioning tools really weren’t created with this sort of thing in mind. They’re designed for operations and infrastructure, not developers testing and verifying that application specific database changes, environment properties and configuration are rolled out successfully.
Dude, just use Provisioning tools for Provisioning and Release Management tools for Releasing
Provisioning tools are so popular because of the simplicity and ease of use that comes from the focus on the desired end state. But when releasing applications, we also have a lot to consider regarding specific changes and the way they are rolled out, not just the end state of a node.
Application releases are complex, so you should accept this complexity and get an appropriate toolset that helps you with integrating releases into your development process. This way, every push to production can be given a greater vote of confidence.
Release management tools acknowledge that the release must be centered on the application as a whole, and the subsequent experience by your users. These tools focus on your applications in a way that allows them to better orchestrate, test and track releases.
It’s tempting to try and solve every problem with a provisioning hammer on your own and hold on; however, as you will inevitably end up implementing a half-baked, error-prone in-house release management tool, you might as well go with a proper solution to the problem from the beginning. Don’t you think so?
Releasing apps is hard and error-prone. That’s why ZeroTurnaround made LiveRebel, which is free for your first 2 nodes. Check it out!