Imagine a bacon-wrapped Ferrari. Still not better than our free technical reports.
See all our reports

You’re SO Predictable: Early Results of Developer Productivity Report 2013

Time to speak out against Development injustice!

As you know, we’re in the process building up the data stores for our next Developer Productivity Report, focusing on tools, technologies and management practices that affect developers working in teams. We’ve launched this this 1-page, 5-minute (verified by devs!) survey hosted here:

Our goal is to have as many voices heard in here as possible, so please share, share, SHARE! :-)

Donate 5 minutes and earn geek karma points!

Some early results: How Predictable are your releases?

If I need to assess a new developer in our team, my first question is “Do you miss your release dates?” There is a persistent myth (and another holy war) in the industry that software development cannot be predicted. We have found that it is not true. Reasonably-experienced teams working on a reasonably-established project (outside the first couple of months) can predict the timeline of delivery within a reasonably good margin (e.g. 80%).

One of the reasons for the myth is that people often think that it’s 100% or bust. But some variation tasks that are done vs planned is expectable, so it’s more important to understand the boundaries of that variation.

So how do you measure predictability? We’ve came up with three key questions:

  • How late are your releases (vs initial planned time)?
  • How much of the original plans get done?
  • How much do plans change/expand during development? (scope creep)

We have some preliminary results that we’d like to share with you based on the three questions above.

How late are your releases?

We asked “How late are your releases (vs initial planned time)?” and saw somewhat disturbing results: There is pretty much a split by thirds here, where nearly two-thirds (65%) of development teams release new versions late. Just about one-third (32%) of respondents thus far actually deliver on time.

Result so far: The majority of development teams are not able to release new versions on time.

How much of the original plans get done?

When we asked respondents “How much of the original plans get done?”, we saw that 1 of every 4 developers get everything planned into the release. However, nearly 3/4 don’t. About 40% of respondents are able to get 90 percent or so into their releases, 20% of respondents get three-quarters of their plans into a release, and the final 15%, scarily, get only half of their original plans inside a new version. That’s more than 1 in 7 teams dropping half of their changes!

Results so far: Most of the time, all original development plans do not make it into the release.

How much do plans change/expand during development?

We are curious about the infamous “scope creep”, i.e. when things change during the development process, so we asked respondents “How much do plans change/expand during development?”. We see a fairly standard bell curve here. One-quarter of respondents let just 10% of the scope change, whereas over 40% of developers experience a 25 percent slip. One in five teams (20%) lose control of half of their project, and 6% of respondents see the majority of their release get away from them. Only 4% witness no scope creep at all.

Results so far: nearly 70% of respondents experience some level of “scope creep”, lowering the probability of predictable releases

What can we expect to see from this?

Objections to these three questions might be that it doesn’t measure developer productivity, as the dates might be set by the business, the plans change because of bad project management and the original plans could be badly spec’d and need a substantial rewrite during the development process.

As you can probably guess, in isolation of these matters it’s impossible to measure the predictability and productivity of a development team. Even with the best engineering practices, a team can still be highly unproductive if the surrounding organization does not do good work.

From the answers in the survey we can expect to learn the following:

  • What is the industry average for predictability in software releases?
  • How does predictability change between different company sizes and industries?
  • How does predictability correlate with various engineering tools and practices?

In short, it is our hope to create both a benchmark for you to compare yourself and a quantified set of advice on which practices do the highest-rated teams & companies follow, as well as quantified reasoning to improve your development and management practices.

If you think this is useful information to have, please share the survey with your colleagues, friends, JUG, grandparents, pets and anyone who can click a mouse (and knows what Git is ;-)

Share your dev tools, tech and practices here

  • fd

    I release every day ;-)