Imagine a bacon-wrapped Ferrari. Still not better than our free technical reports.

Developer Productivity Report 2013 – How Engineering Tools & Practices Impact Software Quality & Delivery

“Who is involved in the time estimation for the tasks?”

When it comes to estimating the type and timeline of development tasks, we discovered a direct relationship between quality & predictability and the job role of the individual or people assigning the tasks. The results for ‘job role’ were pretty evenly distributed, so we decided to relate each job role with our metrics for predictability and quality. Check this out:

developer productivity report 2013 rebellabs zeroturnaround

Put simply, when the estimating person or group creating the task is less-technical, there is a decrease in the predictability and quality of the software release. Interestingly, team leads or architects have a null effect on both, whereas estimating with the team as a whole, along with the healthy discussion, generates the most gains in the key metrics. If 6% and 4% gains in predictability and quality respectively are powerful enough drivers to involve the team as a whole, then here is the data to back your move.

“How do you branch?”

The different styles of branching can always generate a discussion both in-house and in public forums. We asked to check off the different styles, and surprisingly, they were pretty evenly represented. Looking at the relations to key metrics we see little strong association either way.

developer productivity report 2013 rebellabs zeroturnaround

The only clear trend is a slightly negative one for the single trunk and a positive one for feature branches. Go figure.

“Who tests your code?”

Are developers or team leads testing your code? If you have a QA/testing department, they are most likely doing it, but perhaps you should consider automating your tests based on these results.

developer productivity report 2013 rebellabs zeroturnaround

If there was even a good case for automation, it would be this. Automated tests showed the largest overall improvements both in the predictability and quality of software deliveries. Quality goes up most when Developers are testing the code (also discussed in Sven Peter’s talk “How to do Kickass Software Development” at GeekOut 2013), which means that you shouldn’t just leave testing to QA team, but bake it into the development process as well. The rest of the measurements were more or less insignificant, although we don’t recommend letting your customers/users test your software for you.

Download the pdf

  • Vjatseslav Rosin

    I’m impressed that FogBugz is not that popular (just 1.7%). We are using Jira, but we were very close to chose FogBugz when we introduced new generation issue tracker.

    Would be interesting to hear reason why FogBugz is not that widely adoped.

  • Vjatseslav Rosin

    Also totally agree on “having Daily standups and Team chat seem to be the best ways to communicate”

  • Loïc Prieto

    As always, a very interesting report! Thanks for the effort, even if I know all is focalized on selling JRebel. But the read is nonetheless very illustrative and helps me deciding wich tecnologies to learn next, so thanks a lot!

    Also, i would love to try JRebel at my work, but i know my bosses will never approve using money for development that could be used for their travels and bonuses… And believe me, i’ve already presented the ROI document you prepared. I think this is one of those situations where a loss in predictably is produced by tech decisions being taken by managers that do not know jack about nothing.

  • balder

    Nice report! It kind of confirmed my gut feeling on what impacts quality and delivery.

  • Oliver White

    What made you choose JIRA over Fogbugz in the end? Was it a single factor?

  • Probably FogBugz doesn’t have such a spread in the developer community compared to BitBucket and Github. It doesn’t have a free plan or it is quite limited. Also BB and GH have so many OSS projects that people have familiarity with BB and GH and this will guide their decision when they need to pick a VCS service.

    I lately read a tweet which said last year GitHub was still the new thing, this year already a standard :). I wouldn’t go to such extremes but kind of true.

  • Oliver White

    Thanks for the compliments! Maybe showing them this report would help your managers understand that both practices AND tools can have a significant effect on your ability to deliver quality software on time.

    At the same time, I’m sorry you see this report as being focused on JRebel. The fact of the matter is that it’s JRebel that pays for RebelLabs to exist and produce all the reports, articles, blogs and surveys that we do. For example, let’s imagine that a report takes 100 hours of just 1 engineer’s time. You see how expensive it can be ;-)

    We created RebelLabs to be a distinct research & content brand outside of the ZeroTurnaround technology products group. We’re a ‘thought product’, if you think of it that way. So you’ll have to hear about JRebel and LiveRebel from time to time with us over here at RebelLabs, unless you suggest we start selling reports for 3 USD per download! ;-)

  • Vjatseslav Rosin

    We really liked FogBugz UX-wise. It’s really clear, simple and straightforward. But I think final factor was a price for on-site installation. JIRA was a way cheaper.

  • Loïc Prieto

    Hi! Thanks for answering, Oliver.

    I’ve expressed badly myself, I’m sorry. This report was clearly not focused on JRebel, but rather I see it as a means to an end, even though the information conveyed in it remains 100% useful and interesting. Now, I don’t mean it in a bad way at all. We’re all working with money that comes from selling something, be it a product, or our services. I’m not contrary to the idea, of course, being myself a professional programmer. In fact, all of the reports I’ve read here in RebelLabs have been very informative and neutral, and closely matched my own experience with the analysed products (or showed me new ones to look at). I think they’ve enriched the community as a whole, so i would like to thank and compliment the team on the very good work.
    I fully understand the situation that you’re being payed by ZeroTurnAround to produce the blog and the reports. The “thought product” you mentioned. And I find it quite nice, because money spent on PR this way is, as I’ve said before, giving to the community freely is a good thing to do. I’ve always found that “positive egoism” is the human trait that moves forward society much better than charity and good will.

    I thoroughly enjoy the articles and reports, with the geeky humour and useful information. (I feel like i’m being targeted by these reports, and JRebel ads with the unicorns and rainbows) So keep up the good work!

  • Jeff123

    I haven’t seen anyone who can accurately predict delivery time for projects beyond a couple weeks. There is fixed scope, flex time and flex scope fixed time. Fixed time fixed scope isn’t realistic. That’s the whole point of scrum and burn down charts, to give constant updates on scope or time to the people who care. At the end of a sprint if your prediction is wrong you change the scope if you have a fixed time release. During the sprint you can tell your off track by looking at a burn down or burn up chart. Tools like pivotal tracker are used to average your estimations so you aren’t predicting when things will get done, the tool takes into account the error in your estimations by averaging how many points you complete in the last few sprints and how many points you’ve estimated tasks to be, and provides estimations based off your average.

  • Mika

    Thanks for the good stuff!!! btw. Have the trends been mixed in the figure: The effects of pairing up on quality and predictability ? According to the text on the page 22, the colours of the should be vice versa.

  • Mika

    Page nbr 18 (slide nbr 22).

  • Oliver White

    Thanks Mika, looks like we had some flipped colors in there. This report is so massive that little errors like this always seem to slip by, no matter how many reviews we do. I guess it’s why coders like to pair up as well–great effect on the “quality” of their code (or document), just like this case! Good job pairing up! :-)

  • Guest

    This is from the document:

    “and JRebel, that time-saving tool we’ve heard
    about before, shows a statistically significant increase of 8% ”

    That you heard before..?

    This is all I have to say.

    Now I am waiting for a report on automatic gear-boxes and how they effect drivers from AUDI. ( I am sure they heard about DSG before. )

  • Whatever gets the giant robots going is good with me:

  • Thanks!

  • Guest

    You should contact Honda then. Or maybe read an article published by Honda. Maybe they heard about some robotics. Don’t go with a paper from Audi or Mercedes. They are not into robotics. You may end up with a tool which they have heard before which improves robots by 8% which has no use to you at all.

  • You’re bringing out one paragraph from a 44 page document. And that one paragraph is based on survey data same as everything else. What’s your problem again?

  • Guest

    “and JRebel, that time-saving tool we’ve heard about before” is based on survey?

    How cute :)

    So it is ok to write a 1000 page report and at the end say that no other tool improves productivity as much as X, where you are related to X? “Based on survey data..” :)

    What is that you were promoting again?

  • The reason why we do productivity related research and articles is because we build productivity tools. We also have to mention our tools in there because they do save you a lot of time. If you are a Java developer then feel free to ping me and I’ll give you a personal demo if you don’t believe this.

  • Guest

    “and JRebel, that time-saving tool we’ve heard about before”

    Does this sentence manipulate people? It makes me think “This is a report from someone who “only” heard about it before, but really does not know about it.. And suprise,suprise… it seems it is the top product after all among the ones they are looking at!

    Why do not you just say, “and JRebel, which is powered by us..” ? or.. “and JRebel…”

    If you believe you are being ethical here, then fine.

    You can clearly see that there are no judgments / opinions on JRebel in my comments, just “ethics”. So you are really replying me on something I am not mentioning really.

    You mentioning your tool there is no problem, you “acting like it is not your tool” is not very nice. If you think you are not really acting like this, then it is probably my wrong judgment. English is not my mother-tongue after all.

    Great article..

  • I think you are reading too much into it. I’ve seen a lot of materials in ZT over there year and we don’t hide the fact that ZT, RebelLabs and JRebel are connected (all reports are a team effort from engineering teams, marketing and design) but of course for every paragraph we have a choice and I think somebody here chose this wording. I’m sure it wasn’t done to hide anything, sorry that it came out like that to you.

  • JimH

    In your report, you indicated that you “publish our raw data for your own analysis”. Where can I find your raw data?

  • We included a whole paragraph to explain the inclusion:

    “*Note: In order to maintain objectivity, we didn’t originally include JRebel in the list of tools. However, we couldn’t help ourselves and matched the emails provided by 47% of all respondents against our client list. We identified that one-third of respondents with email address used JRebel, two-thirds didn’t, and used their data for comparison. The results were too cool to omit :)”

    I don’t know what else should we have done.

  • Guest

    You should have said:

    “We identified that one-third of respondents with email address used JRebel, a tool we heard before…”


  • Sebastian

    Thanks for your post. Regarding Software Development Productivity, I think that you should check which is a tool that tracks Developer’s activity in real-time in order to improve performance through data and facts. This is something like Quantified Self but for software developers. What do you think?

  • Frank

    Insightful report. Thanks. We use online Kanban board to reduce the delivery time and identify the areas that can be improved to increase the quality of our work.

  • K Lenc

    Any opinions about ?