A couple of months ago we ran a survey asking a few questions about Java EE development, containers and redeploy times. Now that over 1100 people have responded it’s time to update the results. Since we’ve had more time to analyze them we also hope to provide a few insights into the data including a more detailed container breakdown.
If you’d like to play with the results on your own we‘ve provided all the data and our calculations in a handy Excel sheet that you can download here. As a note, if you haven’t answered the 3-question survey yet, take two minutes and go for it. As more answers trickle in we’ll update this post with the new data.
The first question in the survey was “What container do you use on your largest current project?”. The breakdown follows:
Chart 1: Which Container is used most often?
We didn’t include the containers that scored less than 10 answers in the survey. These included:
- (2) Adobe JRun
- (1) Geronimo
- (1) Iona
- (7) TmaxSoft JEUS
- (5) JONAS
- (1) Pramati Server
- (4) SAP NetWeaver
- (2) Sybase EAServer
Unsurprisingly Apache Tomcat holds the lead, closely followed by JBoss. Together the OSS servers total over 70% of the responses. Although I wouldn’t go as far as to translate these numbers directly into market share, we did find studies that found similar results, courtesy of SD Times – Slight differences though: we asked which ONE container you use for your largest projects, and theirs appear to allow people to choose multiple containers.
The 2nd question we asked was, “How long does it take to restart your container and redeploy your app?”. The breakdown follows (the horizontal axis is in minutes):
Chart 2: “How long does it take to restart your container and redeploy your app?”
We previously assumed that the redeploy/restart phase lasted approximately 1 minute. Though 38% of respondents agreed or said it was quicker than 1 minute, 62% suggested that this process lasts around 2 or more minutes. The calculated average is approximately 2.5 minutes –indicating that we underestimated the amount of time spent on the redeploy phase.
Moving along, we asked, “In an hour of coding, how many times do you redeploy?”. The breakdown follows:
Chart 3: “How Many Times do you Redeploy in an Hour of Coding?”
The distribution seems to be close to normal, excepting the jump in the end. The average is slightly over five times an hour, which coincides with the measurements we did before.
For those who answered, “I never redeploy”, we asked some of them how they did this. Responses were:
- “I’m not the guy that does the redeploys”
- “We develop on embedded jetty & activemq & atomikos in debug mode instead of nearly unusable target OracleAS. So of course we need redeploy or restart jetty as usual, but not the OAS.”
- “I’m in the very early stages of a project so a lot of out time is spent coding and testing without redeploying – typically I redeploy 3-4 times per hour”
- “We use JavaRebel and it’s awesome” (this obviously came from someone trying to charm our development team – who are still blushing. Note – it’s now called JRebel.)
Next we did some data crunching. We assigned numeric values to each of the intervals (e.g. “3.5″ for the “3-4″ interval) and multiplied the number of redeploys an hour by the amount of time one redeploy takes (basically, Chart 2 times Chart 3), thus finding the approximate amount of time respondents spend redeploying in each hour of development. We broke down the data into five minute intervals:
Chart 4: How Much Time Does a Java Developer Spend Redeploying in an Hour of Coding? (using raw data before improving accuracy)
The average is about twelve and a half minutes per hour, accounting for more than 20% of total development time. However, the standard deviation is over 14, which means that the actual per cent varies greatly. We wanted to show more accurate data, so by looking closely and grouping results, we found this:
- Less than 5 minutes an hour. Looking into this group we see that it is slightly skewed by the respondents who indicated that they never redeploy. We asked the people who “never redeploy” how they do it (see comments above), and they indicated that their responses were often not effective for tracking the actual redeploy time that we are trying to measure. Therefore, it makes sense to remove that data from the chart.
- 5 to 15 minutes an hour. 45% of respondents fall into this category.
- 20 to 25 minutes an hour.
- Over 30 minutes an hour. Looking closer at the data we see that some of the respondents indicated that they redeploy over 60 minutes an hour (choosing both “More than 10 redeploys per hour” and “More than 5 minutes per redeploy” for Charts 2 and 3). This is quite impossible. In fact, we believe any number over 40 minutes per hour seems to be against reason. It makes sense to remove all this data from the chart too.
The chart with the updated data looks as follows:
Chart 5: How Much Time Does a Java Developer Spend Redeploying in an Hour of Coding? (more accurate data)
The average is now ten and a half minutes per hour with a standard deviation of 8, which is more trustworthy. This translates to about 17.5% of total development time, which is considerably more than we expected. We will use this “cleaned up data” for the rest of the post.
Later we’ll describe how to calculate this over a year, but here are some quick figures:
- 5 mins per coding hour becomes 6000 minutes annually (2.5, 40-hour workweeks)
- 9 mins per coding hour becomes 10800 minutes annually (4.5 weeks)
- 14 mins per coding hour becomes 16800 mins annually (7 weeks)
- 21% of respondents report spending more than 14 mins per hour on redeploys
- 71% of respondents report spending more than 2.5 full weeks per year, redeploying.
After looking at this issue in general, we investigated the data on a per-container basis. This is what we found:
Chart 6: Organized By Container, How Much Time is Spent Redeploying?
There are few surprises here. Jetty is leading the pack with only 5.8 minutes per hour spent redeploying and IBM WebSphere is trailing with more than twice more — 13.8 minutes per hour. One thing to note is that although Jetty startup is undoubtedly faster than IBM WebSphere it is likely that most of the difference is due to the size of the applications deployed and the technologies used in them.
Next we have the same chart, but numbers displayed as a percent of development time (time spent per hour divided by 60). Perhaps it’s useful to describe how we define “Development Time”. For us, Development time is what you spend actually writing code, with an IDE, running your server and the Java language. This doesn’t include meetings, gathering requirements, smoking or any other activities that don’t involve a compiler. We’ve asked around, and it sounds like the industry assumes that 5 hours per day are spent coding.
Chart 7: Organized by Container, What Percent of Development Time is Spent Redeploying?
To put those numbers in perspective we can also calculate the amount of time spent redeploying a year. To do that, we’ll make some assumptions:
- We assume 48 work weeks per year. If you’re lucky enough to get more than 4 weeks of vacation, don’t rub it in.
- We assume that 5 hours per day is spent doing development. The other 3 hours are reserved for non-development activities.
The following chart displays the number of 40-hour work weeks a year spent redeploying:
Chart 8: How many 40-hour work weeks are spent on the redeploy phase, over a year?
Depending on the container, we’re looking at 3 to 7 work weeks per year, spent exclusively on the redeploy phase. The average over all of the data is slightly over 5 work weeks a year, but the standard deviation of 4 makes the range of 3-7 more reliable.
Our final graph shows the amount of time spent redeploying per container in more detail. Instead of using averages, we break down the proportion of respondents that redeploy into groups: less than 5 minutes an hour, 5 to 14 minutes and hour, 15 to 29 minutes and hour and over 30 minutes an hour. We expect that this may show the size of projects that use different containers, and potentially shed light on the time your project spends redeploying.
Chart 9: Java EE container market penetration.
We interpret this chart as:
- Jetty is only used in projects that redeploy quickly. This makes all kind of sense, considering that Jetty doesn’t even support redeployment and instead has extremely fast container startup.
- Apache Tomcat and GlassFish are used in same types of project. Both are posed as fully functional yet lightweight alternatives to the classic heavy application servers. Although Tomcat is much more popular today, GlassFish is growing in popularity in the same market share.
- JBoss, Oracle Weblogic and IBM WebSphere compete for pretty much the same market segment. The majority of their projects are large and complex, and the redeploy times reflect that.
- Another interesting fact is that the proportion of projects in the 5-14 group is pretty stable for all containers. This likely corresponds to medium-sized projects that are not actively contested by either lightweight or classical application servers. The only exception is Jetty, which is heavily skewed to the lightweight side.
Coming up next: We’re taking a look at incremental build times and comparing Ant, Maven, Eclipse, IntelliJ, and NetBeans. If you’re interested, take a minute to participate here. Thanks!
