Imagine a bacon-wrapped Ferrari. Still not better than our free technical reports.
See all our reports

The Great Java Application Server Debate with Tomcat, JBoss, GlassFish, Jetty and Liberty Profile

Introduction: Let the debates begin…

What type of Java app server should you choose for your next project? Well, that kinda depends on what kind of app you’re building, what your needs are, what type of organization you work in, and lots of other factors too. Hence the debate. So perk up your ears, and get ready for a showdown.

What makes an App Server, Mr. Lebowski?

What makes an application server is a contentious question to many, as the definition is unclear. Wikipedia, the unquestionable source of all knowledge, states:

“Java Platform, Enterprise Edition or Java EE (was J2EE) defines the core set of API and features of a Java Application Server.”

Well, sorry Wikipedia, but for the sake of this report, we don’t care about pedantic definitions and full Java EE implementations, we care more about what a developer wants and uses.

Most developers work on web applications and rarely use all of the bells and whistles that come with the EE specification. In fact many of the application servers available today with only the basic functionality are the most used, as our Developer Productivity Report section on application servers recently showed:

Java Application Server Usage Chart
What we’re looking at in this part of the report are the real developer concerns and metrics including:

  • Ease of download and installation

  • Real performance metrics

  • Tooling support

  • Server Configuration

We’re going to compare and contrast many aspects of these servers scoring each and placing each of them based on our findings. We then address the big questions: Which app server is the best? Doesn’t someone in a big company have different needs than a hobbyist developer geek working at home? How should we weight what’s important to different profiles of people? And so on…

The Application Servers we will be discussing in this report are Tomcat, Jetty, GlassFish, IBM WAS Liberty Profile and JBoss (aka WildFly):

Java Application Server Logos

At first, we were going to include IBM WebSphere and Oracle WebLogic into this report, following the same pattern as with the other app servers. But it felt unfair, as these servers are really targeted for large-enterprises and apps in production rather than lightweight development; however, we didn’t want to dismiss them completely, so they’ll have their own section at the end :-)

What if you no longer had to redeploy your Java code to see changes? The choice is yours. In just a few clicks you can Say Goodbye to Java Redeploys forever.

DOWNLOAD THE PDF


  • Thomas

    It’s really sad that you guys from ZTA ignore so often Weblogic. It’s one of the oldest and from my point of view one very mature App Server in the big $$ Enterprise $$ market. I think WebLogic developer benefit a lot from JRebel. It allows them to program and test quickly and still can delivery rock solid solutions in combination with powerful administration tools.

    Would be nice to see more WebLogic specific stuff here !

  • http://twitter.com/Rebel_Labs Rebel Labs

    Thomas, thanks for your comment. Keeping in mind that Rebel Labs and JRebel are not connected from a product point of view, did you see Page 13 here? http://zeroturnaround.com/rebellabs/the-great-java-application-server-debate-with-tomcat-jboss-glassfish-jetty-and-liberty-profile/13/ — this discusses Oracle WebLogic and IBM WAS and why we didn’t feel it was appropriate to compare them with Jetty, JBoss, Tomcat, GlassFish and Liberty Profile.

    We originally planned to cover both WebLogic and WebSphere in this report, but as we started looking at the criteria that the everyday developer needs, we saw both of those options as not fitting–being SO focused on enterprise requires (i.e. Production-leaning app servers) made it difficult to compare WebLogic with, for example, Tomcat. In future reports, we may cover these two big players, but for more lightweight development, which is what we wanted to focus on here, those big boys didn’t have any place to fit. Could you imagine downloading a 3GB version of Jetty or JBoss? Stay tuned for more about this topic….and increased covered on WAS and WebLogic! :-)

  • Luccas

    @Hedi Pihlamägi

    You made a mistake you gave GF 31.5 should be 36

  • Aguaflush

    I think this comment maybe out of place but, I will give IBM download site (namely Partner world) -1 out of 10. For the same software there are like 10 names for you to choose.

  • Oliver White

    This is actually a great place to leave comments like that Aguaflush — if users are confused, then maybe our small platform here can reach the ears of IBM folks better :-)

  • Oliver White

    Thanks Luccas, we’ll be updating the report with some small error fixes tomorrow. Cheers!

  • Simon Maple

    Hey Thomas, Thanks for the comment. Both WebLogic and WebSphere are way more focused to production like environments to developer needs. As this was a developer focused report, it would be wrong to compare WebLogic download and install to say Tomcat. In many categories it would finish bottom, not because WebLogic is a bad product, but because it’s not focused around developers and ease of development.

    We’re looking into doing a similar report for production and operations style environments, where WebLogic and WebSphere will be more competitive and will be at the front of discussions among others.

    Also as Rebel labs mentions in a comment, we do briefly mention it on page 13 along with reasons we didn’t include it and WebSphere in the bulk of the report. Please look out for the future report I mentioned! Hope you enjoyed the rest of the report.

    Thanks — Simon

  • Simon Maple

    Hey, before I comment with opinon, are you talking about the download site on WASdev.net or the download site on the main IBM pages? :)

  • J Cerwinske

    I consider Tomcat being more lightweight, and that I can later add only what I need for the project (JPA, JAX-WS, JMS, JTA, Spring, etc) by dropping in some libs, as opposed to out-of-the box overhead of full EE spec, an advantage as a developer, not a detriment. It is also nice to have the flexibility to choose a specific implementation provider instead of having to fight against the configured defaults.

  • Simon Maple

    Hey,

    Thanks for reading and thanks for your comments.

    The items you mention have both advantages and disadvantages, depending on your point of view and what your goals are. For many, Tomcat is perfect in that it is lightweight and you can, as you say, add what you want later. For others, this is the pain point, which has led to other projects being created such as TomEE, which does this all for you.

    I understand the point you make that the full EE spec can be considered overhead, although GlassFish and JBoss are still very fast with it, so the overhead is more HDD space and memory, which, yes is higher, but not huge.

    The flexibility to choose a specific implementation provider that suits you is indeed a nice benefit, but then you’ll also have to test and go through any teething problems that may incur. For some people this is a blocker, while for others, it’s painless.

    So I think from your comments, you’re clearly a developer that doesn’t mind building the infrastructure yourself and fixing problems as they arise as well as knowing exactly what you want from your dev environment, but there are many devs that don’t want to do this.

    Thanks — Simon

  • Greg

    Just a minor correction on the tools support for WebLogic… in addition to JDeveloper, there is great WebLogic Server support in Eclipse through OEPE (as shown in your downloads screenshot) as well as in Netbeans.

  • Sanne

    Great comparison, thank you so much!

    I’m having just two areas not too clear, first is about the scores evaluation of Cost $$$/Licensing:
    – Free & no support available -> 5 stars
    – Free & optional support available at a cost -> 4 stars
    I think the second option looks much better, so I’m wondering what I’m missing.

    Performance: I realize this is about developers experience but really publishing a “performance chart” taking into account only bootup experience is pointless. Performance is about what the platform can deliver after it has started.. especially considering ZT’s tools who cares about some more seconds at boot? As long as it’s reasonable we’re fine.

  • henk53

    I too was afraid that WebLogic was 3GB and took hours to setup, and would start in half an hour etc.

    In reality it’s just 183MB and you can download an archive that you can unzip. The server starts up in just a few seconds.

    WebSphere is still 2GB+ and has a mandatory installer etc, but WebLogic 12c is barely larger than JBoss at 183MB.

    See http://henk53.wordpress.com/2012/05/01/is-weblogic-12c-a-heavy-weight-enterprise-solution for more details.

  • Simon Maple

    Hey, this also takes into account the type of license you’re offered. Also support comes in two parts, corporate/paid for support as well as community support.

    Like you say, our tools do eliminate the need for startup, redeploy and initialisation, but not every one uses them (crazy ;) ). We’ll be looking into producing another report which goes beyond developer experience and closer to the operations/production viewpoint. Here you will find more of the statistics you’re suggesting which go beyond developer metrics and more into what the platform can do once started, as you say.

    Thanks for reading, and glad you enjoyed it!

    Si

  • Simon Maple

    Hey Greg,

    Thanks for the feedback – We’ll be doing a refresh soon, so will look to include this in. Thank you!.

  • Adil

    Great job, I look forward to your App Server production and operations review!

  • Simon Maple

    Thanks Adil :)

  • OutspokenDelroy

    Wow! It’s clear that a bunch of smart people put a lot of effort into this comparison, yet the result seems to be disconnected to the concerns of the development teams that I’ve worked in the 15 years I’ve been coding Java.

    I agree with your idea that developers can use a different app server than the one that their code will run in in production, but ultimately what matters is production code. It’s production code that makes money. If, for argument’s sake, your code will run on Weblogic in production (a more likely assumption than most others, given its native accelerator and scalable clustering implementation) then using Weblogic as a dev environment should at least be considered as an alternative.

    Comparing app servers is hard, and scalability and performance should be key factors. Startup time is not performance. If you don’t have the technical competence to do an evaluation like this, then don’t attempt it – what you have done here is write a pretty report that will mislead inexperienced readers and make you look bad for experienced readers.

  • http://www.zeroturnaround.com/ Toomas Römer

    I don’t think scalability and performance should be the key factors to consider when comparing application servers.

    Firstly high scalability and high performance usually isn’t the most important when considering which app servers to use (how many of us get to work on really high performance stuff?).

    Secondly there are no app servers that don’t scale or don’t offer high performance (from Jetty to Websphere all scale and perform).

  • arhan

    “I agree with your idea that developers can use a different app server than the one that their code will run in in production”

    It is not an idea. It is the reality. This is how a lot of developers work.

  • http://www.zeroturnaround.com/ Toomas Römer

    I don’t like this idea. Will introduce issues on the REAL env.

  • Simon Maple

    Hey,

    Thanks for the comment.

    Scalability and Performance are issues for production runtimes, not development. If you want to make your ‘application code’ run in a more performant way, your container is rarely a consideration. If you want your runtime to be performant, then this is an ops consideration. Both these are operations style items which are outside of the scope of this report, which is focused purely on development. (We are looking at doing a production style report which would cover these topics).

    When we talk about performance in this report, it’s around the performance of the developer. How can the dev be as performant in their role. This is absolutely about startup times, initialisations times and so on, as when they lag, the dev cannot be productive.

    Out of interest, which development concerns have you seen in your 15 years which we did not cover in this report?

  • Simon Maple

    Nobody likes this idea, but the alternative is sometimes worse :)

  • Simon Maple

    Scalability and Performance are absolutely key factors! Particularly for Enterprise environments. But these are concerns for production and operations.

    Also, while Jetty and WebSphere do both scale and perform, there will be different efforts and management limitations you’ll hit quite early on with containers like Jetty in comparison to WAS or WL.

  • Zack Belcher

    Just a suggestion – Why not use JBoss EAP (rather than Wildfly, the community edition) in your comparison with Oracle/IBM since it (EAP) is hardened, integrated and enterprise class? Then we could also see your analysis of difference between JBoss Wildfly and JBoss EAP.

    The Wildfly/JBoss AS 7 / EAP 6 architecture is a big step forward, I believe.

    Thanks for the writing this though – great read.

  • Andreas Schmidt

    We have an application here that needs about 15 minutes to start in a dev environment using eclipse + WAS8. 8 Minutes alone are needed for the EAR deployment. The project deployed a lot faster on WAS 6.1 with EJBs being declared in XML. Very likely the annotation scanning in WAS is the problem, because there are a lot of annotated EJBs. IBM really failed here. The same EAR deploys in 45s in LibertyProfile. Not to mentions that WAS8 throws a lot of Exceptions, most popular the “annotation merge NullPointerException” known since 11/2013 and still not fixed?!?!?! Last not least, now I have to run WAS8 with 1.8GB of heap instead of the 768 MB that could satsfiy WAS6.1.

    WAS is unusable for development due to its bloatedness, permanent delays and bugs. I would never recommend using that for either dev nor prod. There is so much fiddling involved to ship around weird effects and inconsisntencies..

  • Anonymous

    Beautiful article, but worst trailer ever :)

  • Marcia Harelik

    Now let’s take the Great Java Application Server Debate on the road. A live, head-to-head challenge death match for developers.

  • Simon Maple

    We try hard to be bad… ;)

  • Simon Maple

    Totally agree – I hope you’re using JRebel to turn all that time into < 1 second! http://jrebel.com/download