Blog

Wow, we’re on a roll! Just a couple of months ago we released JRebel 3.5, with lots of OSS framework support goodies and now a new major release that reworks all three of the major IDE plugins and includes a ton of other improvements.

The biggest piece of news is that JRebel is now available (and featured) on the Eclipse Marketplace. This means that you can install JRebel from Eclipse with just one click and update through the usual Eclipse channels. On top of that, we’ve improved debugging behavior for Eclipse and IntelliJ IDEA, specifically, stepping now behaves 100% as you would expect. We are also putting final touches on the new and shiny NetBeans plugin, which includes all the features on par with the other two IDEs. It will make the NetBeans Plugin repository sometime in the next two weeks.

As for the other stuff, we have a ton of improvements for GlassFish, JSF Mojarra, Lift, Stripes and Spring MVC Portlets as well as a ton of bugfixes in the changelog. Also, our Enterprise customers will be happy to know that the license server now provides failover.

Search for “JRebel” in the Eclipse Marketplace or just download it now!

It’s been a month since we released the Java EE Productivity Report 2011 and it’s made a lot of noise since then. Some people called us fakes, some loved the data and the results, some accused us of blatant commercialism. We listened to everyone and reached out to those who were silent. As a result we have opinions from representatives of most communities affected by the report. We decided that those voices need to be heard to help put the data in a real context.

Let’s start by those who weren’t too happy with the data. Joe Ottinger, ex-Chief Editor of TheServerSide had the following to say:

Surveys like this are always hard to do, because the results can be so subjective and because the amount of data required to do a survey is so massive. That said, you can definitely see useful stuff here – the data points collected offer a coarse view of things in use, and the questions asked were useful. Where this survey really shines is in highlighting JRebel’s market focus – and it’s scary to think about time wasted in deployment, something most people know is there but when painted in black and white like this… wow. Just wow.

You don’t say! It’s a mystery why people continue to spend 5 weeks a year on redeployment. However I considered it a bit too toothless for him, so after some goading I got the following blunter answer:

I was interested in seeing the productivity survey – it definitely has good buzzwords for interest. However, the survey itself… look, the survey used a lot of exclusions it shouldn’t have (“what APIs do you use?” and then “what frameworks do you use,” which misses the point that many frameworks are APIs in and of themselves.) What I’d like to see is something that acknowledged a greater percentage of *everything* – as in, a survey question and answer set that covered at least 90% of the development situations out there (including the “I don’t develop this kind of app” situation.) The JRebel stuff itself was actually more interesting. The hook itself was lacking something.

Although he didn’t find the survey too useful I think it’s different for the majority of folks. There are some improvements we plan to do next year, but generally the survey gave a nice insight into the Java EE layout. However that’s my opinion and his opinion, so make up your own :)

Although I didn’t think this chart would generate any controversy, Jason van Zyl of the Maven fame had the following to say:

With a sample size of 1027 the variance from what we see is high, but anyone we see is biased toward Maven. We see most enterprise users have switched to Maven. We know in talking with the SpringSource support staff that the use of Spring in the enterprise correlates almost 1:1 with Maven use. What we typically see from surveys is 60-70%.

Apparently Maven is the new Ant, though in the report they are still neck-and-neck.

IDEs are still fighting for dominance, but unfortunately Oracle declined to comment on the NetBeans share, with only this from Duncan Mills:

I noticed in your recent posting (Java EE Productivity Report 2011). That you refer to JDeveloper as Oracle’s “(ex)flagship IDE”. This is an incorrect assumption. JDeveloper is Oracle’s strategic IDE for it’s customers and so is no no way “ex” or de-emphasized. NetBeans remains as the strategic Open Source IDE for non-aligned Java users who are not using Oracle technology (apart from Java of course)

Three flagships in that armada.

Ian Skerett of the Eclipse Foundation did a write-up on the report including the following:

It is great to see Eclipse continues to do well with 65% respondents using Eclipse and if you include MyEclipse that bring the total Eclipse user base to 69%. I was  surprised to see IntelliJ  in second place at 22%,  well ahead of NetBeans at 12%. I actually thought NetBeans was doing better but it might just reflect a survey bias.

Although I didn’t get any comments from the NetBeans team, Geertjan Wielenga did mention that our survey may be biased geographically as we sent an email to the JavaOne attendees asking them to fill in the survey. Some bias is always present and it’s impossible to measure, so it’s possible that actual NetBeans share around the world is larger than this.

We also got a comment from Todd Williams of Genuitec:

In a marketplace overflowing with free tooling, we’re proud to be the top commercial-only offering delineated in the survey. Placing ahead of tooling supported by leading global corporations is a huge compliment to our MyEclipse development team.

As a fellow commercial-only vendor I sympathize wholeheartedly with that statement. Of course IntelliJ IDEA has also been commercial-only until last year, but the statement is technically correct :)

Unfortunately we didn’t get much feedback to the container chart. I would really have liked to hear from the GlassFish folk, but Oracle doesn’t allow unsanctioned comments and we didn’t get anything back from the PR either (not that it’d be too interesting).

The only write-up we got was from Rich Sharples of JBoss:

Below is the 2009 / 2010 Container Popularity chart. Note the significant decline of Websphere and Weblogic and the growth in leaner, Open Source containers like JBoss, Jetty and Tomcat. Glassfish bucked this trend – likely due to uncertainty about it’s future under it’s new owner Oracle. JBoss showed only a little growth – I’ll put this down to a fairly slow year in 2010. But 2011 is going to be very, very different. We already have a Java EE 6 Web Profile container (released last week) and JBoss AS 7 is taking shape pretty rapidly. With our increased attention to slimming the footprint and increasing the speed of adopting new technology and standards like Java EE 6 — my prediction is that JBoss will catch or overtake Tomcat in the next year.

A bold prediction and a great write-up.

The framework chart was probably the most controversial. Howard Lewis Ship, the Tapestry king, didn’t waste any time eschewing his disappointment:

Well, I’m disappointed to see Tapestry come up so far down the list. I don’t know if this is the same with other frameworks, but I work with four different shops that are all using Tapestry 5 and none of them are active on the Tapestry mailing list (bar one particular person) or do any evangelizing. I don’t understand why this is … the majority of Tapestry users seem to use it but I have trouble getting them to get the word out in blogs, feeds … or surveys!

Elsewhere, I see Tapestry running more neck-and-neck with Wicket, ahead of Stripes and Play!, but behind Spring MVC and JSF. However, if you look at which framework dominates the scene in terms of active projects and developers, I think you would find it was Struts 1 … all those legacy apps!

That is unfortunate. Wicket folks on the other hand are quite happy with their share of the market as worded by Eelco Hillenius:

We’ve never aggressively been after trying to get a large mind share, though early on we were involved in discussions on web frameworks a lot, which I guess was somewhat similar to actively promoting it. I think Wicket’s team is pretty happy with the share we have, and as far as I know we’re not talking about how we should increase our share etc. To my knowledge, no-one of the team is directly dependent on how well Wicket does for his living anyway.

I am very surprised JSF is doing so well! I don’t get it. I truly believe it’s an horrendous framework, just a very slight ‘improvement’ over Struts (1 or 2, I both don’t like em, which is the case for most model 2 frameworks). I am also surprised GWT isn’t doing better; I’d expected it to be the leading framework by now. Though after having done a few projects with it, I can certainly see a lot of drawbacks personally. I wouldn’t pick it over Wicket, and that hasn’t got anything to do with my involvement.

Finally, in my mind the big missing piece in the comparison is JAXRS, or even no server framework at all, together with some JavaScript/ Ajax library. We’ve taken that approach for a recent project, and I’ve heard quite a few people from my personal network who were going for that approach as well.

Jeremy Thomerson had the following to add

I’ve found that in big companies (my market), GWT has buzzworthiness like RoR, but not an actual following other than pet projects.  Internal (proprietary) frameworks have the lead, then a mix of Struts / JSP in my informal straw poll of the companies I’ve worked the past few years.

There is a lot of sentiment against JSF going around. In the original report we quoted Matt Raible, who’s known for his independent web framework comparison:

It’s surprising to see that framework popularity closely aligns with the JVM Web Framework Matrix results I calculated at Devoxx. I had Spring MVC and GWT listed as the top JVM Frameworks, along with Ruby on Rails (running on JRuby of course).
While this matrix was the result of much controversy, I think it gives developers a decent technique for choosing a web framework to use. Of course, the way to choose a web framework is to pick a few you like, prototoype with them and see which one satisfies your needs to best. More than anything, make sure the developers on your team like developing with it, as that’s likely to be key in their productivity using it.

I’m surprised to see so many folks using JSF, but I do understand why many companies choose it because it’s such a “safe” choice as a standard. In fact, you could say that Struts 1 and Spring are pseudo standards based on their popularity. However, just because frameworks are popular doesn’t mean you should automatically use them. There’s other great component-based frameworks in Java (GWT, Wicket and Tapestry come to mind) that are easier to use than JSF.

With that in mind we reached out to Edward Burns, JSF spec lead for comment:

I think the JSF numbers reflect the typical environments where EE is used: conservative IT shops where stability and maintainability are prized as much, if not more, than elegance and design purity.  Web frameworks solve a problem most enterprises have, and there are many ways to solve the problem.  Say what you will about JSF, but I know first hand that it is a real synthesis of the ideas from many respected and intelligent practitioners, over a very long period of time.

I felt it a bit too defensive for the most popular framework out there, so I asked Ed to expand on his opinion. He did more than that and wrote a guest post, challenging other web frameworks to deliver the way JSF does. I think he raises some very interesting questions and would really like to see some reasonable discussion about this.

Nobody seemed to object the second part of the report, which is a bit of a letdown. I was really prepared to fight dirty :) So if you’re still spending 17.5% of time redeploying check out JRebel or take a look at some of the alternatives named in the original report.

That’s all folks, thanks for listening!

Jevgeni Kabanov is the founder and CTO of ZeroTurnaround (www.zeroturnaround.com), a development tools company that focuses on productivity. Jevgeni has been speaking at international conferences for over 5 years, including JavaPolis/Devoxx, JavaZone, JAOO, QCon, TSSJS, JFokus and so on. He also has an active research interest in programming languages, types and virtual machines, publishing several papers on topics ranging from category theoretical notions to typesafe Java DSLs. You can follow Jevgeni on Twitter as @ekabanov.

Jevgeni Kabanov: Java EE Productivity Report 2011 marked JSF as the most popular web framework (or standard). This has drawn quite a few negative comments, including Matt Raible quoted in the report and a few others in comments and in the follow-up post Reactions to the Java EE Productivity Report. Personally, I felt that the comments were too aggressive and I asked Ed Burns, the JSF spec lead, for his opinion. This guest post is written by him. He raises some very interesting questions and I’d really like to see a reasonable discussion about this.

I’ll start by calling out some of the comments on the initial posting of the productivity survey:

I wonder how many would choose JSF if it wasn’t a JCP standard. Bruno Borges

And to answer your question about who choose JSF because it’s good, I think for JSF 2.0 that’s a lot. JSF 2.0 is a very good and productive framework. Everyday more component libs become available for it, and unlike JSF 1.x they work great together! (I.e. PrimeFaces, OpenFaces, RichFaces, they all work with each other)Hank de Boer

Above all what I don’t get is why a couple bloggers feel the need to downplay JSF so much. Is this some kind of jealousy because JSF is one of the few single frameworks with a significant amount of users in the otherwise very fractioned web framework scene?Arjan

Web Frameworks have always provoked heated and opinionated discussions. It’s like the Emacs or vi debate of the early 2000s. The fact that we’re still having the debate in the 2010s is a testament to the role of human nature in IT.

I tried various different frameworks, and a couple of them were quite nice really. But the thing is, JSF is also really quite nice, especially JSF 2.0. The component sets available for it are really cool (I especially like PrimeFaces) and there are many interesting extensions like those offered by Seam. Don’t forget that developers participating in this survey all did so out of free will and most likely indeed the active developers who have a passion for development participated. If they for some reason had to use JSF but didn’t like it, I don’t think they would be giving JSF ‘points’ in a survey such as this.Arjan

In my role as the individual who has the fortune of leading the JSF spec team, the existence of this back and forth is really the best validation available for our work on JSF. It’s important to point out that similar dialogs exist in the comment threads on other sites as well.

Premise: All Web Frameworks Solve the Same Problem

Let’s put opinion aside and look at some technical aspects. I want to limit the discussion to focus on frameworks where the UI logic and state resides mostly on the server. This excludes frameworks such as GWT (and its Vaadin abstraction), JQuery+REST, Flex, and, of course, JavaFX. Even with those exclusions there is still a huge amount of variety, but let’s take a look at what they all have in common. You could call this MVC, but I find these terms are easier to understand.

Problem statement

LIFECYCLE

All of them provide a request processing lifecycle in one way or another.

BINDING

In all of them, the purpose of the request processing lifecycle is to map the request data into a UI that hooks up to some business logic.

UI Construction

In all of them, some part of the development story involves producing some static artifacts that reside on the server (Java Classes, XHTML Pages, Velocity Templates, JSPs). These artifacts are executed by the server to produce HTML that is sent to the browser.

That’s pretty much it. All of them do that in one way or another. Now, let’s take a look at each element and ask some questions about the requirements for each subsystem.

Premise: All Web Frameworks Solve the problem differently, and at different levels of completeness.

Technical considerations, for all the subsystems:

• Is the sufficient tool support?
• How hard is the build process?

Non-technical considerations, for all of the subsystems

• How hard is it to learn? Does it subscribe to the “pay-as-you-go” complexity tax, or the “all-up-front” complexity tax? In other words, can you pretty much forget about it until you actually need to do something specific with it?
• Does the technology scale well to large teams with diverse skill sets and different levels of proficiency?

LIFECYCLE

Technical considerations

• Does the framework handle all aspects of what can happen in a web request/response lifecycle?
• Does it handle typesafe conversion?
• Does it handle per-field validation?
• Does it allow for inspection of the progress of the lifecycle, and interruption if necessary?
• Does it respond well to malicious requests?
• Does it respond well to DoS attacks?
• Does it consume a lot of time? A lot of memory?
• Is it extensible, decoratable, or otherwise customizeable?
• When something goes wrong, how hard is it to understand how to fix it?

BINDING

Technical considerations

• How easy is it to modify the mapping between request data and business objects on the fly?
• Are all the necessary cardinalities of mapping supported?
• How loose is the coupling between the artifacts in the UI Construction subsystem and the business objects?
• Can the bindings be checked for type safety at compile time? Assembly time? Run time?
• When something goes wrong, how hard is it to understand how to fix it?

UI Construction

Technical considerations

• How good is its I18N/L10N story? Its A11Y story?
• How maleable are the artifacts that comprise the UI?
• Do they require redeployment to modify them?
• Can the artifacts be templated for modular reuse? Can they be treated as true black-box components, with properties, methods, and events?
• When something goes wrong, how hard is it to understand how to fix it?
• What skillset must one have in order to author good-looking UIs? Is the skillset transferrable to other frameworks/jobs/professions?
• How hard is it to do abstraction by aggregating artifacts into a composite artifact?

Non-technical considerations

• Is there a third party market for UI artifacts?
• Is the market strong and healthy, with new products entering regularly?
• Can the products in this market be combined together to make new products?

Conclusion

I challenge you to take each of the framework contenders and ask these questions about them. I assert that JSF has very strong answers to all of the questions. Can the same be said of Wicket, Grails? Play? Echo2? Tapestry? Stripes? etc.

There are certainly areas of improvement for JSF. For example:

• Pre runtime type safety checks in BINDING
• Better runtime performance in LIFECYCLE and BINDING
• Better error understandability in LIFECYCLE

Because JSF is a standard, developers can be pretty sure that the framework will receive continued refinement and development. When you combine that with the solid technical and non-technical aspects, that’s pretty darn compelling.

Ed

Ed Burns is a senior Java engineer at Oracle. Ed has worked on a wide variety of client and server side Web technologies since 1994, including NCSA Mosaic, Mozilla, the Sun Java Plugin, Jakarta Tomcat and, most recently JavaServer Faces. At Oracle (Sun), Ed leads a team of web experts from across the industry in developing JavaServer™ Faces Technology through the Java Community Process and in open source. Ed is currently the co-spec lead for JSR 127, JavaServer Faces, a topic on which Ed recently co-authored a book for McGraw-Hill. Ed is an experienced international conference speaker, with consistently high attendance numbers and ratings at the JavaOne conference, JAOO, W-JAX, No Fluff Just Stuff, JA-SIG, The Ajax Experience, and JUGs and Linux User Groups.

Disclaimer: ZeroTurnaround is a commercial vendor, but Twave is not a commercial product or in fact a product or project of any kind. It’s an approach we use at ZeroTurnaround to keep tabs on our team and business activity using Twitter that we would like to share with the world.

It may not be obvious to an outsider, but ZeroTurnaround has exploded during the last year. Our team went from 4 to 12 engineers in half a year and is continuing to grow every month. This may not seem much to someone from a large corporation, but for us it was a pain. Mainly, because everyone on our team is a star in his own right and we didn’t want to compromise productivity even a little bit.

To achieve that we started by cleaning up our tools and process. We switched from SVN to Mercurial to better accomodate  the team size and code complexity. In fact we are using a hosted FogBugz & Kiln solution from the Joel Spolsky’s FogCreek and are pretty happy with the productivity it offers (I have a lot more to say here, but it’s a story for another day). We also use Maven and Hudson to ensure a continuously stable nightly build and a rock solid release.

Unfortunately even with those tools in place we keep making mistakes (and what’s worse not catching them in time). E.g. we rewrote our testing framework three times. Granted it’s a big pile of awesomeness managing a farm of cloud instances, but we did lose a lot of time on going down the wrong path and if we paid more attention we could have avoided it.  We had a few grandiose and embarrassing failures right after the release, because the tests were missing and nobody noticed the changes made. We had people waste time overengineering a bicycle, when a guy sitting in three meter radius would have known a simple and easy solution.

You may notice that all these problems were caused more or less by lack of communication. More precisely, it’s what Alistair Cockburn describes as Osmotic Communication, background information sharing that happens without people even realizing it. We value communication very high at ZeroTurnaround and use our own brand of agilish process. Some of it is like Scrum, except that daily stand-up meetings are replaced with daily lunches, much more nourishing for both body and soul. But largely it’s inspired by Crystal Clear methods of Alistair Cockburn and our own experiences in the field. But clearly we were missing something in our process that would enable the busybodies like me to interfere when something smells wrong.

What we needed was some way to follow the team activity without too much overhead and spike conversations when something goes wrong before long. One appealing idea in that area is repeteadly expressed in Malcolm Gladwell’s Blink: The Power of Thinking Without Thinking: it’s basically well known that people are good at pattern matching at break necking speeds (mind you, Blink generally makes a lot of unscientific claims, but this one is fairly well supported by the current evidence). E.g. just scrolling through a bunch of text you can get the gist of the meaning. Same way by looking at the team activity even for a fraction of a second some things stand out. That doesn’t mean that whenever something wrong comes in it’ll be noticed by everyone, but with the whole team watching there’s a good chance someone will notice.

We looked for solutions around the web and didn’t find anything we liked too much (IBM Jazz purports to enable a similar style of information sharing, but come’on). However we found inspiration in several Open Source efforts. Specifically cia.vc allows for great insight into the Open Source projects linked to it. You can follow activity of specific developers or the project as a whole and have instant conversation starters when something goes wrong. We also liked the  the IRC bots and the coolness of having conversations in the same medium.

Twaves

After some investigation we decided to use Twitter to follow the team activity stream (or twave, name having no connection to me being a Firefly fan). There are several reasons for that:

• Twitter is really easy to use. In fact most of the people on our team were already using it. It has a multitude of desktop and mobile clients, it has a great web site and it keeps improving, but still stays simple.
• Twitter is real easy to integrate with. Besides the OAuth, which can be a pain, but needs to be done only once, the API is all simple REST and JSON. We hacked on it mostly from PHP, but it’s easy to use “as is” in any language (except Java, but it has a nice library).
• Twitter has a great ecosystem around it with a bunch of ready-made apps we could use. In fact our first integration was just using twitterfeed.com to feed a bunch of RSS we already had into our protected Twitter account.
• Finally, Twitter is short. Considering that we want to take advantage of pattern matching machinery of the human mind it’s important to keep it as short as possible, but not shorter :)

There are also specific reasons we didn’t choose other mediums:

• The classical way to follow such activity is email. Email is deceivingly better, as it can have unlimited amount of content and comes to the inbox where it surely will be read. Reality, however, is that the sheer amount of content makes the email hard to scan quickly and generally automated emails quickly overwhelm the recipients and get filtered in a nice dusty folder that nobody ever reads.
• Another established way is to have an IRC bot. It’s a great alternative for those who are used to it, though probably a little harder to integrate and maintain. The main reason we didn’t go with it is that nobody on our team uses it. Jabber would also be a nice option, but it’s support for group chat is still not in mainstream use. And Skype does not have an API to hack on.
• Yammer, as a walled off Twitter alternative was also evaluated as an option. However as Twitter allows protected accounts, has an established and easy-to-use API, has a multitude of clients and was already used in the team we didn’t see much reasons to use it. But it is definitely a reasonable alternative if you ara already using it or if you need centralized administration or other enterprizy options.

So after the medium was decided it was time to build the twave. We registered a protected account and decided to include the following events:

• Commits, including the message and the committer, naturally.
• Bug tracker activity. Initially we used RSS so only new bugs were shown, but later we included assignments, closed bugs and updates.
• Support emails. We use FogBugz for support so it was actually included in the bug tracker activity.
• Hudson build notifications.

Additionally we also included purchases to keep the team motivated as well as some web statistics. In your case, depending on the app and team responsibilities, you might want to include:

• Monitoring alerts: hardware failures, low resources, slowdowns and whatever else.
• Failures: exceptions, lost user sessions, etc.
• Heartbeat: just as important can be for the servers to report that all’s well and include some running stats.
• Business events: purchases, complected workflows, etc

In the end we found it important to include relevant team and business events, that allow to follow the activity of the team, the apps and the business as a whole. However the reasons for that were not solely to prevent failures, but also to react to them quickly, align the team goals and increase motivation.

So did it work for us? You betcha! It’s not used in equal measure by everyone on the team, but it became an invaluable tool for the leads at the very least. Some things that we noticed:

• The beauty of Twitter is that everyone uses it differently. Some prefer the desktop client, some prefer the web. Some have it notifying in real-time via Growl or similar, some scroll through a queue of tweets a few times a day. A common pattern for smartphone owners is to go through canned tweets while on the can :)
• We noticed quite a few weird things that would otherwise slip our notice. E.g. a well intending engineer renamed all plugins to follow the same convention of “xxx_plugin” (whereas previously some of them used the dash, like “xxx-plugin”. Unfortunately since the plugin names spill into the user configuration it would have been a much larger and more dangerous change than he intended. More dangerous things were noticed and prevented as well, some that would cause crashes.
• In some cases things that someone thought were dangerous or bad, were actually well motivated. But the discussions spiked by the activity stream allowed us to improve them to be better understood by others and make sure that everyone is on the same page.
• Finally, the fact that we can follow sales and other business events gives the team a day-to-day feedback on their work and their success. We are very thankful to everyone who purchases our products and are proud that we can add value to the Java community. This pride is what motivates us to continue innovating.

One issue we haven’t touched is security. In some companies it might be impossible to keep any data in the cloud, but there are a few tricks to make it less sensitive:

• The data includes mostly titles, urls and maybe some emails. URLs are internal, so they cannot be accessed even if known. Some companies don’t like to have any data about their netwoks outside their tight control, but that’s security by obscurity :)
• You can run a nightly script that will delete all history longer than a day/week. This ensures that even if account is broken into it only has the latest history available.

Outer Rim

So now that we’ve explained our reasons and the basic setup we used let’s dig deeper and examine how exactly we integrated with Twitter and how the twave is shared, aggregated and accessed:

• Although in the previous section we were talking about a single protected accounts, we eventually created several. They corresponded to different twaves produced by our team, by our business and by our web site (e.g. correspondingly dev, sales, web). This was done because not everyone was interested in all events equally and this way they had a choice to subscribe to the relevant ones. This also allows us to easily create in the future additional “noisy” twaves without adding noise to the current ones. This is also important for access control to “sensitive” twaves.
• It’s important to meaningfully aggregate events that would otherwise produce too much noise. E.g. we get a lot of downloads, so producing an event for each download would quickly drown the twave in noise. So instead we can have hourly/daily stats visible in the twave and only include anomalous events (e.g. a download from Somalia) that may be of interest to us.
• We would also like to be able to discuss the stream in the same medium. It’s quite possible to register protected accounts for this purpose and freely discuss this on Twitter, but this experiment is still ongoing. Usually we just discuss the stream events in real life, so it’s not a big problem for us. It’s also fairly easy to copy paste the events into the group chat (we use Skype) so I’m not sure if it’s a real issue.
• Sometimes you want the alerts to be delivered immediately, not wait until you go through the Twitter queue. For this purposes we use Prowl that delivers push notifications to iPhone. However in countries where Twitter will send SMSes you can also register a custom account associated with your phone and push the events there. You can also use Twilio to script a creepy robotic call in the middle of the night when something occurs.

Unification

When it comes to the actual implementation there are quite a few annoyances that you can copy-paste from us instead of solving yourself again.

To gain access to the Twitter API you will need to hack through the gordian knot of OAuth. It’s a bit of a hassle to use, even with the help of the libraries. Since we need to access accounts we control we can take a shortcut. Namely, sign in at dev.twitter.com with your twave account and register a new app at dev.twitter.com/apps/new. You can register a “Client” app, but make sure it has “read and write” access. Once it’s done go to your app and copy “Consumer key” and “Consumer secret”. The click on the “My Access Token” and copy the “Access Token” and “Access Token Secret”. You will need all four for each twave as shown in the the following PHP example:

function notify_dev($message) {
  $connection = new TwitterOAuth(
    DEV_CONSUMER_KEY,
    DEV_CONSUMER_SECRET,
    DEV_OAUTH_TOKEN,
    DEV_OAUTH_TOKEN_SECRET);
 
  $parameters = array('status' => "$message");
  $status = $connection->post('statuses/update', $parameters);
 
  if ($connection->http_code != "200")
    error_log(print_r($status, true));
}

We are using the twitteroauth library, which is a very thin PHP wrapper over the Twitter API, but any other will behave similarly.

The next problem we faced is that Twitter does not automatically shorten URLs. However bit.ly provides an extremely easy to use API that can be accessed like this:

function shorten_url($url) {
  $ch = curl_init("http://api.bit.ly/v3/shorten?"
            ."login=$login&apiKey=$apiKey&domain=j.mp"
            ."&longUrl=".urlencode($url));
  curl_setopt($ch, CURLOPT_HEADER, false);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $result = json_decode(curl_exec($ch));
  curl_close($ch);
 
  return $result->data->url;
}

All you have to do is register a user to get an API key and decode the returned JSON.

An additional thing we do at ZeroTurnaround to help twitterize the events, is to replace names, usernames and emails in the twave to Twitter @usernames (they are highlighted nicely, easy to reply and show up in mentions). We do that with a simple search & replace:

function normalize_email_address($email) {
  $pos = strpos($email, "<");
  if ($pos !== false) {
    $email = substr($email, $pos + 1, -1);
  }
  return $email;
}
 
function contact_to_twitter($contact) {
  $ZT_CONTACTS = array (
    array (
        'name' => ''Malcolm Reynolds",
        'email' => 'mal@zeroturnaround.com',
        'twitter' => '@malcolmreynolds'
    ));
 
  $contact = normalize_email_address($contact);
  foreach ($ZT_CONTACTS as $c) {
    $contact = str_replace($c['name'], $c['twitter'], $contact);
    $contact = str_replace($c['email'], $c['twitter'], $contact);
  }
  return $contact;
}

One thing that you need to know here is that Twitter will hide the tweet starting with @username if you do not follow that user. Therefore you need to add something in the beginning of the tweet to avoid that. It can be as simple as putting a dot in the beginning “.@javarebel” or you can make use of that and add a unicode symbol like “☯” that distinguishes the event type.

To make sure that events come in realtime we either post them directly when the action happens (e.g. purchases or downloads) or use callback scripts. A very useful feature in many cloud-based services (like the hosted FogBugz & Kiln we use) is WebHooks. Basically you get a callback HTTP REST request to a URL of your liking with payload as GET parameters or in the POST body. So in the final code snippet of the article, here’s the whole of our commit hook:

<?php
 if(!$_REQUEST['payload']) exit();
 require_once 'zeroturnaround/lib/notify.php';
 
 $payload = json_decode(stripslashes($_REQUEST['payload']), true);
 
 foreach ($payload['commits'] as $c) {
   $message = "☯ ".contact_to_twitter($c['author'])
   ." committed ".$c['message']
   ." to ".$payload['repository']['name'].": ";
 
  if (strlen($message) > 120)
    $message = substr($message, 0, 118)."..";
 
  $message .= " ".shorten_url($c['url']);
  notify_dev($message);
 }
?>

As you can see from the script it is important to keep the Twitter message short. The total size is 140 symbols, so we reserve 20 symbols for the shortened URL and make sure that the rest of the message is 120 symbols or less. The script is called with a JSON payload in the POST “payload” parameter as documented in Kiln and the rest is straightforward.

Serenity

It’s my hope that the solution described here will be of help to those feeling the stress of the communication failures. As we came up with this approach a while ago I spent some time searching the web for similar efforts. I couldn’t find any, which means that either this stress is unique to us or the issue doesn’t have the prominence it deserves (or perhaps I suck at googling, which is totally an option as well). I’d like this to change, so I ask that if you have done or planning to do something similar (or want to adopt this solution), please write to jevgeni at zeroturnaround dot com. Also if you know of any public records of similar approaches be nice enough to link to them in the comments.

Can’t stop the signal!

Jevgeni Kabanov is the founder and CTO of ZeroTurnaround (www.zeroturnaround.com), a development tools company that focuses on productivity. Jevgeni has been speaking at international conferences for over 5 years, including JavaPolis/Devoxx, JavaZone, JAOO, QCon, TSSJS, JFokus and so on. He also has an active research interest in programming languages, types and virtual machines, publishing several papers on topics ranging from category theoretical notions to typesafe Java DSLs. You can follow Jevgeni on Twitter as @ekabanov.

Our newest product, LiveRebel, has been in development for a while now and we have gathered a ton of feedback. The latest beta release is stable enough that you can just take it, install it and run it. So we thought we’d give you a chance to try it out.

But hold on – I’m getting ahead of myself.

The first question you probably would ask is what is LiveRebel. Basically, it’s the first cousin of JRebel, but running in the production environment.

The current version applies live updates to your clustered application instantly – from your choice of web console, command line or through the REST API. As the Rebel engine has some limitations, it does a compatibility check before applying the upgrade, so nothing interferes with the application stability.

You can use the functionality provided by LiveRebel to rollback failed updates and to create a fast track for critical fixes, ensuring that your application is stable with maximum availability at all times.

I hope this is as exciting for you as it is for us! Just sign up here and you’ll get your download link right away:

The last few months were very busy here at ZeroTurnaround HQ in the cold and gloomy Estonia. We wanted to deliver something awesome to our users, before we fully switch to working on the larger things planned for the 4.0. To that effect we decided to address some of the long standing requests as well as make some improvements to deliver better on our promise of “All changes reloaded, instantly!”. Whereas the previous major release focused mainly on the Java EE standards, this time we put our attention to supporting various Open Source frameworks and tools.

Last month we ran a survey on development environments (which is still open, btw, feel free to take a couple of minutes and make your voice heard). In the preliminary results we saw that Spring is used in almost half the Java EE apps today, so it’s great to say the 3.5 improves your Spring experience by working better in applications without Spring MVC and with the new plugin for Spring WebFlow that reloads your flow configuration.

The new plugins for OSS frameworks include Tiles, Wicket, Lift and others. We reworked the JRebel core to better support the OSGi containers, like Equinox and Felix. We improved serialization behavior and reflection performance. Scala and Groovy are now officially supported with JRebel. A score of other improvements and fixes is available in the changelog as well. Whew, that’s a lot of things for one release!

Oh, and one more thing — for years one minor, but persistant, annoyance for our users was the extra frames in the stack trace that JRebel introduced. Starting with 3.5 JRebel no longer adds them!

Get it while it’s hot!

Last year, we published a report on turnaround time, tools and application containers in the Java ecosystem. Over 1300 Java developers ending up sharing info about their development environment, and over 40,000 people found these results helpful.

This year we expanded on that, asking more and better questions to give you better and more accurate information. This preliminary release of our data includes 946 responses that were analyzed to create this report. The raw data, including all calculations, will be made available for download when the final version of the report is issued in December 2010. We want to discover:

• What is more popular, Ant or Maven?
• What Java IDE is the most popular?
• What Java EE standards are used the most?
• Which Java Framework are most prevalent?
• What Java Container / App Server is used the most?
• How much development time is spent redeploying Java containers?

The survey is still live. It’s 6 questions (slightly different than those above), and takes 2 minutes. Take the survey.

Build Tools and IDEs – Setting the scene

We asked everyone to mark the tools they use for building applications. The only two build systems to receive more than 1 vote were Maven and Ant:



As you can see Maven and Ant are used almost equally and some respondents chose both. Clearly they are each useful in certain ways.

The IDE chart is more interesting:

Eclipse has emerged here as the clear leader (in terms of # of users at least), with only one-third of respondents using IntelliJ IDEA (#2) and Netbeans (#3).

Servers & Frameworks – Who are the popular kids at school?

Compared to the last year’s chart we see % gains for the open source containers. We can also see that Jetty and Tomcat have a bigger share than last year, while Glassfish is sliding a bit. Oracle Weblogic and IBM Websphere have lost a total of 8% of the market to the open source containers compared to the last year (according to this sample of developers at least. A larger sample = better results. Contribute here :) ).

As far as Java EE standards are concerned we have the following picture:

JSP keeps the throne as the most popular Java EE standard (along with Servlets and JDBC not present here). JPA is amazingly popular, having over a third of the market. EJB versions altogether have 39%, which would make them the most popular standard, but there’s likely some overlap between EJB2 and EJB3 users, so the total is likely a bit smaller.

Let’s take a look at the framework chart:

Spring and Hibernate are by the most popular and in fact are still used more than standards. However, as far as web frameworks go JSF looks like a popular choice with 24% of answers. Unfortunately, we haven’t separated Spring MVC from Spring, but it’s likely at least as popular as JSF. The rest of the frameworks hold a share of 10% or less (GWT is barely over that and is the third most popular).

Turnaround times – How does your environment compare?

This year we asked questions about the time it takes to redeploy the application and number of redeploys per hour. We also asked those who don’t have to redeploy to comment why this happens. We were glad to see the development in this area since last year – these were some of the responses we received:

• “I wait till I have enough changes to warrant a redeploy”
• “We use tests (unit tests, integration tests, etc)”
• “We use Eclipse/Sysdeo/Intellij IDEA/etc HotDeploy”
• “We use HotSwap/Eclipse Debugger Session/etc”
• “We use OSGi”
• “We use Grails, Play! or Tapestry 5″
• “We use JRebel” :)

(Commentary on these responses, provided by ZeroTurnaround Founder & CTO Jevgeni Kabanov, will appear elsewhere and in different forums).

We originally did not include a “30sec” answer to the question “How long does take to redeploy your application?” However, we have since received a few comments about that – and to compensate for that, we will use a “45sec” value for the answer “1 minute” answer. But you can see that 30 sec has since been added to the survey.

The first Turnaround question was “How long does it take to restart your container and redeploy your app?” The answers are distributed like this:

We didn’t ask with such accuracy last year, which makes it harder to compare, but it seems the trends have remained the same.

Last year’s average redeploy time was 2.5 minutes, and this year the average redeploy time is 3.1 minutes. But with a standard deviation of 2.8, which means that the redeploy time varies greatly. It can be noted that a statistically significant segment of respondents chose answers over 10 minutes.

The next question was “In an hour of coding, how many times do you redeploy?”:

The average frequency is 4 times an hour with a standard deviation of 3.2. Last year, our average frequency for redeploys was 5 times per hour.

To calculate the total time spent redeploying in an hour we first removed those who “don’t redeploy” at all and those that reported redeploying “more than 45 minutes” of every hour (woah!). Calculating the total we get:

Average respondent spends about 9.7 minutes an hour redeploying with a standard deviation of 8. This is about 16.1% of total coding time.

As with last year, we took the assumption that the average Java developer spends about 5 hours of every day coding. The other 3 hours of each day we assume are spent answering emails, having meetings, drinking coffee, swordfighting, and doing other tasks.

Taking into account this 5-hour coding day and 4 weeks a year of vacation [some get], we get about 4.8 work-weeks each year spent redeploying. So, even a lucky developer that gets 4 weeks a year of vacation time will spend on average even more time than that waiting for redeploys.

Finally, we can also see how the choice of container correlates to the time spent redeploying. Note that it doesn’t mean that bigger containers are that much slower, rather bigger apps influence the choice of container:

Compared to last year, not many application servers changed, with the exception of Glassfish v3, which is apparently faster than before.

Stay tuned for the final report in December. More good stuff (including analysis) to come.

Remember, this survey is still open and the more developers that come to share their Java environments, the better and more applicable our results will be! Did we mention that before? ;-)

JRebel 3.1.2 incorporates improved support for OSGi (all containers), JBoss 6 and Freemarker as well as a host of fixes. See the full change log or download right away.

This is a story of an interesting support request handled by our guru Lauri Tulmin. The inquiry was about a rare occurring ArrayIndexOutOfBoundsException that JRebel seemed to cause in Wicket. After some investigation he discovered that the root exception was thrown from the following Wicket code:

private final Map<imodifiable, Entry> modifiableToEntry =
   new ConcurrentHashMap();
 
  public void start(Duration pollFrequency) {
    this.task = new Task("ModificationWatcher");
 
    this.task.run(pollFrequency, new ICode() {
      public void run(Logger log) {
        Iterator iter = new ArrayList(
          ModificationWatcher.this.modifiableToEntry.values()).iterator();
        while (iter.hasNext())

Specifically the exception was thrown from the constructor of ArrayList. How was that possible?

Let’s take a step back and review the reasons for this code. One issue with collections is that when iterating a ConcurrentModificationException is thrown if a collection is changed (usually by a different thread). This is done to protect the Iterator from unpredictable behaviour. A common idiom to avoid it is to copy the collection before iteration like that:

for(Iterator i = new ArrayList(collection).iterator(); i.hasNext();) {...}

Note that the collection must either to be synchronized or otherwise suitable for a multi-threaded environment.

This idiom is a very common one, as easily proven by a simple Google Code search. In fact we used it several times in the JRebel code and is present in several places in Wicket as well. So how could this cause an ArrayIndexOutOfBoundsException?

When Lauri investigated in depth he found out that this idiom is inherently unsafe in a multi-threaded environment (even when the underlying collection is synchronized!). The reason for that is the way ArrayList was constructed before Java 1.6. In my 1.5 Java SDK source code it looks like this:

public ArrayList(Collection<? extends E> collection) {
  int size = collection.size();
  firstIndex = 0;
  array = newElementArray(size + (size / 10));
  collection.toArray(array);
  lastIndex = size;
  modCount = 1;
}

The issue is that there is a race condition between the time the size is recorded and the collection.toArray(array) is called. During that time it is conceivable (and indeed possible) that the size of the collection is changed by a different thread. If the size is now bigger, the array copying in toArray() fails with the dreaded ArrayIndexOutOfBoundsException. When researching the issue further we found that the constructor in Java 1.6 ArrayList has been changed to avoid this issue, so at least on Oracle Java 1.6 or later you are safe. Unfortunately I couldn’t find a specific bug referring to that issue, so likely as not it’s an accidental fix.

So what should you do to avoid this issue? One way is to put the synchronized block around the whole loop, but this requires that you only access that collection from other threads in the same synchronized block. An easier solution is to use the toArray()method like this:

for (Iterator i = Arrays.asList(collection.toArray()).iterator(); i.hasNext();)

This minor release includes a score of improvements across the board. Among the most important is that the Google App Engine support is now out of the beta and Google Web Toolkit support has been substantially improved, and the only reason we still keep it in beta is because we don’t have a full automated test harness for it yet. We have also tested JRebel with Tomcat 7 and Jetty 7.1 and can now declare them officially supported. Beta support for serializing added fields has been added, but is still disabled by default. Please add -Drebel.serialization=true and let us know how it works for you. Also we worked some of our magic with AspectJ and Groovy, so those of you who had issues should definitely upgrade to this version. See full changelog for details, or grab the download!

« Newer Entries | Older Entries »