Blog

We’re glad to announce the JRebel 2.2.1 release. It is a maintenance release incorporating all the bugfixes that have made since the 2.2 release. You can see the details from the full changelog.

Pick up the new version at our download page.

It is our great pleasure to announce JRebel 2.2, the “Easy Peasy” release. In this release we have focused heavily on ease of installation, configuration and use. The main new feature is the semi-automatic installer and configuration wizard, that makes installing JRebel and configuring your application a snap. We have also included a configuration utility that supplements all those funky system properties with a centralized GUI configuration. For those who prefers the Zen of the Command Line we have compiled a comprehensive reference manual about all things JRebel included in the distribution.

We also invested heavily into making the JRebel IDE integration as seamless as possible. The new NetBeans plugin now supports debugging when the JRebel agent is enabled. The numerous updates to Eclipse and IntelliJ IDEA plugins allow you to run the application with JRebel effortlessly from inside the IDE.

Take a look at the full changelog, download now or check out the screenshots:

Next week we plan to release the 2.2 version of JRebel. The two main features of this release are:

• The new semi-automatic installer and configuration wizard, that make installing and configuring JRebel a snap.
• New or updated releases of IDE plugins for Eclipse, IntelliJ and NetBeans (both 6.5 and 6.7).

While we’re finishing the polish on the release itself, we have already uploaded the IDE plugins to their respective repositories. Just hit “Update” and you can start using them right away. We have also uploaded a preview version of the Installer-enabled JRebel distribution, which you can now get from Downloads.

Other Articles in the Reloading Java Classes Series

• RJC101: Objects, Classes and ClassLoaders
• RJC201: How do Classloader leaks happen?
• RJC301: Classloaders in Web Development — Tomcat, GlassFish, OSGi, Tapestry 5 and so on
• RJC401: HotSwap and JRebel — Behind the Scenes
• RJC501: How Much Does Turnaround Cost?

A video presentation “Do you really get class loaders” by Jevgeni Kabanov

From ClassLoaders to Classes

If you have programmed in Java for some time you know that memory leaks do happen. Usually it’s the case of a collection somewhere with references to objects (e.g. listeners) that should have been cleared, but never were. Classloaders are a very special case of this, and unfortunately, with the current state of the Java platform, these leaks are both inevitable and costly: routinely causing OutOfMemoryError’s in production applications after just a few redeploys.

Let’s get started. Recalling RJC101: to reload a class we threw away the old classloader and created a new one, copying the object graph as best we could:

Every object had a reference to its class, which in turn had a reference to its classloader. However we didn’t mention that every classloader in turn has a reference to each of the classes it has loaded, each of which holds static fields defined in the class:

This means that

1. If a classloader is leaked it will hold on to all its classes and all their static fields. Static fields commonly hold caches, singleton objects, and various configuration and application states. Even if your application doesn’t have any large static caches, it doesn’t mean that the framework you use doesn’t hold them for you (e.g. Log4J is a common culprit as it’s often put in the server classpath). This explains why leaking a classloader can be so expensive.
2. To leak a classloader it’s enough to leave a reference to any object, created from a class, loaded by that classloader. Even if that object seems completely harmless (e.g. doesn’t have a single field), it will still hold on to its classloader and all the application state. A single place in the application that survives the redeploy and doesn’t do a proper cleanup is enough to sprout the leak. In a typical application there will be several such places, some of them almost impossible to fix due to the way third-party libraries are built. Leaking a classloader is therefore, quite common.

To examine this from a different perspective let’s return to the code example from our previous article. Breeze through it to quickly catch up.

Introducing the Leak

We will use the exact same Main class as before to show what a simple leak could look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

public class Main {
  private static IExample example1;
  private static IExample example2;
 
  public static void main(String[] args)  {
    example1 = ExampleFactory.newInstance().copy();
 
    while (true) {
      example2 = ExampleFactory.newInstance().copy();
 
      System.out.println("1) " +
        example1.message() + " = " + example1.plusPlus());
      System.out.println("2) " +
        example2.message() + " = " + example2.plusPlus());
      System.out.println();
 
      Thread.currentThread().sleep(3000);
    }
  }
}

The ExampleFactory class is also exactly the same, but here’s where things get leaky. Let’s introduce a new class called Leak and a corresponding interface ILeak:

1
2
3
4
5
6
7
8
9
10

interface ILeak {
}
 
public class Leak implements ILeak {
  private ILeak leak;
 
  public Leak(ILeak leak) {
    this.leak = leak;
  }
}

As you can see it’s not a terribly complicated class: it just forms a chain of objects, with each doing nothing more than holding a reference to the previous one. We will modify the Example class to include a reference to the Leak object and throw in a large array to take up memory (it represents a large cache). Let’s omit some methods shown in the previous article for brevity:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

public class Example implements IExample {
  private int counter;
  private ILeak leak;
 
  private static final long[] cache = new long[1000000];
 
  /* message(), counter(), plusPlus() impls */
 
  public ILeak leak() {
    return new Leak(leak);
  }
 
  public IExample copy(IExample example) {
    if (example != null) {
      counter = example.counter();
      leak = example.leak();
    }
    return this;
  }
}

The important things to note about Example class are:

1. Example holds a reference to Leak, but Leak has no references to Example.
2. When Example is copied (method copy() is called) a new Leak object is created holding a reference to the previous one.

If you try to run this code an OutOfMemoryError will be thrown after just a few iterations:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at example.Example.<clinit>(Example.java:8)

With the right tools, we can look deeper and see how this happens.

Post Mortem

Since Java 5.0, we’ve been able to use the jmap command line tool included in the JDK distribution to dump the heap of a running application (or for that matter even extract the Java heap from a core dump). However, since our application is crashing we will need a feature that was introduced in Java 6.0: dumping the heap on OutOfMemoryError. To do that we only need to add -XX:+HeapDumpOnOutOfMemoryError to the JVM command line:

java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid37266.hprof ...
Heap dump file created [57715044 bytes in 1.707 secs]
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at example.Example.<clinit>(Example.java:8)

After we have the heap dump we can analyze it. There are a number of tools (including jhat, a small web-based analyzer included with the JDK), but here we will use the more sophisticated Eclipse Memory Analyzer (EMA).

After loading the heap dump into the EMA we can look at the Dominator Tree analysis. It is a very useful analysis that will usually reliably identify the biggest memory consumers in the heap and what objects hold a reference to them. In our case it seems quite obvious that the Leak class is the one that consumes most of the heap:

Now let’s run a search for all of the Leak objects and see what are they holding to. To do that we run a search List objects -> with outgoing references for “example.Leak”:

The results include several Leak objects. Expanding the outgoing references we can see that each of them holds on to a separate instance of Example through a bunch of intermediate objects:

You may notice that one of the intermediate objects is ExampleFactory$1, which refers to the anonymous subclass of URLClassLoader we created in the ExampleFactory class. In fact what is happening is exactly the situation we described in the beginning of the article:

• Each Leak object is leaking. They are holding on to their classloaders
• The classloaders are holding onto the Example class they have loaded:

Conclusions

Though this example is slightly contrived, the main idea to take away is that it’s easy to leak a single object in Java. Each leak has the potential to leak the whole classloader if the application is redeployed or otherwise a new classloader is created. Since preventing such leaks is very challenging, it’s a better idea to use Eclipse Memory Analyzer and your understanding of classloaders to hunt them down after you get an OutOfMemoryError on redeploy.

This article addressed the following questions:

• How does reloading a class cause the classloader to leak?
• What are some consequences of leaking classloaders?
• What tools can be used to troubleshoot these memory leaks?

Resources

• How to fix the dreaded “java.lang.OutOfMemoryError: PermGen space” exception (classloader leaks)
• Download full source code

Other Articles in the Reloading Java Classes Series

• RJC101: Objects, Classes and ClassLoaders
• RJC201: How do Classloader leaks happen?
• RJC301: Classloaders in Web Development — Tomcat, GlassFish, OSGi, Tapestry 5 and so on
• RJC401: HotSwap and JRebel — Behind the Scenes
• RJC501: How Much Does Turnaround Cost?

We’re glad to announce the JRebel 2.1.1 release. It is a maintenance release incorporating all the bugfixes that have accumulated during the past month or so. It also includes the new Log4J plugin, support for Jetty 7 and GlassFish v3 Preview (apparently Prelude and Preview differ a lot, go figure).

Changes include:

• Support for Jetty 7
• Preliminary support for GlassFish v3 Preview
• Log4J plugin now reloads changes to log configuration on-the-fly, contributed by Julien Richard.
• Fixed an issue causing the annotations on constructors to disappear after class reload.
• Fixed an issue with Google Web Toolkit client side classes in hosted mode
• Fixed an issue with FileNotFoundException thrown by JavaRebelResourceServlet
• Fixed an issue with ClassCastException when defining web services in web.xml
• Fixed several issues in the Wicket plugin.

You can pick up the new version on our download page, by choosing the standard download.

Welcome to Turnaround article series from ZeroTurnaround.

In this article we will review how to reload a Java class using a dynamic classloader. To get there we'll see how objects, classes and classloaders are tied to each other and the process required to make changes. We begin with a bird's eye view of the problem, explains the reloading process, and then proceed to a specific example to illustrate typical problems and solutions. Other articles in the series include:

• RJC101: Objects, Classes and ClassLoaders
• RJC201: How do Classloader leaks happen?
• RJC301: Classloaders in Web Development -- Tomcat, GlassFish, OSGi, Tapestry 5 and so on
• RJC401: HotSwap and JRebel — Behind the Scenes
• RJC501: How Much Does Turnaround Cost?

A video presentation "Do you really get class loaders" by Jevgeni Kabanov

A Bird's Eye View

The first thing to understand when talking about reloading Java code is the relation between classes and objects. All Java code is associated with methods contained in classes. Simplified, you can think of a class as a collection of methods, that receive "this" as the first argument. The class with all its methods is loaded into memory and receives a unique identity. In the Java API this identity is represented by an instance of java.lang.Class that you can access using the MyObject.class expression.

Every object created gets a reference to this identity accessible through the Object.getClass() method. When a method is called on an object, the JVM consults the class reference and calls the method of that particular class. That is, when you call mo.method() (where mo is an instance of MyObject), then the JVM will call mo.getClass().getDeclaredMethod("method").invoke(mo) (this is not what the JVM actually does, but the result is the same).

Every Class object is in turn associated with its classloader (MyObject.class.getClassLoader()). The main role of the class loader is to define a class scope -- where the class is visible and where it isn't. This scoping allows for classes with the same name to exist as long as they are loaded in different classloaders. It also allows loading a newer version of the class in a different classloader.

The main problem with code reloading in Java is that although you can load a new version of a class, it will get a completely different identity and the existing objects will keep referring the previous version of the class. So when a method is called on those objects it will execute the old version of the method.

Let's assume that we load a new version of the MyObject class. Let's refer to the old version as MyObject_1 and to the new one as MyObject_2. Let's also assume that MyObject.method() returns "1" in MyObject_1 and "2" in MyObject_2. Now if mo2 is an instance of MyObject_2:

• mo.getClass() != mo2.getClass()
• mo.getClass().getDeclaredMethod("method").invoke(mo)
!= mo2.getClass().getDeclaredMethod("method").invoke(mo2)
• mo.getClass().getDeclaredMethod("method").invoke(mo2) throws a ClassCastException, because the Class identities of mo and mo2 do no match.

This means that any useful solution must create a new instance of mo2 that is an exact copy of mo and replace all references to mo with it. To understand how hard it is, remember the last time you had to change your phone number. It's easy enough to change the number itself, but then you have to make sure that everyone you know will use the new number, which is quite a hassle. It's just as difficult with objects (in fact, it's actually impossible, unless you control the object creation yourself), and we're talking about many objects that you must update at the same time.

Down and Dirty

Let's see how this would look in code. Remember, what we're trying to do here is load a newer version of a class, in a different classloader. We'll use an Example class that looks like this:

public class Example implements IExample {
  private int counter;
  public String message() {
    return "Version 1";
  }
  public int plusPlus() {
    return counter++;
  }
  public int counter() {
    return counter;
  }
}

We'll use a main() method that will loop infinitely and print out the information from the Example class. We'll also need two instances of the Example class: example1 that is created once in the beginning and example2 that is recreated on every roll of the loop:

public class Main {
  private static IExample example1;
  private static IExample example2;
 
  public static void main(String[] args)  {
    example1 = ExampleFactory.newInstance();
 
    while (true) {
      example2 = ExampleFactory.newInstance();
 
      System.out.println("1) " +
        example1.message() + " = " + example1.plusPlus());
      System.out.println("2) " +
        example2.message() + " = " + example2.plusPlus());
      System.out.println();
 
      Thread.currentThread().sleep(3000);
    }
  }
}

IExample is an interface with all the methods from Example. This is necessary because we'll be loading Example in an isolated classloader, so Main cannot use it directly (otherwise we'd get a ClassCastException).

public interface IExample {
  String message();
  int plusPlus();
}

From this example, you might be surprised to see how easy it is to create a dynamic class loader. If we remove the exception handling it boils down to this:

public class ExampleFactory {
  public static IExample newInstance() {
    URLClassLoader tmp =
      new URLClassLoader(new URL[] {getClassPath()}) {
        public Class loadClass(String name) {
          if ("example.Example".equals(name))
            return findClass(name);
          return super.loadClass(name);
        }
      };
 
    return (IExample)
      tmp.loadClass("example.Example").newInstance();
  }
}

The method getClassPath() for the purposes of this example could return the hardcoded classpath. However, in the full source code (available in the Resources section below) you can see how we can use the ClassLoader.getResource() API to automate that.

Now let's run Main.main and see the output after waiting for a few loop rolls:

1) Version 1 = 3
2) Version 1 = 0

As expected, while the counter in the first instance is updated, the second stays at "0". If we change the Example.message() method to return "Version 2". The output will change as follows:

1) Version 1 = 4
2) Version 2 = 0

As we can see, the first instance continues incrementing the counter, but uses the old version of the class to print out the version. The second instance class was updated, however all of the state is lost.

To remedy this, let's try to reconstruct the state for the second instance. To do that we can just copy it from the previous iteration.

First we add a new copy() method to Example class (and corresponding interface method):

  public IExample copy(IExample example) {
    if (example != null)
      counter = example.counter();
    return this;
  }

Next we update the line in the Main.main() method that creates the second instance:

example2 = ExampleFactory.newInstance().copy(example2);

Now waiting for a few iterations yields:

1) Version 1 = 3
2) Version 1 = 3

And changing Example.message() method to return "Version 2" yields:

1) Version 1 = 4
2) Version 2 = 4

As you can see even though it's possible for the end user to see that the second instance is updated and all its state is preserved, it involves managing that state by hand. Unfortunately, there is no way in the Java API to just update the class of an existing object or even reliably copy its state, so we will always have to resort to complicated workarounds.

In subsequent articles we'll review how web containers, OSGi, Tapestry 5, Grails and others confront the problem of managing state when reloading classes, then we'll dig into how HotSwap, Dynamic Languages, and the Instrumentation API work, and go behind the scenes with JRebel as well.

Resources

• Internals of Java Class Loading
• Download full source code

Other Articles in the Reloading Java Classes Series

• RJC101: Objects, Classes and ClassLoaders
• RJC201: How do Classloader leaks happen?
• RJC301: Classloaders in Web Development -- Tomcat, GlassFish, OSGi, Tapestry 5 and so on
• RJC401: HotSwap and JRebel — Behind the Scenes
• RJC501: How Much Does Turnaround Cost?

On September 30th, 2009, Apache released Struts 2.1.8 for general availability.  Though we couldn’t find much info on the differences between 2.1.6 and 2.1.8, here’s what Musachy Barroso said about “Why web developers should choose Struts 2″, in his interview on InfoQ.

“Struts 2 is probably the most loosely-coupled framework available. Out of the box, many features are usable with little or no customization and it is easy to learn. The same knowledge can then be applied to add plugins to override default behaviors. The loose coupling also allows business logic to be written with no knowledge of the existence of Struts. Despite this, Struts scales up really well and is currently powering some very high-traffic sites.”

We like the evolution of Struts, and wanted to do our part to help minimize the time between writing Struts code & seeing the changes, so with the release of JRebel 2.1, we are proud to present extended support for Struts 1.x & 2.x, with full support for reloading action mappings. Combined with JRebel’s previously released features (skipping redeploys by reloading changes to Java classes in the running application, and skipping builds by mapping the project workspace to the deployed WARs or EARs), JRebel is now an even more potent time-saving tool for Struts 1 and Struts 2 users.

See the screencast for a demonstration of coding with Struts 2 and JRebel:

If you’re not familiar with JRebel, catch up in under 3 mins with this screencast, which shows other JRebel features, including support for the Spring framework (or take a look at the feature list):

We’re glad to see the success of the Struts 2 framework, and happy to support the community.  Did you know that on average there are more than 2 million project downloads of Struts per month, since March of 2008?

Some time ago we ran a survey asking a few questions about the build process, specifically the tools that are used to do incremental builds and how much time those builds take. We had over 600 responses, so now it’s time to count the results.

This is the first time that we’ve published results on the incremental build process, so the information is more likely to serve as a guide than an authoritative information source. That being said, the information is still quite interesting, and if it serves to start a conversation that improves the process of even one team, then we’re proud to have helped out. If you haven’t answered the 3-question survey yet, take two minutes and go for it – and do let your community know about it – as more answers trickle in we’ll update this post with the new data. If you’d like to play with the results on your own we‘ve provided all the data and our calculations in a handy Excel sheet that you can download here.

(more…)

While finishing the polish on the 2.1 release we were also preparing for you a glance at what’s to come in the longer term. Without further ado, allow us to introduce the JRebel 3.0 M1 release features:

• Support for adding static fields and changing enums. Previously when you added a static field to your class you’d see a discouraging warning in the console and get an exception when trying to access the field. Now JRebel will happily print “Reinitialized class” and your application will continue working as if nothing happened. To top it off you can also now change Java 1.5 enums in any way you like and they should just continue working. This feature is still a tad experimental, so please help us out to smooth it out!
• Full JSP <scriptlet> support. Now when you change your Java code (e.g. add a method) it can be immediately used in the Java code snippets in the JSP.
• 25%-30% less memory use. It is not uncommon to need a lot of PermGen heap with JRebel enabled. Now we optimized the memory use and will continue to drive it down in the upcoming milestones.
• No more classloader leaks. JRebel no longer holds any references to class loaders a second longer than necessary. This should help to get less OutOfMemoryErrors on application redeploys with JRebel enabled.

There are also quite a few changes to the API that will enabled more features in the upcoming milestones.

Note that since we don’t charge for upgrades, it is not our goal to put all the cool features in the 3.0 release. Rather we’ll continue to add features to the 2.x branch (expect at least a 2.2 release this year) and only put large or risky changes that require a lot of testing or feedback in the 3.x branch.

Download the release.

We are proud to present JRebel 2.1, the “Struts” edition. The main features of this release are the reworked Struts 2.x plugin and the brand new Struts 1.x plugin that reload changes to Struts action mappings on-the-fly both from XML and Java 5 annotations. Developing Struts applications with JRebel is now easier than ever, as no restarts are necessary anymore.

This release also includes support for GlassFish v3 and the Felix OSGi container it is based on. According to this survey developers spend 13% of development time or 4.3 full-time weeks every year redeploying on the GlassFish v2 container. GlassFish v3 boasts improved startup time, but with JRebel you can take the cost of making a change down to zero.

Starting with this release JRebel will report some anonymous usage statistics to our servers (including jvm name and version, container name and version, frameworks you use and redeploy stats). You can see all of the data in the jrebel.info text file created next to jrebel.jar This data will be used to help us prioritize development, but if you wouldn’t like to send it please add -Drebel.usage_reporting=false to the JVM command line.

Finally this release includes a multitude of fixes that were found since the 2.0.3 release. A lot of them concern the Spring plugin, though a few bugs were also found in the JRebel core.

BIG NOTE: We have renamed “javarebel.jar” to “jrebel.jar” and you’ll have to update your installation command line accordingly.

Proceed to download or view the full changelog.

Older Entries »