Imagine a bacon-wrapped Ferrari. Still not better than our free technical reports.
See all our reports

Don’t Test Blindly: The Right Methods for Unit Testing Your Java Apps

Quick-tro

In my previous blog post Why Your Next Cloud App Will Probably Suck Without…Unit Testing, we gave an overview promoting the benefits of unit tests, and how these small, simple actions can make a big difference in getting bug-free, solidly-built applications out the door.

After reading this, I can assume that you have jumped out of your chair and started frantically writing them for your big enterprise project, right? Well, stop now!

Before writing a single unit test, it’s necessary to determine exactly what to test. A big enterprise application can hold a billion lines of code, so do you think it’s realistic to write tests for everything? Not exactly. So, it’s important to make sure your tests are focused on what actually matters to you, your app and your users.

What you should NOT test

You can spend your entire life testing away if you like, but it’s probably best to to make some assumptions in the beginning to save yourself some time. After all, if you end on testing your own database all the time, then you’ve got bigger problems than error-free app deployment. So before explaining where you should focus your unit testing efforts, let’s see what you should NOT test and get it out of the way first.

I’ll ask you to trust my 8 years in software development when I say you should NOT write units tests for:

  1. Other framework libraries (you should assume they work correctly)
  2. The database (you should assume it works correctly when it is available)
  3. Other external resources (again you assume they work correctly when available)
  4. Really trivial code (like getters and setters for example)
  5. Code that has non deterministic results (Think Thread order or random numbers)
  6. Code that deals only with UI (e.g. Swing toolkit, Wicket)

Funny story: When I was just starting to get into code and learned about unit tests, the first thing I did was write a test that saved an Entity on the database using Hibernate, read it back and then verified that it is the same. Turns out that writing unit tests against your own database or Hibernate is not very productive. Awesome :-/

What you SHOULD test

One of the golden rules of unit testing is that your tests should cover code with “business logic”. Here is the typical flow in a back-end processing system. Your web application could look like this:

YourAppV2
In this case, the highlighted part in gold is where you should focus your testing efforts. This is the part of the code where usually most bugs manifest. It is also the part that changes a lot as user requirements change since it is specific to your application.

So what happens if you get across a legacy application with no unit tests? What if the “business logic” part ends up being thousands of lines of code? Where do you start?

In this case you should prioritize things a bit and just write tests for the following:

  1. Core code that is accessed by a lot of other modules
  2. Code that seems to gather a lot of bugs
  3. Code that changes by multiple different developers (often to accommodate new requirements)

How much of the code in these areas should we test, you might ask. Well, now that we know which areas to focus on, we can now start to analyze just how much testing we need to feel confident about our code.

Code coverage: How much is enough/too much?

A useful metric that shows what part of your code is “touched” by unit tests is Code Coverage. Code coverage is the percentage that shows to what depth your internal checks affect your Java classes. To obtain this metric, you can run one of multiple tools available for this purpose.

For you Eclipse users out there, an easy way to find code coverage is to install the Eclemma plugin from the Eclipse Marketplace. In our previous post on unit tests, we ran tests on a simple calculation for determining the weight of basket or shopping cart for an e-shop, so we will continue with this example in this post.

With Eclemma, right-click on your unit test as shown before in our previous post on unit tests, but this time choose the “Coverage as” option.

After the unit test runs properly you will see these results:

full-coverage

As you can see, for the BasketWeightCalculator class we reach 100% coverage. If we select only one test and run the coverage again we get a different percentage:

half-coverage

The image shows that we are missing the “if” statement in the code, the red color indicating that tests on this part were not run at all. The part in yellow is for code that was run and tested for one condition, but not for the other.

Determining the right amount of code coverage

Getting a 100% code coverage on the whole application is a bit unrealistic. A more realistic example is 60%-70%, assuming that this is your business logic code; however, depending on the kind of application, even that high of a % might be too time consuming.

I tend to follow the Pareto principle as a good rule of thumb (the 80-20 rule) so that I can be sure to get code coverage of at least 20%. This 20% should be the “heart” of your application, e.g. code that changes often, breaks often and is a dependency to most other system components.

I can hear some of the more senior developers out there saying, “Aren’t you going to mention cyclomatic complexity as a metric for understanding how to write easier tests?” But this extends into other camps as well, so I’d prefer to cover that in another post on another day.

Learn which tools rock!

As an additional section, I wanted to share with you a few advanced unit testing techniques that I’ve relied on a lot in the past. In our previous post on unit testing, we introduced JUnit and showed its basic capabilities.

JUnit luckily has several helper annotations that can cut down a lot on the amount of code you have to write. (N.B. if your application still uses JUnit 3.x, upgrade to JUnit 4.x NOW. The new features will save you a lot of time in the long run.)

Assume that you have already implemented a class that checks the validity for URLs called MyUriValidator. The class needs several statements to set up correctly. So your first attempt for a  unit test might be:

public class MyUriValidatorTest {
 
	@Test
	public void test1() {
		MyUriValidator myValidator = new MyUriValidator();
		myValidator.allowFileUrls(true);
		myValidator.allowInternationlizedDomains(false);
		myValidator.allowReservedDomains(false);
		myValidator.allowCustomPorts(true);
 
		assertTrue("Domain is valid",myValidator.isValidUrl("http://www.google.com"));
	}
 
	@Test
	public void test2() {
		MyUriValidator myValidator = new MyUriValidator();
		myValidator.allowFileUrls(true);
		myValidator.allowInternationlizedDomains(false);
		myValidator.allowReservedDomains(false);
		myValidator.allowCustomPorts(true);
 
		assertTrue("Domain is valid",myValidator.isValidUrl("file://home/users"));
	}
 
	@Test
	public void test3() {
		MyUriValidator myValidator = new MyUriValidator();
		myValidator.allowFileUrls(true);
		myValidator.allowInternationlizedDomains(false);
		myValidator.allowReservedDomains(false);
		myValidator.allowCustomPorts(true);
 
		assertFalse("Domain is invalid",myValidator.isValidUrl("http://localhost:8080/"));
	}

There is clearly a lot of code of duplication here. JUnit has a @Before annotation for code that runs automatically before each test method. So your test can be simplified to:

public class MyUriValidatorTest {
 
	private MyUriValidator myValidator = null;
 
	@Before
	public void beforeEachTest() { //Name of method does not actually matter
		myValidator = new MyUriValidator();
		myValidator.allowFileUrls(true);
		myValidator.allowInternationlizedDomains(false);
		myValidator.allowReservedDomains(false);
		myValidator.allowCustomPorts(true);
	}
 
	@Test
	public void test1() {
		assertTrue("Domain is valid",
				myValidator.isValidUrl("http://www.google.com"));
	}
 
	@Test
	public void test2() {
		assertTrue("Domain is valid",
				myValidator.isValidUrl("file://home/users"));
	}
 
	@Test
	public void test3() {
		assertFalse("Domain is invalid",
				myValidator.isValidUrl("http://localhost:8080/"));
	}
 
}

There is also the respective @After annotation (runs after each test), as well as @BeforeClass and @AfterClass annotations for code that runs ONCE before/after all tests.

So now we have an improved version. But even so, we have to create a new method each time a new URL needs to be tested. JUnit supports parameterised tests where you write a general test method once and then there is a separate method that provides the data.

Here is this approach for that:

@RunWith(Parameterized.class)
public class MyUriValidatorTest {
 
	private MyUriValidator myValidator = null;
	private String uriTestedNow =null;
	private boolean expectedResult = false;
 
	public MyUriValidatorTest(String uriTestedNow,boolean expectedResult)
	{
		this.uriTestedNow = uriTestedNow;
		this.expectedResult = expectedResult;
	}
 
	@Parameters
        public static Collection data() {
    	/* First element is the URI, second is the expected result */
    	 List uriToBeTested =  Arrays.asList(new Object[][] {
                            {  "http://www.google.com", true }, 
                            	{ "file://home/users", true }, 
                         	{ "http://staging:8080/sample", true },
                            	{"http://localhost:8080/", false }  });
 
    	 return uriToBeTested;
    }
 
	@Before
	public void beforeEachTest() {
		myValidator = new MyUriValidator();
		myValidator.allowFileUrls(true);
		myValidator.allowInternationlizedDomains(false);
		myValidator.allowReservedDomains(false);
		myValidator.allowCustomPorts(true);
	}
 
	@Test
	public void testCurrentUri() {
		assertEquals("Testing for "+uriTestedNow, expectedResult,myValidator.isValidUrl(uriTestedNow));
	}
 
}

As you can see, adding a new URL is a single line change in the the method annotated as Parameters. You should examine all the capabilities of JUnit as contained in the WIKI.
Be sure not to miss Exception Testing, Theories, Rules and Categories.

For even more advanced capabilities check out TestNG which is a lot more advanced than JUnit. In fact JUnit is playing catch-up and some features of the latest versions are stolen/inspired (take your pick) from TestNG.

Closing remarks…

In this post we offered you some hints on which code you should test and which you should ignore. We also introduced the notion of code coverage and gave our recommendation for how much of the total code you should deal with. From the final example you should also see that investing some time in learning how to properly utilize unit testing features with JUnit, or if you feel more adventurous, TestNG, is highly recommended.

In future posts, we will look at how to write testable code (so that the unit tests you prepare to cover it are easier to write), the concept of mocking and when we need to use it. Thanks for tuning in–feel free to leave comments below, or reach me at kostis.kapelonis@zeroturnaround.com.

  • tomek

    Why no unit tests for ui? Especially in wicket as its great framework for TDD. Just don’t tell me you prefer writing selenium tests :-)

  • Kostis Kapelonis

    Hello Tomek. In my experience unit tests for UI are not very effective since changes to the UI can be very frequent and from unrelated parties (a.k.a. the bicycle shed effect).

    Also my main point is that failing unit tests are a serious warning signal that need immediate action. A failing unit test in the UI means nothing. Maybe a button has been moved, maybe a screen has been altered, maybe the graphic designer did something fancy.

    Also bugs on UI are usually very easy to spot and fix, so it is a bit hard to have regressions on this part. This is because they are instantly recognizable (and not hidden like back-end bugs)

    If you have already covered your business logic, then by all means feel free to write tests for UI as well. However for most organizations my advice is to focus on back-end code.

  • tomek

    Hi Kostis,
    In wicket if your designer changes the hierarchy of html your page/panel/component it will simply throw RuntimeException, which would require immediate action. So at least have a unit test which just renders the panel without testing what all buttons/fields are doing.

    Btw like your article but was interested to see the reason behind not testing ui… especially after you mention wicket.

  • Dmitriy

    Well, don’t really understood what did you mean with the point 2, but I simply can’t agree with the last point (but only wicket :)).

    If you have a set of complex forms/fields with ajax handling and with some dependencies among them it is not so simple “to spot” a bug or to predict where the problem can appear. Every time you will be forced to start your app and to go through complete application just to verify that everything is well… ZeroTurnaround??? Not really…

    In Wicket you just start your test-suite and see the result of your change. ZeroTurnaround !!!

    “A failing unit test in the UI means nothing” Just imagine in a e-commerce suite the “Buy” button was disappeared :)

  • Madhusudana Reddy

    Hi Kostis,
    I don’t think I completely agree with the statement “unit tests, the first thing I did was write a test that saved an Entity on the database using Hibernate, read it back and then verified that it is the same.”. I see it as more of an integration test rather than an unit test. In the unit test we would just verify whether entityManager.persist/save is getting called or not which is still a valid scenario and is worth unit testing. And in this particular case, in my view, this integration test is ensuring that the Hibernate entity mapping is inline or is compatible with the underlying database table schema. IMO, we would see the benefit of these kind of this integration tests in a long run. For ex: when the underlying table schema gets modified and these integration test detect the failure early in the development.

  • Kostis Kapelonis

    Hello. You are right, so this is exactly why I wrote “you should NOT write UNIT tests for”. Reading and writing to a database is not a unit test. It is indeed an integration test.

    Integration tests will be covered in a later post. In the present post I am still talking about unit tests only.

  • Kostis Kapelonis

    Assume that I go to work to a legacy application that has already 500 unit tests. Some of them are with the UI where they check labels of buttons or positions of button. I haven’t written these unit tests. I found them there already.Everything is green and I am happy.

    As I work and do refactoring in business logic code when I see a red bar I find the regression and I fix it. Suddenly marketing comes in and says that all buttons named “Go to cart” become “See my cart”. Somebody commits the change and I now have 125 failing unit tests. These checked the presence of labels in the screens.

    Now I am doomed. I have to stop refactoring because the build server won’t even built because the failing tests. (which are completely unrelated to business logic)

    These are the GUI unit tests that are problematic. These are the ones I am talking about.
    When the build fails I want it to fail for some serious reasons. With GUI tests you are never sure. Sometimes the reason is important (“the buy” button has disappeared) and sometimes is in trivial (the “buy” button has changed label).

  • http://twitter.com/ndeverge Nicolas Deverge

    Hello,

    the JUnit wiki links are broken, it seems that the github repo has moved : https://github.com/junit-team/junit

    Cheers,
    Nicolas

  • Kostis Kapelonis

    Thanks! Should be fixed now…

  • Josh Ribakoff

    Sounds like a management issue not a testing issue