Software QA FYI - SQAFYI

JUnit 4.x Howto

By: Blaine Simpson

Chapter 1. Introduction
What is JUnit
JUnit is a program used to perform unit testing of virtually any software. JUnit testing is accomplished by writing test cases using Java, compiling these test cases and running the resultant classes with a JUnit Test Runner. I will explain a teensy bit about software and unit testing in general. What and how much to test depends on how important it is to know that the tested software works right, in relation to how much time you are willing to dedicate to testing. Since you are reading this, then for some reason you have decided to dedicate at least some time to unit testing. As an example, I'm currently developing an SMTP server. An SMTP server needs to handle email addresses of the format specified in RFC documents. To be confident that my SMTP server complies with the RFC, I need to write a test case for each allowable email address type specified in the RFC. If the RFC says that the character "#" is prohibited, then I should write a test case that sends an address with "#" and verifies that the SMTP server rejects it. You don't need to have a formal specification document to work from. If I wasn't aiming for compliance, I could just decide that it's good enough if my server accepts email addresses consisting only of letters, numbers and "@", without caring whether other addresses succeed or fail.

Another important point to understand is that, everything is better if you make your tests before you implement the features. For my SMTP server, I did some prototyping first to make sure that my design will work, but I did not attempt to satisfy my requirements with that prototype. I am now writing up my test cases and running them against existing SMTP server implementations (like Sendmail). When I get to the implementation stage, my work will be extremely well defined. Once my implementation satisfies all of the unit tests, just like Sendmail does, then my implementation will be completed. At that point, I'll start working on new requirements to surpass the abilities of Sendmail and will work up those tests as much as possible before starting the second implementation iteration. (If you don't have an available test target, then there sometimes comes a point where the work involved in making a dummy test target isn't worth the trouble... so you just develop the test cases as far as you can until your application is ready). Everything I've said so far has to do with unit testing, not JUnit specifically. Now on to JUnit...

Obtaining and Installing JUnit
There is very little to say about installing JUnit. All you really need from the distribution is the jar file named like junit-4.5.jar. There's a good chance that you already have this jar bundled with an IDE or a Java project that you work with. A typical user will not need any of the other jars (which include source code, or lack a third-party library which you probably do need). The only reason to permanently install the distribution is if you want a local copy of the documentation which is readily available online at http://junit.sourceforge.net/#Documentation .

Test Coverage -- quantity of test cases
With respect to test coverage, an ideal test would have test cases for every conceivable variation type of input, not every possible instance of input. You should have test cases for combinations or permutations of all variation types of the implemented operations. You use your knowledge of the method implementation (including possible implementation changes which you want to allow for) to narrow the quantity of test cases way down.

I'll discuss testing the multiplication method of a calculator class as an example. First off, if your multiplcation method hands its parameters (the multiplication operands) exactly the same regardless of order (now and in the future), then you have just dramatically cut your work from covering all permutations to covering all combinations. You will need to cover behavior of multiplying by zero, but it will take only a few test cases for this. It is extremely unlikely that the multiplication method code does anything differently for an operand value of 2 different than it would for an operand value of 3, therefore, there is no reason to have a test for "2 x 0" and for "3 x 0". All you need is one test with a positive integer operand and zero, like "46213 x 0". (Two test cases if your implementation is dependent upon input parameter sequence). You may or may not need additional cases to test other types of non-zero operands, and for the case of "0 x 0". Whether a test case is needed just depends upon whether your current and future code could ever behave differently for that case.

Chapter 2. Test Expressions
The most important thing is to write tests like
(expectedResult == obtainedResult)
or
expectedResult.equals(obtainedResult)
You can do that without JUnit, of course. For the purpose of this document, I will call the smallest unit of testing, like the expression above, a test expression . All that JUnit does is give you flexibility in grouping your test expressions and convenient ways to capture and/or summarize test failures.

Chapter 3. Class org.junit.Assert
JUnit-specific Test Expressions
The Assert class has a bunch of static convenience methods to help you with individual test expressions. For example,
without JUnit I might code
if (obtainedResult == null || !expectedResult.equals(obtainedResult))
throw new MyTestThrowable("Bad output for # attempt");
With JUnit, you can code
import static org.junit.Assert.*;
...
assertEquals("Bad output for # attempt", expectedResults, obtainedResults);
The static import statement is not required. It just allows you to code more concisely with assert... instead
of Assert.assert.... Every JUnit-supplied assert method throws a java.lang.AssertionErrors when
the test fails.
There are lots of assert* methods to take care of the several common Java types. They take care of checking
for nulls. Both assert() failures and failure()s (which are effectively the same thing) throw
java.lang.AssertionError Throwables, which the test-runners and listeners know what to do with. If a test expression fails, the containing test method quits (by virtue of throwing the AssertionError internally), then the test-runner performs cleanup before executing the next test method in the sequence (with the error being noted for reporting now or later). Take a look at the API spec for org.junit.Assert [http:// junit.sourceforge.net/javadoc_40/org/junit/Assert.html] .
If Assert doesn't supply a specific expression test which you need, then code up your own expression and use Assert.assertTrue() or Assert.assertFalse() on the boolean result of your expression. Or you can just call one of the Assert.fail() methods once you determine a failure case is at hand. If you want to generalize custom failure recognition, you can make your own utility methods or catch blocks which generate and throw a new AssertionError. In this case, you will often want to run initCause() on the AssertionError instance before throwing it, to indicate the original source location of the failure.

Chapter 4. Test Methods
Composition and Execution of Test Methods
The smallest groupings of test expressions are the methods that you put them in. Whether you use JUnit or not, you need to put your test expressions into Java methods, so you might as well group the expressions, according to any criteria you want, into methods. If you designate such a test method with the "@org.junit.Test" annotation, then JUnit will execute it as explained in the Test Classes and the JUnit Test Execution Environment chapter (and that is the ultimate purpose of doing so). For this document, I call such a method a test method.
The general recommendation is to test only one aspect of behavior in a method. (I know how vague this is). If that can be done with a single expression, then your test method should contain just one expression. There are two main reasons for keeping test methods very fine-grained.
First, your report output is only as granular as the test methods. If you have 10 methods each of which tests handling of a different type of input character, your test report will have a count (at least) for the success or failure of each character type. If you put the expressions to test all ten character types into a single test method, you will only get a single test result for all character types. The difference is drastic. If your first test fails, none of the other tests will even execute in the single-method scenario.
Secondly, setup and execution of one test may cause unintended consequences for a test which follows. This risk is usually acceptable for a single-author class, but for jointly developed software, or test classes which will outlast the tenure of the original author, the risk can be considerable.
With JUnit version 4, we implement test methods as follows.
Example 4.1. Implementation of a JUnit Test Method
@org.junit.Test
public void testDoubler() {
TargetClass targetObject = new TargetClass();
targetObject.init(); // Any setup needed to prepare the target object
assertEquals("Target object failed to double 'xyz'",
"xyzxyz", targetObject.double("xyz"));
}

There are several important observations to be made here. It must be an instance method of type void with no parameters.
Unlike JUnit 3.x, the name of the method doesn't matter to the JUnit 4.x test-runners (or to anything else).
So, you can implement your own method naming convention. I didn't prefix the method name with "test" to comply with any JUnit rule, but because it makes the methods easy to recognize (both for human readers, and for scripts, Ant build files, etc.). Your test methods are allowed to throw anything by the rules of JUnit, but a good test method won't declare any Throwables (more on this topic below).

Test methods are single-threaded, but different test methods must be independent and can't depend on state (including values of instance or static variables) set by another test method. If you are going to make use of instance variables, you must take pains to initialize the variable values for each test method run. The following chapter explains how the test-runners set up and execute your test methods, and how you can write shared setup and shut down code. Handling Throwables in your Test Methods

Caution
Be careful when catching Throwables or Errors in your test methods. If you catch java.lang.AssertionError and don't rethrow it, you will have intercepted JUnit's test failure indication.

Full article...


Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/

JUnit 4.x Howto