Software QA FYI - SQAFYI

Test Categorization Techniques with TestNG

By: Andy Glover

TestNG is an annotation-based testing framework, which draws on some of the shortcomings of JUnit by adding features such as flexible fixtures, test categorization, parametric testing, and dependent methods, to name a few. What's more, TestNG runs on both Java 5.0 (via annotations) and 1.4 (via JavaDoc style comments). Through TestNG's ability to easily categorize developer tests into unit, component, and system groups, build times can be kept manageable. Via the use of the groups annotation and multiple Ant tasks, test groups can be run at varying frequencies whether on a workstation or in a continuous integration environment.

This article examines the concept of test categories and demonstrates how to incorporate TestNG's groups annotation tied with flexible fixtures to facilitate running tests at different frequencies via specific Ant targets. It assumes knowledge of TestNG. TestNG Groups Explored

TestNG supports an intuitive mechanism for grouping test classes and associated test methods. At its most basic level, the grouping feature of TestNG is enabled via the groups parameter of the test annotation, which can be attached to classes or individual methods. As the name implies, a class or individual method can belong to 1...n groups.

For example, the following class contains two public methods, which by default are labeled as tests and further categorized as belonging to group one: 


/**
 * @testng.test groups="one"
 */
public class SimpleGroupedTest {
 private String fixture;
 /**
  * @testng.before-class = "true" 
  */
 private void init(){
  this.fixture = "Hello World";
 }
 
 public void verifyEquality(){
  Assert.assertEquals("Hello World", this.fixture);
 }    
 
 public void verifySame(){
  String value = this.fixture;
  Assert.assertSame(this.fixture, value);
 }
}

In contrast, the next class defines two test methods. However, one method belongs to two different groups—one and two. Accordingly, any associated fixture logic must be associated with a desired group. In this case, the init() method must be configured to run before either group one or two is executed. 


public class SimpleGroupedTwoTest {
 private String fixture;
 /**
  * @testng.before-class = "true"  \
  *    groups = "one, two"
  */
 private void init(){
  this.fixture = "Hello World";
 }
 /**
  * @testng.test groups="one, two"
  */
 public void verifyEqualityAgain(){
  Assert.assertEquals(this.fixture, "Hello World");
 }    
 /**
  * @testng.test groups="two"
  */
 public void verifySameAgain(){
  String value = this.fixture;
  Assert.assertSame(value, this.fixture);
 }
}

TestNG supports running desired groups in a number of ways, ranging from specifying them via the TestNG Eclipse plug-in to listing them in the TestNG Ant task.

Introducing Test Categorization

Performing a build (that is, compiling source code and executing tests) is one of the easiest ways to verify working software; therefore, it should come as no surprise that long-running builds are a drag on developer productivity. There are few annoyances that are worse than having to wait for an excessively long build to complete. Encountering an unexpectedly blue screen of death reboot in the middle of coding ranks up there too, but at least we can easily do something about long builds.

The test execution step is almost always the reason for long builds (unless the build compiles millions and millions of .java files). Plus, the aggregate time to execute a test suite has the tendency to lengthen when there are extensive set-up steps, such as configuring a database or deploying an EAR file, to name a few. Therefore, crafting a test categorization strategy and running categories at prescribed intervals will facilitate manageable build durations.

Categorizing tests, however, requires that we define specific categories, which means refining the generic term unit tests. Unit testing, as it turns out, is just one piece in a three-sliced pie, with the two remaining pieces being component tests and system tests. The next section examines the various types of tests that are typically written by developers, such as unit tests, component tests, and system tests. Subsequently, these test types will be implemented in TestNG and integrated into an Ant build script. Unit Tests Defined

Unit tests verify the behavior of isolated objects; however, due to class coupling, they can also verify the behavior of related objects. For example, the following unit test, which verifies object identity and is implemented in TestNG, solely focuses on one type: PartOfSpeechEnum. 

/**
 * @testng.test 
 */
public class PartOfSpeechEnumTest {
    
 public void verifyNotEquals() throws Exception{
  assert PartOfSpeechEnum.ADJECTIVE != 
     PartOfSpeechEnum.NOUN: "NOUN == ADJECTIVE?"; 
 }

 public void verifyEquals() throws Exception{
  assert PartOfSpeechEnum.ADJECTIVE == 
    PartOfSpeechEnum.ADJECTIVE "ADJECTIVE != ADJECTIVE";
 }
}

Sometimes, a unit test verifies the behavior of more 
than one object. These objects, however, have few other
outside dependencies. For example, the following test
verifies two objects: Dependency and DependencyFinder.

//imports removed...

public class DependencyFinderTest {
 private DependencyFinder finder;
 /**
  * @testng.test
  */
 public void verifyFindDependencies() throws Exception{
  String targetClss  = "test.com.sedona.frmwrk.dep.Finder";
  Filter[] filtr = new Filter[] { 
    new RegexPackageFilter("java|org")};
  Dependency[] deps = 
   finder.findDependencies(targetClss, filtr);
  Assert.assertNotNull(deps, "deps was null");
  Assert.assertEquals(deps.length, 5, "should be 5 large");
 }
 /**
  * @testng.before-class = "true" 
  */
 protected void init() throws Exception {        
  this.finder = new DependencyFinder();
 }
}

The key thing to remember is that unit tests do not rely on outside dependencies such as databases, which have the tendency to increase the amount of time it takes to set up and run tests. Unit tests do not have a configuration cost (measured in terms of time), and the resource consumption to run them is negligible.

Because unit tests run so quickly, they should be run any time a build is run, including in continuous integration environments, where builds are usually run if the source repository (such as CVS) changes.

Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/

Test Categorization Techniques with TestNG