Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Mastering Software Testing with JUnit 5

You're reading from   Mastering Software Testing with JUnit 5 Comprehensive guide to develop high quality Java applications

Arrow left icon
Product type Paperback
Published in Oct 2017
Publisher Packt
ISBN-13 9781787285736
Length 350 pages
Edition 1st Edition
Languages
Tools
Arrow right icon
Author (1):
Arrow left icon
Boni Garcia Boni Garcia
Author Profile Icon Boni Garcia
Boni Garcia
Arrow right icon
View More author details
Toc

Testing frameworks for the JVM

JUnit is a testing framework which allows to create automated tests. The development of JUnit was started by Kent Beck and Erich Gamma in late 1995. Since then, the popularity of the framework has been growing. Nowadays, it is broadly considered as the de facto standard for testing Java applications.

JUnit was designed to be a unit-testing framework. Nevertheless, it can be used to implement not just unit tests, but also other kinds of tests. As we will discover in the body of this book, depending on how the test logic exercises the piece of software under test, a test case implemented with JUnit can be considered as an unit, integration, system, and even acceptance test. All in all, we can think of JUnit as a multi-purpose testing framework for Java.

JUnit 3

Since the early versions of JUnit 3, the framework can work with Java 2 and higher. JUnit3 is open source software, released under Common Public License (CPL) Version 1.0 and hosted on SourceForge (https://sourceforge.net/projects/junit/). The latest version of JUnit 3 was JUnit 3.8.2, released on May 14, 2007. The main requirements introduced by JUnit in the world of testing frameworks were the following:

  1. It should be easy to define which tests will run.
  2. The framework should be able to run tests independently of all other tests.
  3. The framework should detect and report errors test by test.

Standard tests in JUnit 3

In JUnit 3, in order to create test cases, we need to extend the class junit.framework.TestCase. This base class includes the framework code that JUnit needs to automatically run the tests. Then, we simply make sure that the method name follows the testXXX() pattern. This naming convention makes it clear to the framework that the method is a unit test and that it can be run automatically.

The test life cycle is controlled in the setup() and tearDown()methods. The TestCase calls setup() before running each of its tests and then calls teardown() when each test is complete. One reason to put more than one test method into the same test case is to share the same test fixture.

Finally, in order to implement the verification stage in the test case, JUnit 3 defines several assert methods in a utility class named junit.framework.Assert. The following table summarizes the main assertions provided by this class:

Method Description
assertTrue Asserts that a condition is true. If it isn’t, the method throws an AssertionFailedError with the given message (if any).
assertFalse Asserts that a condition is false. If it isn’t, the method throws an AssertionFailedError with the given message (if any).
assertEquals Asserts that two objects are equal. If they are not, the method throws an AssertionFailedError with the given message (if any).
assertNotNull Asserts that an object is not null. If it is, the method throws an AssertionFailedError with the message (if any).
assertNull Asserts that an object is null. If it isn’t, the method throws an AssertionFailedError with the given message (if any).
assertSame Asserts that two objects refer to the same object. If they do not, the method throws an AssertionFailedError with the given message (if any).
assertNotSame Asserts that two objects do not refer to the same object. If they do, the method throws an AssertionFailedError with the given message (if any).
fail Fails a test (throwing AssertionFailedError) with the given message (if any).

The following class shows a simple test implemented with JUnit 3.8.2. As we can see, this test case contains two tests. Before each test, the method setUp() will be invoked by the framework, and after the execution of each test, the method tearDown() will be also invoked. This example has been coded so that the first test, named testSuccess() finishes correctly, and the second test named testFailure() ends with an error (the assertion throws an exception):

package io.github.bonigarcia;

import junit.framework.TestCase;

public class TestSimple extends TestCase {

// Phase 1: Setup (for each test)
protected void setUp() throws Exception {
System.out.println("<Setup>");
}

// Test 1: This test is going to succeed
public void testSuccess() {
// Phase 2: Simulation of exercise
int expected = 60;
int real = 60;
System.out.println("** Test 1 **");

// Phase 3: Verify
assertEquals(expected + " should be equals to "
+ real, expected, real);
}

// Test 2: This test is going to fail
public void testFailure() {
// Phase 2: Simulation of exercise
int expected = 60;
int real = 20;
System.out.println("** Test 2 **");

// Phase 3: Verify
assertEquals(expected + " should be equals to "
+ real, expected, real);
}

// Phase 4: Teardown (for each test)
protected void tearDown() throws Exception {
System.out.println("</Ending>");
}

}
All the code examples explained in this book are available on the GitHub repository https://github.com/bonigarcia/mastering-junit5.

Test execution in JUnit 3

JUnit 3 allows to run test cases by means of Java applications called test runners. JUnit 3.8.2 provides three different test runners out of the box: two graphical (Swing and AWT based) and one textual that can be used from the command line. The JUnit framework provides separate class loaders for each test, in order to avoid side effects among tests.

It is a common practice that build tools (such as Ant or Maven) and Integrated Development Environments -IDE- (such as Eclipse and IntelliJ) implement its own JUnit test runner.

The following image shows what the previous test looks like when we use the JUnit Swing runner, and also when we use Eclipse to run the same test case.

Execution of an JUnit 3 test case using the graphical Swing test runner and also with the Eclipse test runner

When a test is not succeeded in JUnit, it can be for two reasons: a failure or an error. On the one hand, a failure is caused by an assertion (Assert class) which is not meet. On the other hand, an error is an unexpected condition not expected by the test, such as a conventional exception in the software under test.

Another important contribution of JUnit 3 is the concept of the test suite, which is a convenient way to group tests that are related. Test suites are implemented by means of the JUnit class junit.framework.TestSuite. This class, in the same way as TestCase, implements the framework interface junit.framework.Test.

A diagram containing the main classes and methods of JUnit 3 is depicted as follows:

Core JUnit 3 classes

The following snippet shows an example of the use of test suites in JUnit 3. In short, we can create a group of tests simply instantiating a TestSuite object, and then add single test cases using the method addTestSuite():

package io.github.bonigarcia;

import junit.framework.Test;
import junit.framework.TestSuite;

public class TestAll {

public static Test suite() {
TestSuite suite = new TestSuite("All tests");
suite.addTestSuite(TestSimple.class);
suite.addTestSuite(TestMinimal.class);
return suite;
}
}

This test suite can be later executed using a test runner. For example, we could use the command-line test runner (junit.textui.TestRunner) and the command line, as follows:

Test suite executed using the textual test runner and the command line

JUnit 4

JUnit 4 is still an open source framework, though the license changed with respect to JUnit 3, from CPL to Eclipse Public License (EPL) Version 1.0. The source code of JUnit 4 is hosted on GitHub (https://github.com/junit-team/junit4/).

On February 18, 2006, JUnit 4.0 was released. It follows the same high-level guidelines than JUnit 3, that is, easily define test, the framework run tests independently, and the framework detects and report errors by the test.

One of the main differences of JUnit 4 with respect to JUnit 3 is the way that JUnit 4 allows to define tests. In JUnit 4, Java annotations are used to mark methods as tests. For this reason, JUnit 4 can only be used for Java 5 or later. As the documentation of JUnit 4.0 stated back in 2006:

The architecture of JUnit 4.0 is a substantial departure from that of earlier releases. Instead of tagging test classes by subclassing junit.framework.TestCase and tagging test methods by starting their name with 'test', you now tag test methods with the @Test annotation.

Standard tests in JUnit 4

In JUnit 4, the @Test annotation (contained in package org.junit) represents a test. Any public method can be annotated with @Test to make it a test method.

In order to set up the test fixture, JUnit 4 provides the @Before annotation. This annotation can be used in any public method. Similarly, any public method annotated with @After gets executed after each test method execution. JUnit 4 provides two more annotations to enhance the test life cycle: @BeforeClass and @AfterClass. They are executed only once per test class, before and after all tests, respectively. The following picture depicts the life cycle of a JUnit 4 test case:

JUnit 4 test life cycle
@Before and @After can be applied to any public void methods. @AfterClass and @BeforeClass can be applied to only public static void methods.

The following table summarizes the main differences between JUnit 3 and JUnit 4 seen so far:

Feature JUnit 3 JUnit 4
Test definition testXXX pattern @Test annotation
Run before the first test Not supported @BeforeClass annotation
Run after all the tests Not supported @AfterClass annotation
Run before each test Override setUp() method @Before annotation
Run after each test Override tearDown() method @After annotation
Ignore tests Not supported @Ignore annotation

The org.junit.Assert class provides static methods to carry out assertions (predicates). The following are the most useful assertion methods:

  • assertTrue: If the condition becomes false, the assertion fails and AssertionError is thrown.
  • assertFalse: If the condition becomes true, the assertion fails and AssertionError is thrown.
  • assertNull: This checks whether the argument is null, otherwise throws AssertionError if the argument is not null.
  • assertNotNull: This checks whether the argument is not null; otherwise, it throws AssertionError
  • assertEquals: This compares two objects or primitive types. Moreover, if the actual value doesn't match the expected value, AssertionError is thrown.
  • assertSame: This supports only objects and checks the object reference using the == operator.
  • assertNotSame: This is the opposite of assertSame.

The following snippets provide a simple example of a JUnit 4 test case. As we can see, it is the equivalent test case as seen in the previous section, this time using the JUnit 4 programming model, that is, using @Test annotation to identify tests and other annotations (@AfterAll, @After, @BeforeAll, @Before) to implement the test life cycle (setup and teardown test fixture):

package io.github.bonigarcia;

import static org.junit.Assert.assertEquals;

import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;

public class TestSimple {

// Phase 1.1: Setup (for all tests)
@BeforeClass
public static void setupAll() {
System.out.println("<Setup Class>");
}

// Phase 1.2: Setup (for each test)
@Before
public void setupTest() {
System.out.println("<Setup Test>");
}

// Test 1: This test is going to succeed
@Test
public void testSuccess() {
// Phase 2: Simulation of exercise
int expected = 60;
int real = 60;
System.out.println("** Test 1 **");

// Phase 3: Verify
assertEquals(expected + " should be equals to "
+ real, expected, real);
}

// Test 2: This test is going to fail
@Test
public void testFailure() {
// Phase 2: Simulation of exercise
int expected = 60;
int real = 20;
System.out.println("** Test 2 **");

// Phase 3: Verify
assertEquals(expected + " should be equals to "
+ real, expected, real);
}

// Phase 4.1: Teardown (for each test)
@After
public void teardownTest() {
System.out.println("</Ending Test>");
}

// Phase 4.2: Teardown (for all test)
@AfterClass
public static void teardownClass() {
System.out.println("</Ending Class>");
}

}

Test execution in JUnit 4

The concept of the test runner is also present in JUnit 4, but it was slightly improved with respect to JUnit 3. In JUnit 4, a test runner is a Java class used to manage a test’s life cycle: instantiation, calling setup and teardown methods, running the test, handling exceptions, sending notifications, and so on. The default JUnit 4 test runner is called BlockJUnit4ClassRunner, and it implements the JUnit 4 standard test case class model.

The test runner to be used in a JUnit 4 test case can be changed simply using the annotation @RunWith. JUnit 4 provides a collection of built-in test runners that allows to change the nature of the test class. In this section, we are going to review the most important ones.

  • To run a group of tests (that is, a test suite) JUnit 4 provides the Suite runner. In addition to the runner, the class Suite.SuiteClasses allows to define the individual test classes belonging to the suite. For example:
     package io.github.bonigarcia;

import org.junit.runner.RunWith;
import org.junit.runners.Suite;

@RunWith(Suite.class)
@Suite.SuiteClasses({ TestMinimal1.class, TestMinimal2.class })
public class MySuite {
}
  • Parameterized tests are used to specify different input data that is going to be used in the same test logic. To implement this kind of tests, JUnit 4 provides the Parameterized runner. To define the data parameters in this type of test, we need to annotate a static method of the class with the annotation @Parameters. This method should return a Collection of the two-dimensional array providing input parameters for the test. Now, there will be two options to inject the input data into the test:
    1. Using the constructor class.
    2. Annotating class attributes with the annotation @Parameter.

The following snippets show an example of the latter:

package io.github.bonigarcia;

import static org.junit.Assert.assertTrue;

import java.util.Arrays;
import java.util.Collection;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameter;
import org.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)
public class TestParameterized {

@Parameter(0)
public int input1;

@Parameter(1)
public int input2;

@Parameter(2)
public int sum;

@Parameters(name = "{index}: input1={0} input2={1} sum={2}?")
public static Collection<Object[]> data() {
return Arrays.asList(
new Object[][] { { 1, 1, 2 }, { 2, 2, 4 }, { 3, 3, 9 } });
}

@Test
public void testSum() {
assertTrue(input1 + "+" + input2 + " is not " + sum,
input1 + input2 == sum);
}

}

The execution of this test on Eclipse would be as follows:

Execution of a Parameterized test in Eclipse
  • JUnit theories are an alternative to JUnit's parameterized tests. A JUnit theory is expected to be true for all datasets. Thus, in JUnit theories, we have a method providing data points (that is, the input values to be used for the test). Then, we need to specific a method annotated with @Theory which takes parameters. The theories in a class get executed with every possible combination of data points:
     package io.github.bonigarcia;

import static org.junit.Assert.assertTrue;

import org.junit.experimental.theories.DataPoints;
import org.junit.experimental.theories.Theories;
import org.junit.experimental.theories.Theory;
import org.junit.runner.RunWith;

@RunWith(Theories.class)
public class MyTheoryTest {

@DataPoints
public static int[] positiveIntegers() {
return new int[] { 1, 10, 100 };
}

@Theory
public void testSum(int a, int b) {
System.out.println("Checking " + a + "+" + b);
assertTrue(a + b > a);
assertTrue(a + b > b);
}
}

Take a look at the execution of this example, again in Eclipse:

Execution of a JUnit 4 theory in Eclipse

Advanced features of JUnit 4

One of the most significant innovations introduced in JUnit 4 was the use of rules. Rules allow flexible addition or redefinition of the behavior of each test method in a test class. A rule should be included in a test case by annotating a class attribute with the annotation @Rule. The type of this attribute should inherit the JUnit interface org.junit.rulesTestRule. The following rules are provided out of the box in JUnit 4:

  • ErrorCollector: This rule allows execution of a test to continue after the first problem is found
  • ExpectedException: This rule allows to verify that a test throws a specific exception
  • ExternalResource: This rule provides a base class for Rules that set up an external resource before a test (a file, socket, server, database connection, and so on) and guarantee to tear it down afterward
  • TestName: This rule makes the current test name available inside test methods
  • TemporaryFolder: This rule allows creation of files and folders that should be deleted when the test method finishes
  • Timeout: This rule applies the same timeout to all test methods in a class
  • TestWatcher: It is a base class for rules that will keep a log of each passing and failing test

Another advance JUnit 4 features allow to:

  • Execute tests is a given order, using the annotation @FixMethodOrder.
  • Create assumptions using the class Assume. This class offers many static methods, such as assumeTrue(condition), assumeFalse(condition), assumeNotNull(condition), and assumeThat(condition). Before executing a test, JUnit checks the assumptions present in the test. If one of the assumptions fail, the JUnit runner ignores the tests with failing assumptions.
  • JUnit provides a timeout value (in milliseconds) in the @Test annotation to make sure that if a test runs longer than the specified value, the test fails.
  • Categorize tests using the test runner Categories and identify the types of test annotating the tests method with the annotation Category.
Meaningful examples for each of one of the earlier mentioned features can be found in the GitHub repository (https://github.com/bonigarcia/mastering-junit5).

JUnit ecosystem

JUnit is one of the most popular test frameworks for the JVM, and it is considered one of the most influential frameworks in software engineering. We can find several libraries and frameworks that provide additional functionality on top of JUnit. Some examples of these ecosystem enhancers are:

  • Mockito (http://site.mockito.org/): This is the mock framework, which can be used in conjunction with JUnit.
  • AssertJ (http://joel-costigliola.github.io/assertj/): This is the fluent assertions library for Java.
  • Hamcrest (http://hamcrest.org/): This is the library with matchers that can be combined to create flexible and readable assertions.
  • Cucumber (https://cucumber.io/): This is the testing framework that allows to run automated acceptance tests written in a Behavior-Driven Development (BDD) style.
  • FitNesse (http://www.fitnesse.org/): This is the testing framework designed to support acceptance testing by facilitating detailed readable descriptions of system functions.

While JUnit is the largest testing framework for the JVM, it is not the only one. There are several other testing frameworks available for the JVM. Some examples are:

Thanks to JUnit, testing has moved to a central part of programming. Consequently, the underlying testing model implemented in JUnit, has been ported to a set of testing frameworks outside the boundary of the JVM, in the so-called xUnit family. In this model, we find the concepts of test case, runner, fixture, suite, test execution, report, and assertion. To name a few, consider the following frameworks. All of them fall into the xUnit family:

You have been reading a chapter from
Mastering Software Testing with JUnit 5
Published in: Oct 2017
Publisher: Packt
ISBN-13: 9781787285736
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at €18.99/month. Cancel anytime