JUnit is a testing framework which allows to create automated tests. The development of JUnit was started by Kent Beck and Erich Gamma in late 1995. Since then, the popularity of the framework has been growing. Nowadays, it is broadly considered as the de facto standard for testing Java applications.

JUnit was designed to be a unit-testing framework. Nevertheless, it can be used to implement not just unit tests, but also other kinds of tests. As we will discover in the body of this book, depending on how the test logic exercises the piece of software under test, a test case implemented with JUnit can be considered as an unit, integration, system, and even acceptance test. All in all, we can think of JUnit as a multi-purpose testing framework for Java.

Since the early versions of JUnit 3, the framework can work with Java 2 and higher. JUnit3 is open source software, released under Common Public License (CPL) Version 1.0 and hosted on SourceForge (https://sourceforge.net/projects/junit/). The latest version of JUnit 3 was JUnit 3.8.2, released on May 14, 2007. The main requirements introduced by JUnit in the world of testing frameworks were the following:

1. It should be easy to define which tests will run.
2. The framework should be able to run tests independently of all other tests.
3. The framework should detect and report errors test by test.

In JUnit 3, in order to create test cases, we need to extend the class junit.framework.TestCase. This base class includes the framework code that JUnit needs to automatically run the tests. Then, we simply make sure that the method name follows the testXXX() pattern. This naming convention makes it clear to the framework that the method is a unit test and that it can be run automatically.

The test life cycle is controlled in the setup() and tearDown()methods. The TestCase calls setup() before running each of its tests and then calls teardown() when each test is complete. One reason to put more than one test method into the same test case is to share the same test fixture.

Finally, in order to implement the verification stage in the test case, JUnit 3 defines several assert methods in a utility class named junit.framework.Assert. The following table summarizes the main assertions provided by this class:

The following class shows a simple test implemented with JUnit 3.8.2. As we can see, this test case contains two tests. Before each test, the method setUp() will be invoked by the framework, and after the execution of each test, the method tearDown() will be also invoked. This example has been coded so that the first test, named testSuccess() finishes correctly, and the second test named testFailure() ends with an error (the assertion throws an exception):

package io.github.bonigarcia;

import junit.framework.TestCase;

public class TestSimple extends TestCase {

    // Phase 1: Setup (for each test)
    protected void setUp() throws Exception {
        System.out.println("<Setup>");
    }

    // Test 1: This test is going to succeed
    public void testSuccess() {
        // Phase 2: Simulation of exercise
        int expected = 60;
        int real = 60;
        System.out.println("** Test 1 **");

        // Phase 3: Verify
        assertEquals(expected + " should be equals to " 
         + real, expected, real);
    }

    // Test 2: This test is going to fail
    public void testFailure() {
        // Phase 2: Simulation of exercise
        int expected = 60;
        int real = 20;
        System.out.println("** Test 2 **");

        // Phase 3: Verify
        assertEquals(expected + " should be equals to " 
         + real, expected, real);
    }

    // Phase 4: Teardown (for each test)
    protected void tearDown() throws Exception {
        System.out.println("</Ending>");
    }

}

JUnit 3 allows to run test cases by means of Java applications called test runners. JUnit 3.8.2 provides three different test runners out of the box: two graphical (Swing and AWT based) and one textual that can be used from the command line. The JUnit framework provides separate class loaders for each test, in order to avoid side effects among tests.

It is a common practice that build tools (such as Ant or Maven) and Integrated Development Environments -IDE- (such as Eclipse and IntelliJ) implement its own JUnit test runner.

The following image shows what the previous test looks like when we use the JUnit Swing runner, and also when we use Eclipse to run the same test case.

When a test is not succeeded in JUnit, it can be for two reasons: a failure or an error. On the one hand, a failure is caused by an assertion (Assert class) which is not meet. On the other hand, an error is an unexpected condition not expected by the test, such as a conventional exception in the software under test.

Another important contribution of JUnit 3 is the concept of the test suite, which is a convenient way to group tests that are related. Test suites are implemented by means of the JUnit class junit.framework.TestSuite. This class, in the same way as TestCase, implements the framework interface junit.framework.Test.

A diagram containing the main classes and methods of JUnit 3 is depicted as follows:

The following snippet shows an example of the use of test suites in JUnit 3. In short, we can create a group of tests simply instantiating a TestSuite object, and then add single test cases using the method addTestSuite():

package io.github.bonigarcia;

import junit.framework.Test;
import junit.framework.TestSuite;

public class TestAll {

    public static Test suite() {
        TestSuite suite = new TestSuite("All tests");
        suite.addTestSuite(TestSimple.class);
        suite.addTestSuite(TestMinimal.class);
        return suite;
    }
}

This test suite can be later executed using a test runner. For example, we could use the command-line test runner (junit.textui.TestRunner) and the command line, as follows:

JUnit 4 is still an open source framework, though the license changed with respect to JUnit 3, from CPL to Eclipse Public License (EPL) Version 1.0. The source code of JUnit 4 is hosted on GitHub (https://github.com/junit-team/junit4/).

On February 18, 2006, JUnit 4.0 was released. It follows the same high-level guidelines than JUnit 3, that is, easily define test, the framework run tests independently, and the framework detects and report errors by the test.

One of the main differences of JUnit 4 with respect to JUnit 3 is the way that JUnit 4 allows to define tests. In JUnit 4, Java annotations are used to mark methods as tests. For this reason, JUnit 4 can only be used for Java 5 or later. As the documentation of JUnit 4.0 stated back in 2006:

In JUnit 4, the @Test annotation (contained in package org.junit) represents a test. Any public method can be annotated with @Test to make it a test method.

In order to set up the test fixture, JUnit 4 provides the @Before annotation. This annotation can be used in any public method. Similarly, any public method annotated with @After gets executed after each test method execution. JUnit 4 provides two more annotations to enhance the test life cycle: @BeforeClass and @AfterClass. They are executed only once per test class, before and after all tests, respectively. The following picture depicts the life cycle of a JUnit 4 test case:

The following table summarizes the main differences between JUnit 3 and JUnit 4 seen so far:

The org.junit.Assert class provides static methods to carry out assertions (predicates). The following are the most useful assertion methods:

• assertTrue: If the condition becomes false, the assertion fails and AssertionError is thrown.
• assertFalse: If the condition becomes true, the assertion fails and AssertionError is thrown.
• assertNull: This checks whether the argument is null, otherwise throws AssertionError if the argument is not null.
• assertNotNull: This checks whether the argument is not null; otherwise, it throws AssertionError
• assertEquals: This compares two objects or primitive types. Moreover, if the actual value doesn't match the expected value, AssertionError is thrown.
• assertSame: This supports only objects and checks the object reference using the == operator.
• assertNotSame: This is the opposite of assertSame.

The following snippets provide a simple example of a JUnit 4 test case. As we can see, it is the equivalent test case as seen in the previous section, this time using the JUnit 4 programming model, that is, using @Test annotation to identify tests and other annotations (@AfterAll, @After, @BeforeAll, @Before) to implement the test life cycle (setup and teardown test fixture):

package io.github.bonigarcia;

import static org.junit.Assert.assertEquals;

import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;

public class TestSimple {

    // Phase 1.1: Setup (for all tests)
    @BeforeClass
    public static void setupAll() {
        System.out.println("<Setup Class>");
    }

    // Phase 1.2: Setup (for each test)
    @Before
    public void setupTest() {
        System.out.println("<Setup Test>");
    }

    // Test 1: This test is going to succeed
    @Test
    public void testSuccess() {
        // Phase 2: Simulation of exercise
        int expected = 60;
        int real = 60;
        System.out.println("** Test 1 **");

        // Phase 3: Verify
        assertEquals(expected + " should be equals to " 
          + real, expected, real);
    }

    // Test 2: This test is going to fail
    @Test
    public void testFailure() {
        // Phase 2: Simulation of exercise
        int expected = 60;
        int real = 20;
        System.out.println("** Test 2 **");

        // Phase 3: Verify
        assertEquals(expected + " should be equals to " 
          + real, expected, real);
    }

    // Phase 4.1: Teardown (for each test)
    @After
    public void teardownTest() {
        System.out.println("</Ending Test>");
    }

    // Phase 4.2: Teardown (for all test)
    @AfterClass
    public static void teardownClass() {
        System.out.println("</Ending Class>");
    }

}

The concept of the test runner is also present in JUnit 4, but it was slightly improved with respect to JUnit 3. In JUnit 4, a test runner is a Java class used to manage a test’s life cycle: instantiation, calling setup and teardown methods, running the test, handling exceptions, sending notifications, and so on. The default JUnit 4 test runner is called BlockJUnit4ClassRunner, and it implements the JUnit 4 standard test case class model.

The test runner to be used in a JUnit 4 test case can be changed simply using the annotation @RunWith. JUnit 4 provides a collection of built-in test runners that allows to change the nature of the test class. In this section, we are going to review the most important ones.

• To run a group of tests (that is, a test suite) JUnit 4 provides the Suite runner. In addition to the runner, the class Suite.SuiteClasses allows to define the individual test classes belonging to the suite. For example:

     package io.github.bonigarcia;

     import org.junit.runner.RunWith;
     import org.junit.runners.Suite;

     @RunWith(Suite.class)
     @Suite.SuiteClasses({ TestMinimal1.class, TestMinimal2.class })
     public class MySuite {
     }

• Parameterized tests are used to specify different input data that is going to be used in the same test logic. To implement this kind of tests, JUnit 4 provides the Parameterized runner. To define the data parameters in this type of test, we need to annotate a static method of the class with the annotation @Parameters. This method should return a Collection of the two-dimensional array providing input parameters for the test. Now, there will be two options to inject the input data into the test:
  1. Using the constructor class.
  2. Annotating class attributes with the annotation @Parameter.

The following snippets show an example of the latter:

package io.github.bonigarcia;

import static org.junit.Assert.assertTrue;

import java.util.Arrays;
import java.util.Collection;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameter;
import org.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)
public class TestParameterized {

    @Parameter(0)
    public int input1;

    @Parameter(1)
    public int input2;

    @Parameter(2)
    public int sum;

    @Parameters(name = "{index}: input1={0} input2={1} sum={2}?")
    public static Collection<Object[]> data() {
        return Arrays.asList(
                new Object[][] { { 1, 1, 2 }, { 2, 2, 4 }, { 3, 3, 9 } });
    }

    @Test
    public void testSum() {
        assertTrue(input1 + "+" + input2 + " is not " + sum,
                input1 + input2 == sum);
    }

}

The execution of this test on Eclipse would be as follows:

• JUnit theories are an alternative to JUnit's parameterized tests. A JUnit theory is expected to be true for all datasets. Thus, in JUnit theories, we have a method providing data points (that is, the input values to be used for the test). Then, we need to specific a method annotated with @Theory which takes parameters. The theories in a class get executed with every possible combination of data points:

     package io.github.bonigarcia;

     import static org.junit.Assert.assertTrue;

     import org.junit.experimental.theories.DataPoints;
     import org.junit.experimental.theories.Theories;
     import org.junit.experimental.theories.Theory;
     import org.junit.runner.RunWith;

      @RunWith(Theories.class)
     public class MyTheoryTest {

         @DataPoints
         public static int[] positiveIntegers() {
             return new int[] { 1, 10, 100 };
         }

         @Theory
         public void testSum(int a, int b) {
             System.out.println("Checking " + a + "+" + b);
             assertTrue(a + b > a);
             assertTrue(a + b > b);
         }
     }

Take a look at the execution of this example, again in Eclipse:

One of the most significant innovations introduced in JUnit 4 was the use of rules. Rules allow flexible addition or redefinition of the behavior of each test method in a test class. A rule should be included in a test case by annotating a class attribute with the annotation @Rule. The type of this attribute should inherit the JUnit interface org.junit.rulesTestRule. The following rules are provided out of the box in JUnit 4:

• ErrorCollector: This rule allows execution of a test to continue after the first problem is found
• ExpectedException: This rule allows to verify that a test throws a specific exception
• ExternalResource: This rule provides a base class for Rules that set up an external resource before a test (a file, socket, server, database connection, and so on) and guarantee to tear it down afterward
• TestName: This rule makes the current test name available inside test methods
• TemporaryFolder: This rule allows creation of files and folders that should be deleted when the test method finishes
• Timeout: This rule applies the same timeout to all test methods in a class
• TestWatcher: It is a base class for rules that will keep a log of each passing and failing test

Another advance JUnit 4 features allow to:

• Execute tests is a given order, using the annotation @FixMethodOrder.
• Create assumptions using the class Assume. This class offers many static methods, such as assumeTrue(condition), assumeFalse(condition), assumeNotNull(condition), and assumeThat(condition). Before executing a test, JUnit checks the assumptions present in the test. If one of the assumptions fail, the JUnit runner ignores the tests with failing assumptions.
• JUnit provides a timeout value (in milliseconds) in the @Test annotation to make sure that if a test runs longer than the specified value, the test fails.
• Categorize tests using the test runner Categories and identify the types of test annotating the tests method with the annotation Category.

JUnit is one of the most popular test frameworks for the JVM, and it is considered one of the most influential frameworks in software engineering. We can find several libraries and frameworks that provide additional functionality on top of JUnit. Some examples of these ecosystem enhancers are:

• Mockito (http://site.mockito.org/): This is the mock framework, which can be used in conjunction with JUnit.
• AssertJ (http://joel-costigliola.github.io/assertj/): This is the fluent assertions library for Java.
• Hamcrest (http://hamcrest.org/): This is the library with matchers that can be combined to create flexible and readable assertions.
• Cucumber (https://cucumber.io/): This is the testing framework that allows to run automated acceptance tests written in a Behavior-Driven Development (BDD) style.
• FitNesse (http://www.fitnesse.org/): This is the testing framework designed to support acceptance testing by facilitating detailed readable descriptions of system functions.

While JUnit is the largest testing framework for the JVM, it is not the only one. There are several other testing frameworks available for the JVM. Some examples are:

• TestNG (http://testng.org/): This is the testing framework inspired from JUnit and NUnit.
• Spock (http://spockframework.org/): This is the testing and specification framework for Java and Groovy applications.
• Jtest (https://www.parasoft.com/product/jtest/): This is the automated Java testing and static analysis framework made and distributed by the company Parasoft.
• Scalatest (http://www.scalatest.org/): This is the testing framework for Scala, Scala.js (JavaScript), and Java applications.

Thanks to JUnit, testing has moved to a central part of programming. Consequently, the underlying testing model implemented in JUnit, has been ported to a set of testing frameworks outside the boundary of the JVM, in the so-called xUnit family. In this model, we find the concepts of test case, runner, fixture, suite, test execution, report, and assertion. To name a few, consider the following frameworks. All of them fall into the xUnit family:

• Google Test (https://github.com/google/googletest): Google's C++ testing framework.
• JSUnit (http://www.jsunit.net/): Unit testing framework for JavaScript.
• Mocha (https://mochajs.org/): Unit testing framework running on Node.js.
• NUnit (https://www.nunit.org/): Unit testing framework for Microsoft.NET.
• PHPUnit (https://phpunit.de/): Unit testing framework for PHP.
• SimplyVBUnit (http://simplyvbunit.sourceforge.net/): Unit testing framework for VB.NET.
• Unittest (https://docs.python.org/3/library/unittest.html): Unit testing framework for Python.