While unit testing using MRUnit is very useful, there can be certain integration test scenarios that have to be tested in a cluster environment. MiniYARNCluster of Hadoop YARN is a cluster simulator that we can use to create a testing environment for such integration tests. In this recipe, we'll be using MiniYARNCluster to perform integration testing of the WordCountWithTools MapReduce application.
The source code for the test program used in this recipe is available in the chapter3\test\chapter3\minicluster\WordCountMiniClusterTest.java
file in the Git repository.
We use Gradle as the build tool for our sample code base. If you haven't already done so, please install Gradle by following the instructions given in the introduction section of Chapter 1, Getting Started with Hadoop v2. Export the JAVA_HOME
environmental variable pointing to your JDK installation.