Book Image

Hadoop MapReduce v2 Cookbook - Second Edition: RAW

Book Image

Hadoop MapReduce v2 Cookbook - Second Edition: RAW

Overview of this book

Table of Contents (19 chapters)
Hadoop MapReduce v2 Cookbook Second Edition
Credits
About the Author
Acknowledgments
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Integration testing Hadoop MapReduce applications using MiniYarnCluster


While unit testing using MRUnit is very useful, there can be certain integration test scenarios that have to be tested in a cluster environment. MiniYARNCluster of Hadoop YARN is a cluster simulator that we can use to create a testing environment for such integration tests. In this recipe, we'll be using MiniYARNCluster to perform integration testing of the WordCountWithTools MapReduce application.

The source code for the test program used in this recipe is available in the chapter3\test\chapter3\minicluster\WordCountMiniClusterTest.java file in the Git repository.

Getting ready

We use Gradle as the build tool for our sample code base. If you haven't already done so, please install Gradle by following the instructions given in the introduction section of Chapter 1, Getting Started with Hadoop v2. Export the JAVA_HOME environmental variable pointing to your JDK installation.

How to do it...

The following steps show you how...