Book Image

Learning Big Data with Amazon Elastic MapReduce

By : Amarkant Singh, Vijay Rayapati
Book Image

Learning Big Data with Amazon Elastic MapReduce

By: Amarkant Singh, Vijay Rayapati

Overview of this book

<p>Amazon Elastic MapReduce is a web service used to process and store vast amount of data, and it is one of the largest Hadoop operators in the world. With the increase in the amount of data generated and collected by many businesses and the arrival of cost-effective cloud-based solutions for distributed computing, the feasibility to crunch large amounts of data to get deep insights within a short span of time has increased greatly.</p> <p>This book will get you started with AWS so that you can quickly create your own account and explore the services provided, many of which you might be delighted to use. This book covers the architectural details of the MapReduce framework, Apache Hadoop, various job models on EMR, how to manage clusters on EMR, and the command-line tools available with EMR. Each chapter builds on the knowledge of the previous one, leading to the final chapter where you will learn about solving a real-world use case using Apache Hadoop and EMR. This book will, therefore, get you up and running with major Big Data technologies quickly and efficiently.</p>
Table of Contents (18 chapters)
Learning Big Data with Amazon Elastic MapReduce
Credits
About the Authors
Acknowledgments
About the Reviewers
www.PacktPub.com
Preface
Index

Driver implementation


The driver class is the one which has the main method and provides the place where the Hadoop job is created and its mapper and reducer along with a bunch of other configurations and settings are declared. The job is initiated from here itself.

We will name this class HitsByCountry and let's create this class inside the learning.bigdata.main package. Your driver class should have the following signature:

public class HitsByCountry extends Configured implements Tool {
    // Here we will have the main method as well as the overridden implementation of run method
}

The driver class extends the Configured class and implements the Tool interface. There are many Hadoop configurations you can set in the driver class while creating a job. For example, you can set the number of reducers using the mapred.reduce.tasks configuration and you can set the separator between the key and value you will have in your reducer output while using TextOutputFormat with mapred.textoutputformat...