Apache Hadoop is an open source distributed computing technology that assists users in processing large volumes of data with relative ease, helping them to generate tremendous insights into their data. Cloudera, with their open source distribution of Hadoop, has made data analytics on Big Data possible and accessible to anyone interested.
This book fully prepares you to be a Hadoop administrator, with special emphasis on Cloudera. It provides step-by-step instructions on setting up and managing a robust Hadoop cluster running Cloudera's Distribution Including Apache Hadoop (CDH).
This book starts out by giving you a brief introduction to Apache Hadoop and Cloudera. You will then move on to learn about all the tools and techniques needed to set up and manage a production-standard Hadoop cluster using CDH and Cloudera Manager.
In this book, you will learn the Hadoop architecture by understanding the different features of HDFS and walking through the entire flow of a MapReduce process. With this understanding, you will start exploring the different applications packaged into CDH and will follow a step-by-step guide to set up HDFS High Availability (HA) and HDFS Federation.
You will learn to use Cloudera Manager, Cloudera's cluster management application. Using Cloudera Manager, you will walk through the steps to configure security using Kerberos, learn about events and alerts, and also configure backups.
Chapter 1, Getting Started with Apache Hadoop, introduces you to Apache Hadoop and walks you through the different Apache Hadoop daemons.
Chapter 2, HDFS and MapReduce, provides you with an in-depth understanding of HDFS and MapReduce.
Chapter 3, Cloudera's Distribution Including Apache Hadoop, introduces you to Cloudera's Apache Hadoop Distribution and walks you through its installation steps.
Chapter 4, Exploring HDFS Federation and Its High Availability, introduces you to the steps to configure a federated HDFS and also provides step-by-step instructions to set up HDFS High Availability.
Chapter 5, Using Cloudera Manager, introduces you to Cloudera Manager, Cloudera's cluster management application and walks you through the steps to install Cloudera Manager.
Chapter 6, Implementing Security Using Kerberos, walks you through the steps to secure your cluster by configuring Kerberos.
Chapter 7, Managing an Apache Hadoop Cluster, introduces you to all the cluster management capabilities available within Cloudera Manager.
Chapter 8, Cluster Monitoring Using Events and Alerts, introduces you to the different events and alerts available within Cloudera Manager that will assist you in monitoring your cluster effectively.
Chapter 9, Configuring Backups, walks you through the steps to configure backups and snapshots using Cloudera Manager.
You will need access to a cluster of around three to four nodes (physical server or virtual machines) running Linux, preferably the CentOS distribution. The steps to acquire the software needed is explained in detail in this book.
This book is ideal for anyone interested in administering an Apache Hadoop cluster. This book will prove to be a good guide for administrators managing clusters running Cloudera's Distribution Including Apache Hadoop (CDH) and will be introduced to the various tools and techniques such as cluster management, security, monitoring, and backups. The reader will acquire all the knowledge required to run production scale clusters ranging from a few nodes to thousands of nodes.
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "It is important to note that the fsimage file is not updated for every write operation."
A block of code is set as follows:
<property> <name>dfs.namenode.servicerpc-address</name> <value>node1.hcluster:8022</value> </property>
Any command-line input or output is written as follows:
$ sudo yum install flume-ng $ sudo yum install flume-ng-agent
New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: "The Coordinators tab lists all the coordinator applications that have been configured in Oozie."
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
You can contact us at <[email protected]> if you are having a problem with any aspect of the book, and we will do our best to address it.