Book Image

Simplify Big Data Analytics with Amazon EMR

By : Sakti Mishra
Book Image

Simplify Big Data Analytics with Amazon EMR

By: Sakti Mishra

Overview of this book

Amazon EMR, formerly Amazon Elastic MapReduce, provides a managed Hadoop cluster in Amazon Web Services (AWS) that you can use to implement batch or streaming data pipelines. By gaining expertise in Amazon EMR, you can design and implement data analytics pipelines with persistent or transient EMR clusters in AWS. This book is a practical guide to Amazon EMR for building data pipelines. You'll start by understanding the Amazon EMR architecture, cluster nodes, features, and deployment options, along with their pricing. Next, the book covers the various big data applications that EMR supports. You'll then focus on the advanced configuration of EMR applications, hardware, networking, security, troubleshooting, logging, and the different SDKs and APIs it provides. Later chapters will show you how to implement common Amazon EMR use cases, including batch ETL with Spark, real-time streaming with Spark Streaming, and handling UPSERT in S3 Data Lake with Apache Hudi. Finally, you'll orchestrate your EMR jobs and strategize on-premises Hadoop cluster migration to EMR. In addition to this, you'll explore best practices and cost optimization techniques while implementing your data analytics pipeline in EMR. By the end of this book, you'll be able to build and deploy Hadoop- or Spark-based apps on Amazon EMR and also migrate your existing on-premises Hadoop workloads to AWS.
Table of Contents (19 chapters)
1
Section 1: Overview, Architecture, Big Data Applications, and Common Use Cases of Amazon EMR
6
Section 2: Configuration, Scaling, Data Security, and Governance
11
Section 3: Implementing Common Use Cases and Best Practices

Chapter 2: Exploring the Architecture and Deployment Options

This chapter will dive deep into the Elastic MapReduce (EMR) architecture. We will also look at the different deployment options it provides, such as Amazon EMR on Amazon Elastic Compute Cloud (EC2), Amazon EMR on Amazon Elastic Kubernetes Service (EKS), and Amazon EMR on AWS Outposts. It will also explain details around different EMR cluster node types, its life cycle, and ways to submit work to the cluster.

Toward the end of the chapter, you will learn how EMR pricing works with different deployment options and how you can use AWS Budgets and Cost Explorer for cost-related monitoring.

As we proceed to further chapters of this book, where we will cover different use cases and implementation patterns around EMR, an understanding of the architecture and deployment options will be a prerequisite.

The following topics will be covered in this chapter:

  • EMR architecture deep dive
  • Understanding clusters and...