Book Image

Modern Big Data Processing with Hadoop

By : V Naresh Kumar, Manoj R Patil, Prashant Shindgikar

Book Image

Modern Big Data Processing with Hadoop

By: V Naresh Kumar, Manoj R Patil, Prashant Shindgikar

Overview of this book

The complex structure of data these days requires sophisticated solutions for data transformation, to make the information more accessible to the users.This book empowers you to build such solutions with relative ease with the help of Apache Hadoop, along with a host of other Big Data tools. This book will give you a complete understanding of the data lifecycle management with Hadoop, followed by modeling of structured and unstructured data in Hadoop. It will also show you how to design real-time streaming pipelines by leveraging tools such as Apache Spark, and build efficient enterprise search solutions using Elasticsearch. You will learn to build enterprise-grade analytics solutions on Hadoop, and how to visualize your data using tools such as Apache Superset. This book also covers techniques for deploying your Big Data solutions on the cloud Apache Ambari, as well as expert techniques for managing and administering your Hadoop cluster. By the end of this book, you will have all the knowledge you need to build expert Big Data systems.

Preface

Who this book is for

What this book covers

To get the most out of this book

Free Chapter

Enterprise Data Architecture Principles

Enterprise Data Architecture Principles

Data architecture principles

The importance of metadata

Data governance

Data as a Service

Evolution data architecture with Hadoop

Hadoop Life Cycle Management

Hadoop Life Cycle Management

Hadoop Design Consideration

Hadoop Design Consideration

Understanding data structure principles

Installing Hadoop cluster

Exploring HDFS architecture

Introducing YARN

Configuring HDFS high availability

Configuration of HA NameNodes with QJM

Hadoop cluster composition

Best practices Hadoop deployment

Hadoop file formats

Data Movement Techniques

Data Movement Techniques

Batch processing versus real-time processing

Data Modeling in Hadoop

Data Modeling in Hadoop

Supported datatypes

Hive architecture

Hive data model management

JSON documents using Hive

Designing Real-Time Streaming Data Pipelines

Designing Real-Time Streaming Data Pipelines

Real-time streaming concepts

Real-time streaming components

Other popular real-time data streaming frameworks

Apache Flink versus Spark

Apache Spark versus Storm

Large-Scale Data Processing Frameworks

Large-Scale Data Processing Frameworks

Hadoop MapReduce

Building Enterprise Search Platform

Building Enterprise Search Platform

The data search concept

The need for an enterprise search engine

How to index documents in Elasticsearch?

Elasticsearch-supported data types

Designing Data Visualization Solutions

Designing Data Visualization Solutions

Data visualization

Practical data visualization in Hadoop

Developing Applications Using the Cloud

Developing Applications Using the Cloud

What is the Cloud?

Available technologies in the Cloud

Planning the Cloud infrastructure

Building a Hadoop cluster in the Cloud

Data access in the Cloud

Production Hadoop Cluster Deployment

Production Hadoop Cluster Deployment

Apache Ambari architecture

Setting up a Hadoop cluster with Ambari

Hadoop clusters

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Apache Spark 2

Apache Spark is a general-purpose cluster computing system. It's very well suited for large-scale data processing. It performs 100 times better than Hadoop when run completely in-memory and 10 times better when run entirely from disk. It has a sophisticated directed acyclic graph execution engine that supports an acyclic data flow model.

Apache Spark has first-class support for writing programs in Java, Scala, Python, and R programming languages to cater to a wider audience. It offers more than 80 different operators to build parallel applications without worrying about the underlying infrastructure.

Apache Spark has libraries catering to Structured Query Language, known as Spark SQL; this supports writing queries in programs using ANSI SQL. It also has support for computing streaming data, which is very much needed in today's real-time data processing...