Book Image

Big Data Architect's Handbook

By : Syed Muhammad Fahad Akhtar
Book Image

Big Data Architect's Handbook

By: Syed Muhammad Fahad Akhtar

Overview of this book

The big data architects are the “masters” of data, and hold high value in today’s market. Handling big data, be it of good or bad quality, is not an easy task. The prime job for any big data architect is to build an end-to-end big data solution that integrates data from different sources and analyzes it to find useful, hidden insights. Big Data Architect’s Handbook takes you through developing a complete, end-to-end big data pipeline, which will lay the foundation for you and provide the necessary knowledge required to be an architect in big data. Right from understanding the design considerations to implementing a solid, efficient, and scalable data pipeline, this book walks you through all the essential aspects of big data. It also gives you an overview of how you can leverage the power of various big data tools such as Apache Hadoop and ElasticSearch in order to bring them together and build an efficient big data solution. By the end of this book, you will be able to build your own design system which integrates, maintains, visualizes, and monitors your data. In addition, you will have a smooth design flow in each process, putting insights in action.

Related Content you might be interested in

Current Title:

Small book image
Big Data Architect's Handbook
Syed Muhammad Fahad Akhtar
Jun, 2018 | 486 pages
Small book image
Apache Hadoop 3 Quick Start Guide
Hrishikesh Vijay Karambelkar
Oct, 2018 | 220 pages
Small book image
Data Lake for Enterprises
Pankaj Misra | Tomcy John
May, 2017 | 596 pages
Small book image
Hands-on DevOps
Sricharan Vadapalli
Dec, 2017 |
Table of Contents (21 chapters)
Preface
Preface
Free Chapter
1
Why Big Data?
Why Big Data?
2
Big Data Environment Setup
Big Data Environment Setup
3
Hadoop Ecosystem
Hadoop Ecosystem
4
NoSQL Database
NoSQL Database
5
Off-the-Shelf Commercial Tools
Off-the-Shelf Commercial Tools
6
Containerization
Containerization
7
Network Infrastructure
Network Infrastructure
8
Cloud Infrastructure
Cloud Infrastructure
9
Security and Monitoring
Security and Monitoring
10
Frontend Architecture
Frontend Architecture
11
Backend Architecture
Backend Architecture
12
Machine Learning
Machine Learning
13
Artificial Intelligence
Artificial Intelligence
14
Elasticsearch
Elasticsearch
15
Structured Data
Structured Data
16
Unstructured Data
Unstructured Data
17
Data Visualization
Data Visualization
18
Financial Trading System
Financial Trading System
19
Retail Recommendation System
Retail Recommendation System
20
Other Books You May Enjoy
Other Books You May Enjoy
Hadoop Ecosystem

We should now have an understanding of what big data is and how it will impact industries in their decision-making. We have also created and configured our own big data virtual environment so that we can move forward in practical terms and build our own applications. These applications will get more and more mature as we proceed further in this book.

In this chapter, we will look at the Apache Hadoop Project in detail, touching on why it was developed and what the advantages and disadvantages of using Hadoop are. We will also go through the different components and modules of the Hadoop system and will also get our hands dirty with some practical activities. The main topics that we are going to cover in this chapter are as follows:

  • Apache Hadoop
  • HDFS
  • MapReduce Framework
  • YARN

We will then take a look at other Apache Projects that are directly or indirectly related...

Previous Section
End of Chapter 3
Next Chapter