Book Image

HDInsight Essentials - Second Edition

By : Rajesh Nadipalli
Book Image

HDInsight Essentials - Second Edition

By: Rajesh Nadipalli

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
HDInsight Essentials - Second Edition
Rajesh Nadipalli
Jan, 2015 | 178 pages
No titles found
Table of Contents (16 chapters)
HDInsight Essentials Second Edition
HDInsight Essentials Second Edition
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Hadoop and HDInsight in a Heartbeat
Hadoop and HDInsight in a Heartbeat
Data is everywhere
Hadoop concepts
Hadoop distributions
HDInsight overview
Hadoop on Windows deployment options
Summary
2
Enterprise Data Lake using HDInsight
Enterprise Data Lake using HDInsight
Enterprise Data Warehouse architecture
The next generation Hadoop-based Enterprise data architecture
Journey to your Data Lake dream
Tools and technology for Hadoop ecosystem
Use case powered by Microsoft HDInsight
Summary
3
HDInsight Service on Azure
HDInsight Service on Azure
Registering for an Azure account
Azure storage
Provisioning an HDInsight cluster
HDInsight management dashboard
Exploring clusters using the remote desktop
Deleting the cluster
HDInsight Emulator for the development
Summary
4
Administering Your HDInsight Cluster
Administering Your HDInsight Cluster
Monitoring cluster health
Name Node status
Hadoop Service Availability
YARN Application Status
Azure storage management
Azure PowerShell
Summary
5
Ingest and Organize Data Lake
Ingest and Organize Data Lake
End-to-end Data Lake solution
Ingesting to Data Lake using HDFS command
Loading data to Azure Blob storage using Azure PowerShell
Loading files to Data Lake using GUI tools
Using Sqoop to move data from RDBMS to Data Lake
Organizing your Data Lake in HDFS
Managing file metadata using HCatalog
Summary
6
Transform Data in the Data Lake
Transform Data in the Data Lake
Transformation overview
Tools for transforming data in Data Lake
Transformation for the OTP project
Other tools used for transformation
Summary
7
Analyze and Report from Data Lake
Analyze and Report from Data Lake
Data access overview
Analysis using Excel and Microsoft Hive ODBC driver
Analysis using Excel Power Query
Other BI features in Excel
Ad hoc analysis using Hive
Other alternatives for analysis
Summary
8
HDInsight 3.1 New Features
HDInsight 3.1 New Features
HBase
Storm
Apache Tez
Summary
9
Strategy for a Successful Data Lake Implementation
Strategy for a Successful Data Lake Implementation
Challenges on building a production Data Lake
The success path for a production Data Lake
Architectural considerations
Online resources
Summary
Index
Index

Monitoring cluster health

To ensure that development has a stable Hadoop environment, the operations team should have visibility of the cluster status, including the health of various services in a programmatic manner. This section discusses how to monitor an HDInsight cluster using remote connection to the head node in detail. From the HDInsight CONFIGURATION page, you can download the RDP file to connect to the head node remotely.

Once connected, you will see the familiar Windows desktop, as shown in the following screenshot:

There are four key shortcuts on the remote desktop that help you monitor and manage the cluster. The following table lists the different shortcuts and their purpose:

Link name

URL

Purpose

Hadoop Command Line

None

This is a Windows command line shortcut to run any Hadoop command such as listing of files or calling MapReduce program

Hadoop Name Node Status

http://headnodehost:30070

This lists the Hadoop Name Node status and summary statistics

Hadoop Service...

Previous Section
End of Section 2
Next Section