Book Image

Monitoring Hadoop

By : Aman Singh
Book Image

Monitoring Hadoop

By: Aman Singh

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
Monitoring Hadoop
Aman Singh
Apr, 2015 | 100 pages
No titles found
Table of Contents (14 chapters)
Monitoring Hadoop
Monitoring Hadoop
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Introduction to Monitoring
Introduction to Monitoring
The need for monitoring
The monitoring tools available in the market
System logging
Summary
2
Hadoop Daemons and Services
Hadoop Daemons and Services
Hadoop daemons
YARN framework
Summary
3
Hadoop Logging
Hadoop Logging
The need for logging events
System logging
Logging levels
Logging in Hadoop
Summary
4
HDFS Checks
HDFS Checks
HDFS overview
Nagios master configuration
The Nagios client configuration
Summary
5
MapReduce Checks
MapReduce Checks
MapReduce overview
MapReduce control commands
MapReduce health checks
Nagios master configuration
Nagios client configuration
Summary
6
Hadoop Metrics and Visualization Using Ganglia
Hadoop Metrics and Visualization Using Ganglia
Hadoop metrics
Metrics contexts
Metrics system design
Metrics configuration
Configuring Metrics2
Exploring the metrics contexts
Hadoop Ganglia integration
Hadoop configuration
Ganglia graphs
Metrics APIs
Summary
7
Hive, HBase, and Monitoring Best Practices
Hive, HBase, and Monitoring Best Practices
Hive monitoring
Hive metrics
HBase Nagios monitoring
HBase metrics
Monitoring best practices
The Filter class
Nagios and Ganglia best practices
Summary
Index
Index

Hadoop configuration

Now, we must set up the Hadoop Configuration file to point to the Ganglia servers.

Metrics1

Update the hadoop-metrics.properties file with the following lines:

dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
dfs.period=10
dfs.servers=192.168.1.10:8649

mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
mapred.period=10
mapred.servers=192.168.1.10:8649

jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
jvm.period=10
jvm.servers=192.168.1.10:8649

rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
rpc.period=10
rpc.servers=192.168.1.10:8649

ugi.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
ugi.period=10
ugi.servers=192.168.1.10:8649

Metrics2

Update the hadoop-metrics2.properties file with the following lines:

namenode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
namenode.sink.ganglia.period=30
namenode.sink.ganglia.servers=192.168.1.10:8649

datanode.sink.ganglia.class=org.apache.hadoop.metrics2...
Previous Section
End of Section 9
Next Section