Book Image

Monitoring Hadoop

By : Aman Singh
Book Image

Monitoring Hadoop

By: Aman Singh

Overview of this book

Table of Contents (14 chapters)

Metrics system design


Hadoop provides a framework to collect internal events and metrics and report them to the external system. The external system could be simply writing to a file or a tool like Ganglia. The new Hadoop Metrics2 framework has been revamped to integrate better with Ganglia.

The best things about the framework are the pluggable output plugins and the ability to reconfigure it without the need to restart the daemons.

The metrics have three main parts:

  • Producer: The producer is the source of metrics generation and produces metrics for the upstream

  • Consumers: They are basically the sinks of the framework, as they consume the metrics generated by the producers

  • Pollers: They poll the sources and deliver data to the sink or consumers