Book Image

Monitoring Hadoop

By : Aman Singh
Book Image

Monitoring Hadoop

By: Aman Singh

Overview of this book

Table of Contents (14 chapters)

System logging


In Linux, the daemon responsible for logging is syslogd or recently the newer version rsyslogd. Applications or daemons write various logs in different files under /var/log using syslogd daemon, which is controlled by the syslog.conf configuration file. The logs can be collected on independent nodes or at a central location using Apache Flume and can be analyzed by using tools such as Flume, Splunk, Logstash, and so on.

There are different logging levels according to how the verbosity of the information logged into files is decided. Each application calls syslog() using an internal function, the log_level, and writes events to appropriate files.