Both YARN services, ResourceManager and NodeManager generate logs and store them in a
.log file locally inside the folder specified using the
HADOOP_LOGS_DIR variable. By default, the logs are stored in
HADOOP_PREFIX/logs. All the state transitions in YARN are recorded in the log files. In this section, we'll cover few state transitions and the logs generated during those transitions.
Setting the log level: Hadoop-YARN uses Apache
Log4j library and it uses a
log4j.properties file located in the configuration folder of the Hadoop-YARN bundle at
Log4j library supports six log levels –
FATAL. A cluster administrator sets the log level for Hadoop-YARN services and the default log level is
hadoop.root.logger property is used to update the log level for Hadoop-YARN services. To read more about Apache
Log4j library, you can refer to the official site at http://logging.apache.org/log4j...