Both YARN services, ResourceManager and NodeManager generate logs and store them in a .log
file locally inside the folder specified using the HADOOP_LOGS_DIR
variable. By default, the logs are stored in HADOOP_PREFIX/logs
. All the state transitions in YARN are recorded in the log files. In this section, we'll cover few state transitions and the logs generated during those transitions.
Note
Setting the log level: Hadoop-YARN uses Apache Log4j
library and it uses a log4j.properties
file located in the configuration folder of the Hadoop-YARN bundle at HADOOP_PREFIX/etc/hadoop
.
The Log4j
library supports six log levels – TRACE
, DEBUG
, INFO
, WARN
, ERROR
, and FATAL
. A cluster administrator sets the log level for Hadoop-YARN services and the default log level is INFO
. The hadoop.root.logger
property is used to update the log level for Hadoop-YARN services. To read more about Apache Log4j
library, you can refer to the official site at http://logging.apache.org/log4j...