Audit logging might be required for data processing systems such as Hadoop. In Hadoop, audit logging has been implemented using the Log4j Java logging library at the INFO
logging level. By default, Hadoop audit logging is disabled. This recipe will guide you through the steps to configure Hadoop audit logging.
We assume that our Hadoop cluster has been configured properly.
Log in to the master node from the administrator machine using the following command:
ssh hduser@master
Perform the following steps to configure Hadoop audit logging:
Enable audit logging by changing the following line in the
$HADOOP_HOME/conf/log4j.properties
file from:log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=WARN
to the following:
log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=INFO
Try making a directory on HDFS with the following command:
hadoop fs -mkdir audittest
Check the audit log messages in the NameNode...