Hadoop uses a set of counters to aggregate the metrics for MapReduce computations. Hadoop counters are helpful to understand the behavior of our MapReduce programs and to track the progress of the MapReduce computations. We can define custom counters to track the application-specific metrics in MapReduce computations.
The following steps show you how to define a custom counter to count the number of bad or corrupted records in our log processing application:
Define the list of custom counters using
enum:public static enum LOG_PROCESSOR_COUNTER { BAD_RECORDS };Increment the counter in your Mapper or Reducer:
context.getCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS).increment(1);
Add the following to your driver program to access the counters:
Job job = new Job(getConf(), "log-analysis"); …… Counters counters = job.getCounters(); Counter badRecordsCounter = counters.findCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS); System.out.println...