While developing Hadoop MapReduce applications, you may encounter scenarios where your MapReduce application requires a newer version of an auxiliary library that is already included in Hadoop. By default, Hadoop gives classpath precedence to the libraries included with Hadoop, which can result in conflicts with the version of the library you provide with your applications. This recipe shows you how to configure Hadoop to give classpath precedence to user-provided libraries.
The following steps show you how to add external libraries to the Hadoop task classpath and how to provide precedence to user-supplied JARs:
Set the following property in the driver program of your MapReduce computation:
job.getConfiguration().set("mapreduce.job.user.classpath.first","true");Use the
–libjarsoption in the Hadoop command to provide your libraries, as follows:$hadoop jar hcb-c3-samples.jar \ chapter3.WordCountWithTools \ –libjars guava-15...