When reading input, or writing output from a MapReduce application, it is sometimes easier to work with data using an abstract class instead of the primitive
Hadoop Writable classes (for example, Text
and IntWritable
). This recipe demonstrates how to create a custom Hadoop Writable and InputFormat that can be used by MapReduce applications.
You will need to download the Nigeria_ACLE
D_cleaned.tsv
dataset from http://www.packtpub.com/support and place the file into HDFS.
Follow these steps to create custom InputFormat and Writable classes:
First we will define two custom
WritableComparable
classes. These classes represent the key-value pairs that are passed to the mapper, much as howTextInputFormat
passesLongWritable
andText
to the mapper.Write the key class:
public class GeoKey implements WritableComparable { private Text location; private FloatWritable latitude; private...