Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Optimizing Hadoop for MapReduce
  • Table Of Contents Toc
Optimizing Hadoop for MapReduce

Optimizing Hadoop for MapReduce

By : Khaled Tannir
4.3 (4)
close
close
Optimizing Hadoop for MapReduce

Optimizing Hadoop for MapReduce

4.3 (4)
By: Khaled Tannir

Overview of this book

Table of Contents (15 chapters)
close
close

Using appropriate Writable types


Hadoop uses custom datatype serialization/RPC mechanism and defines its own box type classes. These classes are used to manipulate strings (Text), integers (IntWritable), and so on, and they implement the Writable class, which defines a deserialization protocol.

Therefore, all values in Hadoop are Writable type objects and all keys are instances of WritableComparable, which defines a sort order, because they need to be compared.

Writable objects are mutable and considerably more compact as no meta info needs to be stored (class name, fields, super classes, and so on), and straightforward random access gives higher performance. As binary Writable types will take up less space, this will reduce the size of intermediate data written by the Map or Combiner function. Reducing intermediate data can provide a substantial performance gain by reducing network transfer and I/O disk required storage space.

Using the appropriate Writable type in your code will contribute...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Optimizing Hadoop for MapReduce
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon