Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Learning Hbase
  • Table Of Contents Toc
Learning Hbase

Learning Hbase

By : Shashwat Shriparv
4.1 (8)
close
close
Learning Hbase

Learning Hbase

4.1 (8)
By: Shashwat Shriparv

Overview of this book

If you are an administrator or developer who wants to enter the world of Big Data and BigTables and would like to learn about HBase, this is the book for you.
Table of Contents (12 chapters)
close
close
11
Index

Capacity planning


Suppose we have around 2 TB data with a replication factor of 3, which means 3 * 2 = 6 TB, which in turn means that 2 TB of extra space is still needed. So, for 2 TB of data, we can have a cluster with 4 to 8 DataNodes, totaling 8 TB of storage disk.

This extra space is needed for an intermediate temporary file that is generated during read/write operations and MapReduce jobs. If the data on which we run MapReduce is huge and the MapReduce code processes the whole data that requires a huge HDFS storage to store the temporary and intermediate result files, we will need to provide enough disk storage, the absence of which will result in a lot of failing tasks and blacklisted nodes. It is advisable to have 25 to 50 percent more storage than the original data size (without a replication factor) on the cluster; the minimum should be 25 percent more of the whole data size if we want to run a MapReduce task without much failing.

So, we can apply an approximate formula, as follows...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Learning Hbase
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon