4. Optimizing the HBase/Hadoop Cluster | Learning Hbase

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

Learning Hbase

By : Shashwat Shriparv

4.1 (8)

Learning Hbase

4.1 (8)

By: Shashwat Shriparv

Overview of this book

If you are an administrator or developer who wants to enter the world of Big Data and BigTables and would like to learn about HBase, this is the book for you.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Free Chapter

1. Understanding the HBase Ecosystem

1. Understanding the HBase Ecosystem

HBase layout on top of Hadoop

Comparing architectural differences between RDBMs and HBase

HBase features

HBase in the Hadoop ecosystem

Comparing functional differences between RDBMs and HBase

About the internal storage architecture of HBase

Getting started with HBase

Applications of HBase

HBase pros and cons

Summary

2. Let's Begin with HBase

2. Let's Begin with HBase

Understanding HBase components in detail

Reading and writing cycle

HBase housekeeping

The HBase delete request

List of available HBase distributions

Prerequisites and capacity planning for HBase

Summary

3. Let's Start Building It

3. Let's Start Building It

Downloading Java on Ubuntu

Considering host configurations

Installing and configuring SSH

Installing and configuring NTP

Performing capacity planning

Installing and configuring Hadoop

Hadoop start up steps

Configuring Apache HBase

Installing and configuring ZooKeeper

Installing Cloudera Hadoop and HBase

Installing the Hadoop and MapReduce packages

Installing Hadoop on Windows

Summary

4. Optimizing the HBase/Hadoop Cluster

4. Optimizing the HBase/Hadoop Cluster

Setup types for Hadoop and HBase clusters

Recommendations for CDH cluster configuration

Capacity planning

Hadoop optimization

Optimizing HBase

Optimizing ZooKeeper

Important files in Hadoop

Important files in HBase

Summary

5. The Storage, Structure Layout, and Data Model of HBase

5. The Storage, Structure Layout, and Data Model of HBase

Data types in HBase

Storing data in HBase – logical view versus actual physical view

Services of HBase

Data model operations

Versioning and why

Deciding the number of the version

Schema designing

Calculating the data size stored in HBase

Summary

6. HBase Cluster Maintenance and Troubleshooting

6. HBase Cluster Maintenance and Troubleshooting

Hadoop shell commands

HBase shell commands

HBase administration tools

Writing HBase shell scripts

Using the Hadoop tool or JARs for HBase

Connecting HBase with Hive

HBase region management

HBase node management

Implementing security

Troubleshooting the most frequent HBase errors and their explanations

Summary

7. Scripting in HBase

7. Scripting in HBase

HBase backup and restore techniques

HBase on Windows

Scripting in HBase

Contributing to HBase

Summary

8. Coding HBase in Java

8. Coding HBase in Java

Setting up the environment for development

Data types

Data model Java operations

HBase filters

Client APIs

Summary

9. Advance Coding in Java for HBase

9. Advance Coding in Java for HBase

Interfaces, classes, and exceptions

Code related to administrative tasks

Data operation code

MapReduce and HBase

RESTful services and Thrift services interface

Coding for HDFS operations

Some advance topics in brief

Summary

10. HBase Use Cases

10. HBase Use Cases

HBase in industry today

The future of HBase against relational databases

Some real-world project examples' use cases

Useful links and references

Summary

Index

Index

Optimizing ZooKeeper

As mentioned in Chapter 1, Understanding the HBase Ecosystem, ZooKeeper provides distributed synchronization and group service to HBase. It is one of the necessities of HBase, and hence, we have to optimize it. Use the following setting for optimization:

<property>
  <name>zookeeper.session.timeout</name>
  <value>3000</value>
</property>

The default value for this setting is 3 minutes. This decides how often master should check for server crashes. We can decrease it so that the server crashes can be noticed quickly, but if this value decreases, we need to take care of GC. In the case of full GC, the server might not respond while running fine, and it might be reported as crashed. This configuration can be overridden in the hbase-site.xml file.

This value should be increased if there is a timeout while writing to the HBase cluster. If it is too small, and while writing huge amounts of data to HBase, GC happens, resulting in the pause...

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Learning Hbase

Search

Your notes and bookmarks