Book Image

Optimizing Hadoop for MapReduce

By : Khaled Tannir
Book Image

Optimizing Hadoop for MapReduce

By: Khaled Tannir

Overview of this book

Table of Contents (15 chapters)

Hardware tuning and OS recommendations


Recommendations for system tuning depend on the intrinsic capabilities of the system. The following sections suggest different recommendation techniques and tips that you can use as reminder baselines when engaging in your MapReduce optimization process.

The Hadoop cluster checklist

The following checklist describes only the minimal set of steps required to get your Hadoop cluster working optimally:

  • Check and ensure that all cluster nodes can communicate with each other and you have physical and/or remote management access to each cluster node

  • Check whether your cluster is well dimensioned and is able to compensate a failure of (at least) one node per service

  • Check the limitations of your cluster environment (hardware availability resources/rack space, hosting parameters, and so on)

  • Define your cluster strategies for a failover to ensure high availability of your services

  • Define what you need to back up, and what needs to be saved and where, in order to maximize...