"There's nothing that cannot be found through some search engine or on the Internet somewhere." | ||
| --Eric Schmidt, Executive Chairman, Google | ||
Hadoop is the de facto open source framework used in the industry for large scale, massively parallel, and distributed data processing. It provides a computation layer for parallel and distributed computation processing. Closely associated with the computation layer is a highly fault-tolerant data storage layer, the Hadoop Distributed File System (HDFS). Both the computation and data layers run on commodity hardware, which is inexpensive, easily available, and compatible with other similar hardware.
In this chapter, we will look at the journey of Hadoop, with a focus on the features that make it enterprise-ready. Hadoop, with 6 years of development and deployment under its belt, has moved from a framework that supports the MapReduce paradigm exclusively to a more generic cluster-computing framework. This chapter covers the following topics:
An outline of Hadoop's code evolution, with major milestones highlighted
An introduction to the changes that Hadoop has undergone as it has moved from 1.X releases to 2.X releases, and how it is evolving into a generic cluster-computing framework
An introduction to the options available for enterprise-grade Hadoop, and the parameters for their evaluation
An overview of a few popular enterprise-ready Hadoop distributions