Book Image

Learning Hadoop 2

Book Image

Learning Hadoop 2

Overview of this book

Table of Contents (18 chapters)
Learning Hadoop 2
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Chapter 9. Making Development Easier

In this chapter, we will look at how, depending on use cases and end goals, application development in Hadoop can be simplified using a number of abstractions and frameworks built on top of the Java APIs. In particular, we will learn about the following topics:

  • How the streaming API allows us to write MapReduce jobs using dynamic languages such as Python and Ruby

  • How frameworks such as Apache Crunch and Kite Morphlines allow us to express data transformation pipelines using higher-level abstractions

  • How Kite Data, a promising framework developed by Cloudera, provides us with the ability to apply design patterns and boilerplate to ease integration and interoperability of different components within the Hadoop ecosystem