Book Image

Programming MapReduce with Scalding

By : Antonios Chalkiopoulos
Book Image

Programming MapReduce with Scalding

By: Antonios Chalkiopoulos

Overview of this book

Table of Contents (16 chapters)
Programming MapReduce with Scalding
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Summary


In this chapter, we presented some design patterns for solving common problems. Initially, we presented the idea that a job should not contain complex logic. The job has defined responsibilities and delegates all the complexity of data transformations to external operations implemented in a trait. By keeping all schemas in an object and using a wrapper, we can structure our code in a modular and reusable way.

The dependency injection pattern presented how to inject dependencies at compile time, and the late bound pattern displayed how to overcome situations where an object cannot be serialized over the network.

In the next chapter, we will present the various testing strategies around our Scalding data-processing applications.