Book Image

Programming MapReduce with Scalding

By : Antonios Chalkiopoulos
Book Image

Programming MapReduce with Scalding

By: Antonios Chalkiopoulos

Overview of this book

Table of Contents (16 chapters)
Programming MapReduce with Scalding
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

The dependency injection pattern


The dependency injection design pattern allows us to remove hard-coded dependencies to make it possible to change them without recompile or at runtime. It enables us to effectively unit test the code, increase its reuse and flexibility, and support application configuration.

The only cost introduced by the pattern is a slightly more complex structure, since we have to expose the dependency and provide it to our underlying code. This is in general completely justified, whenever we can (or we have to) extract part of our logic into independent components.

Dependency injection is based on the external operations pattern. Again, we have a package object as shown:

package object ExampleSchema {
  val LOG_SCHEMA = List('datetime, 'user, 'url)
  val OUTPUT_SCHEMA = List('datetime,'user,'url, 'email, 'address)
}

This time though we will join the users using an external REST API to fetch the email and address information. The interface and a mock implementation can be...