Book Image

Programming MapReduce with Scalding

By : Antonios Chalkiopoulos
Book Image

Programming MapReduce with Scalding

By: Antonios Chalkiopoulos

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
Programming MapReduce with Scalding
Antonios Chalkiopoulos
Jun, 2014 | 148 pages
No titles found
Table of Contents (16 chapters)
Programming MapReduce with Scalding
Programming MapReduce with Scalding
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Introduction to MapReduce
Introduction to MapReduce
The Hadoop platform
MapReduce
MapReduce abstractions
Introducing Cascading
Summary
2
Get Ready for Scalding
Get Ready for Scalding
Why Scala?
Scala basics
Scala build tools
Hello World in Scala
Development editors
Installing Hadoop in five minutes
Running our first Scalding job
Submitting a Scalding job in Hadoop
Summary
3
Scalding by Example
Scalding by Example
Reading and writing files
Understanding the core capabilities of Scalding
Operations on groups
A simple example
Typed API
Summary
4
Intermediate Examples
Intermediate Examples
Logfile analysis
Exploring ad targeting
Summary
5
Scalding Design Patterns
Scalding Design Patterns
The external operations pattern
The dependency injection pattern
The late bound dependency pattern
Summary
6
Testing and TDD
Testing and TDD
Introduction to testing
MapReduce testing challenges
Development lifecycle with testing strategy
TDD for Scalding developers
Black box testing
Summary
7
Running Scalding in Production
Running Scalding in Production
Executing Scalding in a Hadoop cluster
Scheduling execution
Coordinating job execution
Configuring using a property file
Configuring using Hadoop parameters
Monitoring Scalding jobs
Using slim JAR files
Scalding execution throttling
Summary
8
Using External Data Stores
Using External Data Stores
Interacting with external systems
SQL databases
NoSQL databases
Search platforms
Summary
9
Matrix Calculations and Machine Learning
Matrix Calculations and Machine Learning
Text similarity using TF-IDF
Setting a similarity using the Jaccard index
K-Means using Mahout
Other libraries
Summary
Index
Index

About the Author

Antonios Chalkiopoulos is a developer living in London and a professional working with Hadoop and Big Data technologies. He completed a number of complex MapReduce applications in Scalding into 40-plus production nodes HDFS Cluster. He is a contributor to Scalding and other open source projects, and he is interested in cloud technologies, NoSQL databases, distributed real-time computation systems, and machine learning.

He was involved in a number of Big Data projects before discovering Scala and Scalding. Most of the content of this book comes from his experience and knowledge accumulated while working with a great team of engineers.

I would like to thank Rajah Chandan for introducing Scalding to the team and being the author of SpyGlass and Stefano Galarraga for co-authoring chapters 5 and 6 and being the author of ScaldingUnit. Both these libraries are presented in this book.

Saad, Gracia, Deepak, and Tamas, I've learned a lot working next to you all, and this book wouldn't be possible without all your discoveries. Finally, I would like to thank Christina for bearing with my writing sessions and supporting all my endeavors.

Previous Section
Next Chapter