Book Image

Deep Learning with Hadoop

By : Dipayan Dev
Book Image

Deep Learning with Hadoop

By: Dipayan Dev

Overview of this book

This book will teach you how to deploy large-scale dataset in deep neural networks with Hadoop for optimal performance. Starting with understanding what deep learning is, and what the various models associated with deep neural networks are, this book will then show you how to set up the Hadoop environment for deep learning. In this book, you will also learn how to overcome the challenges that you face while implementing distributed deep learning with large-scale unstructured datasets. The book will also show you how you can implement and parallelize the widely used deep learning models such as Deep Belief Networks, Convolutional Neural Networks, Recurrent Neural Networks, Restricted Boltzmann machines and autoencoder using the popular deep learning library Deeplearning4j. Get in-depth mathematical explanations and visual representations to help you understand the design and implementations of Recurrent Neural network and Denoising Autoencoders with Deeplearning4j. To give you a more practical perspective, the book will also teach you the implementation of large-scale video processing, image processing and natural language processing on Hadoop. By the end of this book, you will know how to deploy various deep neural networks in distributed systems using Hadoop.
Table of Contents (16 chapters)
Deep Learning with Hadoop
Credits
About the Author
About the Reviewers
www.PacktPub.com
Customer Feedback
Dedication
Preface
References

Deep learning for massive amounts of data


In this Exa-Byte scale era, the data are increasing at an exponential rate. This growth of data are analyzed by many organizations and researchers in various ways, and also for so many different purposes. According to the survey of International Data Corporation (IDC), the Internet is processing approximately 2 Petabytes of data every day [51]. In 2006, the size of digital data was around 0.18 ZB, whereas this volume has increased to 1.8 ZB in 2011. Up to 2015, it was expected to reach up to 10 ZB in size, and by 2020, its volume in the world will reach up to approximately 30 ZB to 35 ZB. The timeline of this data mountain is shown in Figure 2.1. These immense amounts of data in the digital world are formally termed as big data.

 

"The world of Big Data is on fire"

 
 --The Economist, Sept 2011

Figure 2.1: Figure shows the increasing trend of data for a time span of around 20 years

Facebook has almost 21 PB in 200M objects [52], whereas Jaguar ORNL...