Book Image

Deep Learning with TensorFlow - Second Edition

By : Giancarlo Zaccone, Md. Rezaul Karim
Book Image

Deep Learning with TensorFlow - Second Edition

By: Giancarlo Zaccone, Md. Rezaul Karim

Overview of this book

Deep learning is a branch of machine learning algorithms based on learning multiple levels of abstraction. Neural networks, which are at the core of deep learning, are being used in predictive analytics, computer vision, natural language processing, time series forecasting, and to perform a myriad of other complex tasks. This book is conceived for developers, data analysts, machine learning practitioners and deep learning enthusiasts who want to build powerful, robust, and accurate predictive models with the power of TensorFlow, combined with other open source Python libraries. Throughout the book, you’ll learn how to develop deep learning applications for machine learning systems using Feedforward Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, Autoencoders, and Factorization Machines. Discover how to attain deep learning programming on GPU in a distributed way. You'll come away with an in-depth knowledge of machine learning techniques and the skills to apply them to real-world projects.
Table of Contents (15 chapters)
Deep Learning with TensorFlow - Second Edition
Contributors
Preface
Other Books You May Enjoy
Index

Preface

Every week, we follow news of applications and the shocking results obtained from them, thanks to the artificial intelligence algorithms applied in different fields. What we are witnessing is one of the biggest accelerations in the entire history of this sector, and the main suspect behind these important developments is called deep learning.

Deep learning comprises a vast set of algorithms that are based on the concept of neural networks and expand to contain a huge number of nodes that are disseminated at several levels of depth.

Though the concept of neural networks, the so-called Artificial Neural Network (ANN), dates back to the late 1940s, initially, they were difficult to be used because of the need for huge computational power resources and the lack of data required to train the algorithms. Presently, the ability to use graphics processors (GPUs) in parallel to perform intensive calculation operations has completely opened the way to the use of deep learning.

In this context, we propose the second edition of this book, with expanded and revised contents that introduce the core concepts of deep learning, using the last version of TensorFlow.

TensorFlow is Google's open-source framework for the mathematical, Machine Learning, and Deep Learning capabilities, released in 2011. Subsequently, TensorFlow has been widely adopted in academia, research, and industry. Recently, the most stable version 1.6 has been released with a unified API. The most stable version of TensorFlow at the time of writing was version 1.6, which was released with a unified API and is thus a significant and stable version in the TensorFlow roadmap. This book also discusses and is compliant with the pre-release version, 1.7, which was available during the production stages of this book.

TensorFlow provides the flexibility needed to implement and research cutting-edge architectures, while allowing users to focus on the structure of their models as opposed to mathematical details.

You will learn deep learning programming techniques with hands-on model building, data collection, transformation, and much more!

Enjoy reading!

Who this book is for

This book is dedicated to developers, data analysts, and deep learning enthusiasts who do not have much background with complex numerical computations, but want to know what deep learning is. The book majorly appeals to beginners who are looking for a quick guide to gain some hands-on experience with deep learning.

What this book covers

Chapter 1, Getting Started with Deep Learning, covers the concepts that will be found in all the subsequent chapters. The basics of machine learning and deep learning are also discussed. We will also look at Deep learning architectures that are distinguished from the more commonplace single-hidden-layer neural networks by their depth, that is, the number of node layers through which data passes in a multistep process of pattern recognition. We will also analyze these architectures with a chart summarizing all the neural networks from where most of the deep learning algorithm evolved. The chapter ends with an analysis of the major deep learning frameworks.

Chapter 2, A First Look at TensorFlow, gives a detailed description of the main TensorFlow features based on a real-life problem, followed by a detailed discussion on TensorFlow installation and configurations. We then look at a computation graph, data, and programming model before getting started with TensorFlow. Toward the end of the chapter, we will look at an example of implementing the linear regression model for predictive analytics.

Chapter 3, Feed-Forward Neural Networks with TensorFlow, demonstrates the theoretical background of different Feed-Forward Neural Networks' (FFNNs) architectures such as Deep Belief Networks (DBNs) and Multilayer Perceptron (MLP). We will then see how to train and analyze the performance metrics that are needed to evaluate the models; also, how to tune the hyperparameters for FFNNs for better and optimized performance. We will also look at two examples using MLP and DBN on how to build very robust and accurate predictive models for predictive analytics on a bank marketing dataset.

Chapter 4, Convolutional Neural Networks, introduces the networks of CNNs that are the basic blocks of a Deep Learning-based image classifier. We will consider the most important CNN architectures, such as Lenet, AlexNet, Vgg, and Inception with hands-on examples, specifically for AlexNet and Vgg. We will then examine the transfer learning and style learning techniques. We will end the chapter by developing a CNN to train a network on a series of facial images to classify their emotional stretch.

Chapter 5, Optimizing TensorFlow Autoencoders, provides sound theoretical background on optimizing autoencoders for data denoising and dimensionality reduction. We will then look at how to implement an autoencoder, gradually moving over to more robust autoencoder implementation, such as denoising autoencoders and convolutional autoencoders. Finally, we will look at a real-life example of fraud analytics using an autoencoder.

Chapter 6, Recurrent Neural Networks, provides some theoretical background of RNNs. We will also look at a few examples for implementing predictive models for classification of images, sentiment analysis of movies, and products spam prediction for NLP. Finally, we'll see how to develop predictive models for time series data.

Chapter 7, Heterogeneous and Distributed Computing, shows the fundamental topic to execute TensorFlow models on GPU cards and distributed systems. We will also look at basic concepts with application examples.

Chapter 8, Advanced TensorFlow Programming, gives an overview of the following TensorFlow-based libraries: tf.contrib.learn, Pretty Tensor, TFLearn, and Keras. For each library, we will describe the main features with applications.

Chapter 9, Recommendation Systems using Factorization Machines, provides several examples on how to develop recommendation system for predictive analytics followed by some theoretical background of recommendation systems. We will then look at an example of developing a movie recommendation engine using collaborative filtering and K-means. Considering the limitations of classical approaches, we'll see how to use Neural Factorization Machines for developing more accurate and robust recommendation systems.

Chapter 10, Reinforcement Learning, covers the basic concepts of RL. We will experience the Q-learning algorithm, which is one of the most popular reinforcement learning algorithms. Furthermore, we'll introduce the OpenAI gym framework that is a TensorFlow compatible toolkit for developing and comparing reinforcement learning algorithms. We end the chapter with the implementation of a Deep Q-Learning algorithm to resolve the cart-pole problem.

To get the most out of this book

  • A rudimentary level of programming in one language is assumed, as is a basic familiarity with computer science techniques and technologies, including a basic awareness of computer hardware and algorithms. Some competence in mathematics is needed to the level of elementary linear algebra and calculus.

  • Software: Python 3.5.0, Pip, pandas, numpy, tensorflow, Matplotlib 2.1.1, IPython, Scipy 0.19.0, sklearn, seaborn, tffm, and many more

  • Step: Issue the following command on Terminal on Ubuntu:

    $ sudo pip3 install pandas numpy tensorflow sklearn seaborn tffm
    

    Nevertheless, installing guidelines are provided in the chapters.

Download the example code files

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

  1. Log in or register at http://www.packtpub.com.

  2. Select the SUPPORT tab.

  3. Click on Code Downloads & Errata.

  4. Enter the name of the book in the Search box and follow the on-screen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of any of the following:

  • WinRAR / 7-Zip for Windows

  • Zipeg / iZip / UnRarX for macOS

  • 7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Deep-Learning-with-TensorFlow-Second-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://www.packtpub.com/sites/default/files/downloads/DeepLearningwithTensorFlowSecondEdition_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. For example; " This means that using tf.enable_eager_execution() is recommended."

A block of code is set as follows:

import tensorflow as tf # Import TensorFlow

x = tf.constant(8) # X op
y = tf.constant(9) # Y op
z = tf.multiply(x, y) # New op Z

sess = tf.Session() # Create TensorFlow session

out_z = sess.run(z) # execute Z op
sess.close() # Close TensorFlow session
print('The multiplication of x and y: %d' % out_z)# print result

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

import tensorflow as tf # Import TensorFlow

x = tf.constant(8) # X op
y = tf.constant(9) # Y op
z = tf.multiply(x, y) # New op Z

sess = tf.Session() # Create TensorFlow session

out_z = sess.run(z) # execute Z op
sess.close() # Close TensorFlow session
print('The multiplication of x and y: %d' % out_z)# print result

Any command-line input or output is written as follows:

>>>
MSE: 27.3749

Bold: Indicates a new term, an important word, or words that you see on the screen, for example, in menus or dialog boxes, also appear in the text like this. For example: " Now let's move to http://localhost:6006 and on click on the GRAPH tab."

Note

Warnings or important notes appear in a box like this.

Note

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email , and mention the book's title in the subject of your message. If you have questions about any aspect of this book, please email us at .

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book we would be grateful if you would report this to us. Please visit, http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit http://authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packtpub.com.