Book Image

Hands-On Mathematics for Deep Learning

By : Jay Dawani
Book Image

Hands-On Mathematics for Deep Learning

By: Jay Dawani

Overview of this book

Most programmers and data scientists struggle with mathematics, having either overlooked or forgotten core mathematical concepts. This book uses Python libraries to help you understand the math required to build deep learning (DL) models. You'll begin by learning about core mathematical and modern computational techniques used to design and implement DL algorithms. This book will cover essential topics, such as linear algebra, eigenvalues and eigenvectors, the singular value decomposition concept, and gradient algorithms, to help you understand how to train deep neural networks. Later chapters focus on important neural networks, such as the linear neural network and multilayer perceptrons, with a primary focus on helping you learn how each model works. As you advance, you will delve into the math used for regularization, multi-layered DL, forward propagation, optimization, and backpropagation techniques to understand what it takes to build full-fledged DL models. Finally, you’ll explore CNN, recurrent neural network (RNN), and GAN models and their application. By the end of this book, you'll have built a strong foundation in neural networks and DL mathematical concepts, which will help you to confidently research and build custom models in DL.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Mathematics for Deep Learning
Jay Dawani
Jun, 2020 | 364 pages
Small book image
Hands-On Java Deep Learning for Computer Vision
Klevis Ramo
Feb, 2019 | 260 pages
Small book image
Hands-On One-shot Learning with Python
Ankush Garg | Shruti Jadon
Apr, 2020 | 156 pages
Small book image
Advanced Deep Learning with Python
Ivan Vasilev
Dec, 2019 | 468 pages
Table of Contents (19 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
1
Section 1: Essential Mathematics for Deep Learning
Section 1: Essential Mathematics for Deep Learning
Free Chapter
2
Linear Algebra
Linear Algebra
Comparing scalars and vectors
Linear equations
Matrix operations
Vector spaces and subspaces
Linear maps
Matrix decompositions
Summary
3
Vector Calculus
Vector Calculus
Single variable calculus
Multivariable calculus
Vector calculus
Summary
4
Probability and Statistics
Probability and Statistics
Understanding the concepts in probability
Essential concepts in statistics
Summary
5
Optimization
Optimization
Understanding optimization and it's different types
Exploring the various optimization methods
Exploring population methods
Summary
6
Graph Theory
Graph Theory
Understanding the basic concepts and terminology
Adjacency matrix
Types of graphs
Graph Laplacian
Summary
7
Section 2: Essential Neural Networks
Section 2: Essential Neural Networks
8
Linear Neural Networks
Linear Neural Networks
Linear regression
Polynomial regression
Logistic regression
Summary
9
Feedforward Neural Networks
Feedforward Neural Networks
Understanding biological neural networks
Comparing the perceptron and the McCulloch-Pitts neuron
MLPs
Training neural networks
Deep neural networks
Summary
10
Regularization
Regularization
The need for regularization
Norm penalties
Early stopping
Parameter tying and sharing
Dataset augmentation
Dropout
Adversarial training
Summary
11
Convolutional Neural Networks
Convolutional Neural Networks
The inspiration behind ConvNets
Types of data used in ConvNets
Convolutions and pooling
Working with the ConvNet architecture
Training and optimization
Exploring popular ConvNet architectures
Summary
12
Recurrent Neural Networks
Recurrent Neural Networks
The need for RNNs
The types of data used in RNNs
Understanding RNNs
Long short-term memory
Gated recurrent units
Deep RNNs
Training and optimization
Popular architecture
Summary
13
Section 3: Advanced Deep Learning Concepts Simplified
Section 3: Advanced Deep Learning Concepts Simplified
14
Attention Mechanisms
Attention Mechanisms
Overview of attention
Understanding neural Turing machines
Exploring the types of attention
Transformers
Summary
15
Generative Models
Generative Models
Why we need generative models
Autoencoders
Generative adversarial networks
Flow-based networks
Summary
16
Transfer and Meta Learning
Transfer and Meta Learning
Transfer learning
Meta learning
Summary
17
Geometric Deep Learning
Geometric Deep Learning
Comparing Euclidean and non-Euclidean data
Graph neural networks
Spectral graph CNNs
Mixture model networks
Facial recognition in 3D
Summary
18
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Exploring the types of attention

Attention has proven to be so effective in machine translation that it has been expanded into natural language processing, statistical learning, speech understanding, object detection and recognition, image captioning, and visual question answering.

The purpose of attention is to estimate how correlated (connected) two or more elements are to one another.

However, there isn't just one kind of attention. There are many types, such as the following:

  • Self-attention: Captures the relationship between different positions of a sequence of inputs
  • Global or soft attention: Focuses on the entire sequence of inputs
  • Local or hard attention: Focuses on only part of the sequence of inputs

Let's take a look at these in more detail.

Self-attention

...
Previous Section
End of Section 4
Next Section