Book Image

Mastering Machine Learning Algorithms

Book Image

Mastering Machine Learning Algorithms

Overview of this book

Machine learning is a subset of AI that aims to make modern-day computer systems smarter and more intelligent. The real power of machine learning resides in its algorithms, which make even the most difficult things capable of being handled by machines. However, with the advancement in the technology and requirements of data, machines will have to be smarter than they are today to meet the overwhelming data needs; mastering these algorithms and using them optimally is the need of the hour. Mastering Machine Learning Algorithms is your complete guide to quickly getting to grips with popular machine learning algorithms. You will be introduced to the most widely used algorithms in supervised, unsupervised, and semi-supervised machine learning, and will learn how to use them in the best possible manner. Ranging from Bayesian models to the MCMC algorithm to Hidden Markov models, this book will teach you how to extract features from your dataset and perform dimensionality reduction by making use of Python-based libraries such as scikit-learn v0.19.1. You will also learn how to use Keras and TensorFlow 1.x to train effective neural networks. If you are looking for a single resource to study, implement, and solve end-to-end machine learning problems and use-cases, this is the book you need.

Related Content you might be interested in

Current Title:

Small book image
Mastering Machine Learning Algorithms
May, 2018 | 576 pages
Small book image
Hands-On Unsupervised Learning with Python
Giuseppe Bonaccorso
Feb, 2019 | 386 pages
Small book image
Machine Learning Algorithms
Giuseppe Bonaccorso
Aug, 2018 | 522 pages
Small book image
Hands-On Markov Models with Python
Ankur Ankan | Abinash Panda
Sep, 2018 | 178 pages
Table of Contents (22 chapters)
Title Page
Title Page
Dedication
Dedication
Packt Upsell
Packt Upsell
Contributors
Contributors
Preface
Preface
Free Chapter
1
Machine Learning Model Fundamentals
Machine Learning Model Fundamentals
Models and data
Features of a machine learning model
Loss and cost functions
Summary
2
Introduction to Semi-Supervised Learning
Introduction to Semi-Supervised Learning
Semi-supervised scenario
Generative Gaussian mixtures
Contrastive pessimistic likelihood estimation
Semi-supervised Support Vector Machines (S3VM)
Transductive Support Vector Machines (TSVM)
Summary
3
Graph-Based Semi-Supervised Learning
Graph-Based Semi-Supervised Learning
Label propagation
Label spreading
Label propagation based on Markov random walks
Manifold learning
Summary
4
Bayesian Networks and Hidden Markov Models
Bayesian Networks and Hidden Markov Models
Conditional probabilities and Bayes' theorem
Bayesian networks
Hidden Markov Models (HMMs)
Summary
5
EM Algorithm and Applications
EM Algorithm and Applications
MLE and MAP learning
EM algorithm
Gaussian mixture
Factor analysis
Principal Component Analysis
Independent component analysis
Addendum to HMMs
Summary
6
Hebbian Learning and Self-Organizing Maps
Hebbian Learning and Self-Organizing Maps
Hebb's rule
Sanger's network
Rubner-Tavan's network
Self-organizing maps
Summary
7
Clustering Algorithms
Clustering Algorithms
k-Nearest Neighbors
K-means
Fuzzy C-means
Spectral clustering
Summary
8
Ensemble Learning
Ensemble Learning
Ensemble learning fundamentals
Random forests
AdaBoost
Gradient boosting
Ensembles of voting classifiers
Ensemble learning as model selection
Summary
9
Neural Networks for Machine Learning
Neural Networks for Machine Learning
The basic artificial neuron
Perceptron
Multilayer perceptrons
Optimization algorithms
Regularization and dropout
Batch normalization
Summary
10
Advanced Neural Models
Advanced Neural Models
Deep convolutional networks
Recurrent networks
Transfer learning
Summary
11
Autoencoders
Autoencoders
Autoencoders
Variational autoencoders
Summary
12
Generative Adversarial Networks
Generative Adversarial Networks
Adversarial training
Wasserstein GAN (WGAN)
Summary
13
Deep Belief Networks
Deep Belief Networks
MRF
RBMs
DBNs
Summary
14
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Reinforcement Learning fundamentals
Policy iteration
Value iteration
TD(0) algorithm
Summary
15
Advanced Policy Estimation Algorithms
Advanced Policy Estimation Algorithms
TD(λ) algorithm
SARSA algorithm
Q-learning
Summary
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index
Index

Batch normalization

Let's consider a mini-batch of k samples:

Before traversing the network, we can measure a mean and a variance:

After the first layer (for simplicity, let's suppose that the activation function, f(•), is the always the same), the batch is transformed into the following:

In general, there's no guarantee that the new mean and variance are the same. On the contrary, it's easy to observe a modification that increases throughout the network. This phenomenon is called covariate shift, and it's responsible for a progressive training speed decay due to the different adaptations needed in each layer. Ioffe and Szegedy (in Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, Ioffe S., Szegedy C., arXiv:1502.03167 [cs.LG]) proposed a method to mitigate this problem, which has been called batch normalization (BN).

The idea is to renormalize the linear output of a layer (before or after applying the activation function), so that the batch has null...

Previous Section
End of Section 7
Next Section