Book Image

Statistics for Machine Learning

By : Pratap Dangeti
Book Image

Statistics for Machine Learning

By: Pratap Dangeti

Overview of this book

Complex statistics in machine learning worry a lot of developers. Knowing statistics helps you build strong machine learning models that are optimized for a given problem statement. This book will teach you all it takes to perform the complex statistical computations that are required for machine learning. You will gain information on the statistics behind supervised learning, unsupervised learning, reinforcement learning, and more. You will see real-world examples that discuss the statistical side of machine learning and familiarize yourself with it. You will come across programs for performing tasks such as modeling, parameter fitting, regression, classification, density collection, working with vectors, matrices, and more. By the end of the book, you will have mastered the statistics required for machine learning and will be able to apply your new skills to any sort of industry problem.

Related Content you might be interested in

Current Title:

Small book image
Statistics for Machine Learning
Pratap Dangeti
Jul, 2017 | 442 pages
Small book image
Mastering Machine Learning with scikit-learn
Gavin Hackeling
Jul, 2017 | 254 pages
Small book image
Ensemble Machine Learning Cookbook
Vijayalakshmi Natarajan | Dipayan Sarkar
Jan, 2019 | 336 pages
Small book image
Hands-On Automated Machine Learning
Sibanjan Das | Umit Mert Cakmak
Apr, 2018 | 282 pages
Table of Contents (16 chapters)
Title Page
Title Page
Credits
Credits
About the Author
About the Author
About the Reviewer
About the Reviewer
www.PacktPub.com
www.PacktPub.com
Customer Feedback
Customer Feedback
Preface
Preface
Free Chapter
1
Journey from Statistics to Machine Learning
Journey from Statistics to Machine Learning
Statistical terminology for model building and validation
Machine learning terminology for model building and validation
Machine learning model overview
Summary
2
Parallelism of Statistics and Machine Learning
Parallelism of Statistics and Machine Learning
Comparison between regression and machine learning models
Compensating factors in machine learning models
Machine learning models - ridge and lasso regression
Summary
3
Logistic Regression Versus Random Forest
Logistic Regression Versus Random Forest
Maximum likelihood estimation
Logistic regression – introduction and advantages
Random forest
Variable importance plot
Comparison of logistic regression with random forest
Summary
4
Tree-Based Machine Learning Models
Tree-Based Machine Learning Models
Introducing decision tree classifiers
Comparison between logistic regression and decision trees
Comparison of error components across various styles of models
Remedial actions to push the model towards the ideal region
HR attrition data example
Decision tree classifier
Tuning class weights in decision tree classifier
Bagging classifier
Random forest classifier
Random forest classifier - grid search
AdaBoost classifier
Gradient boosting classifier
Comparison between AdaBoosting versus gradient boosting
Extreme gradient boosting - XGBoost classifier
Ensemble of ensembles - model stacking
Ensemble of ensembles with different types of classifiers
Ensemble of ensembles with bootstrap samples using a single type of classifier
Summary
5
K-Nearest Neighbors and Naive Bayes
K-Nearest Neighbors and Naive Bayes
K-nearest neighbors
KNN classifier with breast cancer Wisconsin data example
Tuning of k-value in KNN classifier
Naive Bayes
Probability fundamentals
Understanding Bayes theorem with conditional probability
Naive Bayes classification
Laplace estimator
Naive Bayes SMS spam classification example
Summary
6
Support Vector Machines and Neural Networks
Support Vector Machines and Neural Networks
Support vector machines working principles
Kernel functions
SVM multilabel classifier with letter recognition data example
Artificial neural networks - ANN
Activation functions
Forward propagation and backpropagation
Optimization of neural networks
Dropout in neural networks
ANN classifier applied on handwritten digits using scikit-learn
Introduction to deep learning
Summary
7
Recommendation Engines
Recommendation Engines
Content-based filtering
Collaborative filtering
Evaluation of recommendation engine model
8
Unsupervised Learning
Unsupervised Learning
K-means clustering
Principal component analysis - PCA
Singular value decomposition - SVD
Deep auto encoders
Model building technique using encoder-decoder architecture
Deep auto encoders applied on handwritten digits using Keras
Summary
9
Reinforcement Learning
Reinforcement Learning
Introduction to reinforcement learning
Comparing supervised, unsupervised, and reinforcement learning in detail
Characteristics of reinforcement learning
Reinforcement learning basics
Markov decision processes and Bellman equations
Dynamic programming
Grid world example using value and policy iteration algorithms with basic Python
Monte Carlo methods
Temporal difference learning
SARSA on-policy TD control
Q-learning - off-policy TD control
Cliff walking example of on-policy and off-policy of TD control
Applications of reinforcement learning with integration of machine learning and deep learning
Further reading
Summary

Optimization of neural networks

Various techniques have been used for optimizing the weights of neural networks:

  • Stochastic gradient descent (SGD)
  • Momentum
  • Nesterov accelerated gradient (NAG)
  • Adaptive gradient (Adagrad)
  • Adadelta
  • RMSprop
  • Adaptive moment estimation (Adam)
  • Limited memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS)

In practice, Adam is a good default choice; we will be covering its working methodology in this section. If you cannot afford full batch updates, then try out L-BFGS:

Stochastic gradient descent - SGD

Gradient descent is a way to minimize an objective function J(θ) parameterized by a model's parameter θ ε Rd by updating the parameters in the opposite direction of the gradient of the objective function with regard to the parameters. The learning rate determines the size of the steps taken to reach the minimum:

  • Batch gradient descent (all training observations utilized in each iteration)
  • SGD (one observation per iteration)
  • Mini batch gradient descent (size of about 50 training observations...
Previous Section
End of Section 8
Next Section