Book Image

Bayesian Analysis with Python

Book Image

Bayesian Analysis with Python

Overview of this book

The purpose of this book is to teach the main concepts of Bayesian data analysis. We will learn how to effectively use PyMC3, a Python library for probabilistic programming, to perform Bayesian parameter estimation, to check models and validate them. This book begins presenting the key concepts of the Bayesian framework and the main advantages of this approach from a practical point of view. Moving on, we will explore the power and flexibility of generalized linear models and how to adapt them to a wide array of problems, including regression and classification. We will also look into mixture models and clustering data, and we will finish with advanced topics like non-parametrics models and Gaussian processes. With the help of Python and PyMC3 you will learn to implement, check and expand Bayesian models to solve data analysis problems.

Related Content you might be interested in

Current Title:

Small book image
Bayesian Analysis with Python
Nov, 2016 | 282 pages
No titles found
Table of Contents (15 chapters)
Bayesian Analysis with Python
Bayesian Analysis with Python
Credits
Credits
About the Author
About the Author
About the Reviewer
About the Reviewer
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Thinking Probabilistically - A Bayesian Inference Primer
Thinking Probabilistically - A Bayesian Inference Primer
Statistics as a form of modeling
Probabilities and uncertainty
Single parameter inference
Posterior predictive checks
Installing the necessary Python packages
Summary
Exercises
2
Programming Probabilistically – A PyMC3 Primer
Programming Probabilistically – A PyMC3 Primer
Probabilistic programming
PyMC3 introduction
Summarizing the posterior
Summary
Keep reading
Exercises
3
Juggling with Multi-Parametric and Hierarchical Models
Juggling with Multi-Parametric and Hierarchical Models
Nuisance parameters and marginalized distributions
Gaussians, Gaussians, Gaussians everywhere
Comparing groups
Hierarchical models
Summary
Keep reading
Exercises
4
Understanding and Predicting Data with Linear Regression Models
Understanding and Predicting Data with Linear Regression Models
Simple linear regression
Robust linear regression
Hierarchical linear regression
Polynomial regression
Multiple linear regression
The GLM module
Summary
Keep reading
Exercises
5
Classifying Outcomes with Logistic Regression
Classifying Outcomes with Logistic Regression
Logistic regression
Multiple logistic regression
Discriminative and generative models
Summary
Keep reading
Exercises
6
Model Comparison
Model Comparison
Occam's razor – simplicity and accuracy
Regularizing priors
Predictive accuracy measures
Bayes factors
Bayes factors and information criteria
Summary
Keep reading
Exercises
7
Mixture Models
Mixture Models
Mixture models
Model-based clustering
Continuous mixtures
Summary
Keep reading
Exercises
8
Gaussian Processes
Gaussian Processes
Non-parametric statistics
Kernel-based models
Gaussian processes
Summary
Keep reading
Exercises
Index
Index

Predictive accuracy measures

In the previous example, it is more or less easy to see that the order 0 model is very simple and the order 5 model is too complex, but what about the other two? How we can distinguish between those options? We need a more principled way of taking into account the accuracy on one side and the simplicity on the other. Two methods to estimate the out-of-sample predictive accuracy using only the within-sample data are:

  • Cross-validation: This is an empirical strategy based on dividing the available data into subsets that are used for fitting and evaluation in an alternated way

  • Information criteria: This is an umbrella term for several relatively simple expressions that can be considered as ways to approximate the results that we could have obtained by performing cross-validation

Cross-validation

On average, the accuracy of a model will be higher for the within-sample than for the out-of-sample accuracy. As we need data to fit the model and data to test it, one simple...

Previous Section
End of Section 4
Next Section