Learning Probabilistic Graphical Models in R

Learning Probabilistic Graphical Models in R

Overview of this book

Probabilistic graphical models (PGM, also known as graphical models) are a marriage between probability theory and graph theory. Generally, PGMs use a graph-based representation. Two branches of graphical representations of distributions are commonly used, namely Bayesian networks and Markov networks. R has many packages to implement graphical models. We’ll start by showing you how to transform a classical statistical model into a modern PGM and then look at how to do exact inference in graphical models. Proceeding, we’ll introduce you to many modern R packages that will help you to perform inference on the models. We will then run a Bayesian linear regression and you’ll see the advantage of going probabilistic when you want to do prediction. Next, you’ll master using R packages and implementing its techniques. Finally, you’ll be presented with machine learning applications that have a direct impact in many fields. Here, we’ll cover clustering and the discovery of hidden information in big data, as well as two important methods, PCA and ICA, to reduce the size of big problems.

Learning Probabilistic Graphical Models in R

Credits

About the Author

About the Reviewers

www.PacktPub.com

Preface

Free Chapter

Probabilistic Reasoning

Machine learning

Representing uncertainty with probabilities

Probabilistic graphical models

Summary

Exact Inference

Building graphical models

Variable elimination

Sum-product and belief updates

The junction tree algorithm

Examples of probabilistic graphical models

Summary

Learning Parameters

Introduction

Learning by inference

Maximum likelihood

Learning with hidden variables – the EM algorithm

Principles of the EM algorithm

Summary

Bayesian Modeling – Basic Models

The Naive Bayes model

Beta-Binomial

The Gaussian mixture model

Summary

Approximate Inference

Sampling from a distribution

Basic sampling algorithms

Rejection sampling

Importance sampling

Markov Chain Monte-Carlo

MCMC for probabilistic graphical models in R

Summary

Bayesian Modeling – Linear Models

Linear regression

Bayesian linear models

Summary

Probabilistic Mixture Models

Mixture models

EM for mixture models

Mixture of Bernoulli

Mixture of experts

Latent Dirichlet Allocation

Summary

Appendix

References

Index

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Mixture of experts

The idea behind mixture of experts is to use a set of linear regressions for each sub space of the original data space and combine them with weighting functions that will successively give weight to each linear regression.

Consider the following example dataset, which we generate with the following toy code:

x1=runif(40,0,10)
x2=runif(40,10,20)

e1 = rnorm(20,0,2)
e2 = rnorm(20,0,3)
 
y1 = 1+2.5*x1 + e1
y2 = 35+-1.5*x2 + e2
 
xx=c(x1,x2)
yy=c(y1,y2)

Plotting the result, and doing a simple linear regression on it, gives the following:

Obviously, the linear regression does not capture the behavior of the data at all. It barely captures a general trend in the data that more or less averages the data set.

The idea of mixture of experts is to have several sub models within a bigger model—for example, having several regression lines, as the following graph:

In this graph, the red and green lines seems to better represent the data set. However, the model needs to choose when to choose...

Learning Probabilistic Graphical Models in R

Learning Probabilistic Graphical Models in R

Overview of this book

Related Content you might be interested in

Current Title:

Learning Probabilistic Graphical Models in R

Mixture of experts