Book Image

Learning Probabilistic Graphical Models in R

By : David Bellot, Dan Toomey
Book Image

Learning Probabilistic Graphical Models in R

By: David Bellot, Dan Toomey

Overview of this book

Probabilistic graphical models (PGM, also known as graphical models) are a marriage between probability theory and graph theory. Generally, PGMs use a graph-based representation. Two branches of graphical representations of distributions are commonly used, namely Bayesian networks and Markov networks. R has many packages to implement graphical models. We’ll start by showing you how to transform a classical statistical model into a modern PGM and then look at how to do exact inference in graphical models. Proceeding, we’ll introduce you to many modern R packages that will help you to perform inference on the models. We will then run a Bayesian linear regression and you’ll see the advantage of going probabilistic when you want to do prediction. Next, you’ll master using R packages and implementing its techniques. Finally, you’ll be presented with machine learning applications that have a direct impact in many fields. Here, we’ll cover clustering and the discovery of hidden information in big data, as well as two important methods, PCA and ICA, to reduce the size of big problems.
Table of Contents (15 chapters)


The following references were used while writing this book. We encourage those of you who want to go further into the field of probabilistic graphical models and Bayesian modeling to read at least some of them.

Many of our examples and presentations of algorithms took inspiration from these books and papers.

Books on the Bayesian theory

  • Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B, Vehtari, A., and Rubin, D.B.. Bayesian Data Analysis, 3rd Edition. CRC Press. 2013. This is a reference book on Bayesian modeling covering topics from the most fundamental aspects to the most advanced, with the focus on modeling and also on computations.

  • Robert, C.P.. The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation. Springer. 2007. This is a beautiful presentation of the Bayesian paradigm with many examples. The book is more theoretical but has a rigorous presentation of many aspects of the Bayesian paradigm.

  • McGrayne, Sharon Bertsch. The Theory That Would Not Die. Yale University Press. 2011. This talks about how Bayes' rule cracked the Enigma code, hunted down Russian submarines, and emerged triumphant from two centuries of controversy. It is a brilliantly written history of Bayes' rule going from the seminal paper of Thomas Bayes to the latest advances in the 21st century.

Books on machine learning

  • Murphy, K.P.. Machine Learning: A Probabilistic Perspective. The MIT Press. 2012. This is a book on machine learning in general with a lot of algorithms. It covers more than just graphical models and Bayesian models. It is one of the best references.

  • Bishop, C.M. Pattern Recognition and Machine Learning. Springer. 2007. This is one of the best books on machine learning, covering many aspects and going through many details of the implementation of each algorithm.

  • Barber, D.. Bayesian Reasoning and Machine Learning. Cambridge University Press. 2012. This is another excellent reference book covering many aspects of machine learning with a specific focus on Bayesian models.

  • Robert, C.P.. Monte Carlo Methods in Statistics. 2009. ( This is an excellent paper on the Monte Carlo methods, and it is very pedagogical.

  • Koller, D. and Friedman, N.. Probabilistic Graphical Models: Principles and Techniques. The MIT Press. 2009. This is the most complete and advanced book on probabilistic graphical models. It covers all aspects of the domain. This book is very dense, with thorough details on many algorithms related to PGM and useful demonstrations. Probably the best book on PGM.

  • Casella, G. and Berger, R.L.. Statistical Inference, 2nd Edition. Duxbury. 2002: This is a reference book on standard statistics with many detailed demonstrations. It's a book that anyone doing statistics should read.

  • Hastie, T., Tibshirani, R., and Friedman, J.. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer. 2013: This is a book best-seller and covers the most important aspects of machine learning from a statistical point of view.


  • Jacobs, R.A., Jordan, M.I, Nowlan, S.J., and Hinton, G.E. Adaptive mixtures of local experts. 1991 in Neural Computation, 3, 79-87: This is the reference paper on mixture of experts as seen in Chapter 7, Probabilistic Mixture Models.

  • Blei, David M., Ng, Andrew, Y, Jordan, Michael, I. Latent Dirichlet Allocation. January 2003, Journal of Machine Learning Research 3 (4–5), p993–1022: This is a reference paper on the LDA model as seen in Chapter 7, Probabilistic Mixture Models.