Book Image

Learning Bayesian Models with R

By : Hari Manassery Koduvely
Book Image

Learning Bayesian Models with R

By: Hari Manassery Koduvely

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
Learning Bayesian Models with R
Hari Manassery Koduvely
Oct, 2015 | 168 pages
No titles found
Table of Contents (16 chapters)
Learning Bayesian Models with R
Learning Bayesian Models with R
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Introducing the Probability Theory
Introducing the Probability Theory
Probability distributions
Conditional probability
Bayesian theorem
Marginal distribution
Expectations and covariance
Exercises
References
Summary
2
The R Environment
The R Environment
Setting up the R environment and packages
Managing data in R
Writing R programs
Data visualization
Sampling
Exercises
References
Summary
3
Introducing Bayesian Inference
Introducing Bayesian Inference
Bayesian view of uncertainty
Exercises
References
Summary
4
Machine Learning Using Bayesian Inference
Machine Learning Using Bayesian Inference
Why Bayesian inference for machine learning?
Model overfitting and bias-variance tradeoff
Selecting models of optimum complexity
Bayesian averaging
An overview of common machine learning tasks
References
Summary
5
Bayesian Regression Models
Bayesian Regression Models
Generalized linear regression
The arm package
The Energy efficiency dataset
Regression of energy efficiency with building parameters
Simulation of the posterior distribution
Exercises
References
Summary
6
Bayesian Classification Models
Bayesian Classification Models
Performance metrics for classification
The Naïve Bayes classifier
The Bayesian logistic regression model
Exercises
References
Summary
7
Bayesian Models for Unsupervised Learning
Bayesian Models for Unsupervised Learning
Bayesian mixture models
Topic modeling using Bayesian inference
R packages for LDA
Exercises
References
Summary
8
Bayesian Neural Networks
Bayesian Neural Networks
Two-layer neural networks
Bayesian treatment of neural networks
The brnn R package
Deep belief networks and deep learning
Exercises
References
Summary
9
Bayesian Modeling at Big Data Scale
Bayesian Modeling at Big Data Scale
Distributed computing using Hadoop
RHadoop for using Hadoop from R
Spark – in-memory distributed computing
SparkR
Linear regression using SparkR
Computing clusters on the cloud
Other R packages for large scale machine learning
Exercises
References
Summary
Index
Index

Spark – in-memory distributed computing

One of the issues with Hadoop is that after a MapReduce operation, the resulting files are written to the hard disk. Therefore, when there is a large data processing operation, there would be many read and write operations on the hard disk, which makes processing in Hadoop very slow. Moreover, the network latency, which is the time required to shuffle data between different nodes, also contributes to this problem. Another disadvantage is that one cannot make real-time queries from the files stored in HDFS. For machine learning problems, during training phase, the MapReduce will not persist over iterations. All this makes Hadoop not an ideal platform for machine learning.

A solution to this problem was invented at Berkeley University's AMP Lab in 2009. This came out of the PhD work of Matei Zaharia, a Romanian born computer scientist. His paper Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing (reference 4 in...

Previous Section
End of Section 4
Next Section