Book Image

Learning Bayesian Models with R

By : Hari Manassery Koduvely
Book Image

Learning Bayesian Models with R

By: Hari Manassery Koduvely

Overview of this book

Table of Contents (16 chapters)
Learning Bayesian Models with R
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Bayesian treatment of neural networks


To set the neural network learning in a Bayesian context, consider the error function for the regression case. It can be treated as a Gaussian noise term for observing the given dataset conditioned on the weights w. This is precisely the likelihood function that can be written as follows:

Here, is the variance of the noise term given by and represents a probabilistic model. The regularization term can be considered as the log of the prior probability distribution over the parameters:

Here, is the variance of the prior distribution of weights. It can be easily shown using Bayes' theorem that the objective function M(w) then corresponds to the posterior distribution of parameters w:

In the neural network case, we are interested in the local maxima of . The posterior is then approximated as a Gaussian around each maxima , as follows:

Here, A is a matrix of the second derivative of M(w) with respect to w and represents an inverse of the covariance matrix...