Unlike DBNs, Deep Restricted Boltzmann Machines (DRBM) are undirected networks of interconnected hidden layers with the capability to learn joint probabilities over these connections. In the current setup, centering is performed where visible and hidden variables are subtracted from offset bias vectors after every iteration. Research has shown that centering optimizes the performance of DRBMs and can reach higher log-likelihood values in comparison with traditional RBMs.
This section provides the requirements for setting up a DRBM:
- The
MNIST
dataset is loaded and set up - The
tensorflow
package is set up and loaded
This section covers detailed the steps for setting up the DRBM model using TensorFlow in R:
- Define the parameters for the DRBM:
learning_rate = 0.005 momentum = 0.005 minbatch_size = 25 hidden_layers = c(400,100) biases = list(-1,-1)
- Define a sigmoid function using a hyperbolic arc tangent [(log(1+x) -log(1-x))...