The weight values present in a DFN are responsible for making predictions. Any deep network has so many weights that finding perfect values for weights becomes impossible. Hence, we try to search for a set of weight values that will give us sufficiently good prediction results. Thus, training a network implies learning the optimal weight values starting from an initialized set of weights. Suppose we have a DFN and, initially, we don't know what set of weights will perform well. Hence, we initialize the weight values say with random real numbers. Now, we have to go from initialized weight values to optimal weight values. We can break this task into the following three parts:
- First, we need to know whether the initialized weights are a good fit or not. If not, how much does the predicted output differ from the expected output? This...