Learning from data using an MLP was one of the major problems since its conception. As we pointed out before, one of the major problems with neural networks was the computational tractability of deeper models, and the other was stable learning algorithms that would converge to a reasonable minimum. One of the major breakthroughs in machine learning, and what paved the way for deep learning, was the development of the learning algorithm based on backpropagation. Many scientists independently derived and applied forms of backpropagation in the 1960s; however, most of the credit has been given to Prof. G. E. Hinton and his group (Rumelhart, D. E., et.al. 1986). In the next few paragraphs, we will go over this algorithm, whose sole purpose is to minimize the error caused by incorrect predictions made during training.
To begin, we will describe the dataset, which is called spirals. This is a widely known benchmark dataset that has two classes that are separable, yet highly...