The most common form of machine learning is learning; for example, if we are building a system to classify a specific set of images, we first collect a large Dataset of images from the same categories. During training, the machine is shown an image, and it produces an output in the form of a vector of scores, one for each category. As a result of the training, we expect the desired category to have the highest score out of all the categories.
A particular type of deep network--the convolutional neural network (ConvNet/CNN)--is much easier to train and generalizes much better fully-connected networks. In supervised learning scenarios, deep convolutional networks have significantly improved the results of processing images, video, speech, and audio data. Similarly, recurrent nets have shone the light on sequential data, such as text and speech. We will explore these types of neural networks in the subsequent sections.