In Chapter 7, Computer Vision, we showed how to and segment objects in single images. The objects in these images were fixed. However, if we add a temporal dimension to our input, objects can move within a certain scene. Understanding what is happening throughout multiple frames (a video) is a much harder task. In this recipe, we want to demonstrate how to get started when tackling videos. We will focus on combining a CNN and an RNN. The CNN is used to extract features for single frames; these features are combined and used as input for an RNN. This is also known as stacking, where we build (stack) a model on top of another model.
For this recipe, we will be using a dataset that contains 13,321 short videos. These videos are distributed over a total of 101 different classes. Because of the complexity of this task, we don't want to train our models from scratch. Therefore, we will be using the pretrained weights of the InceptionV3 model provided within...