Using incremental learning to train a classifier
One of the problems of traditional machine learning libraries, such as scikit-learn, is that they seldom offer the possibility to train models on high volumes of data, which, coincidentally, is the best type of data for deep neural networks. What good is having large amounts of data if we can't use it?
Fortunately, there is a way to circumvent this limitation, and it's called incremental learning. In this recipe, we'll use a powerful library, creme, to train a classifier on a dataset too big to fit in memory.
Getting ready
In this recipe, we'll leverage creme, an experimental library specifically designed to train machine learning models on huge datasets that are too big to fit in memory. To install creme, execute the following command:
$> pip install creme==0.5.1
We'll use the features.hdf5 dataset we generated in the Implementing a feature extractor using a pre-trained network recipe in this...