Implementing Population-Based Training
Population-Based Training (PBT) is one of the variants of the Heuristic Search hyperparameter tuning group (see Chapter 5) that the NNI package can implement. To show you how to implement PBT with NNI using pure Python code, let’s use the same example provided by the NNI package. Here, the MNIST dataset and a convolutional neural network model are utilized. We’ll use PyTorch to implement the neural network model. For details of the code example provided by NNI, please refer to the NNI GitHub repository (https://github.com/microsoft/nni/tree/1546962f83397710fe095538d052dc74bd981707/examples/trials/mnist-pbt-tuner-pytorch).
MNIST Dataset
MNIST is a dataset of handwritten digits that have been size-normalized and centered in a fixed-size image. Here, we’ll use the MNIST dataset provided directly by the PyTorch package (https://pytorch.org/vision/stable/generated/torchvision.datasets.MNIST.html#torchvision.datasets.MNIST...