Introducing Microsoft NNI
Neural Network Intelligence (NNI) is a package that is developed by Microsoft and can be utilized not only for hyperparameter tuning tasks but also for neural architecture search, model compression, and feature engineering. In this section, we will discuss how to utilize NNI specifically for the hyperparameter tuning task. To install NNI, you can simply call the pip install nni
command.
Although NNI refers to Neural Network Intelligence, it actually supports numerous ML frameworks including (but not limited to) scikit-learn, XGBoost, LightGBM, PyTorch, TensorFlow, Caffe2, and MXNet.
There are numerous hyperparameter tuning methods implemented in NNI; some of them are built-in and others are wrapped from other packages such as Hyperopt
(see Chapter 8) and SMAC3
. Here, in NNI, the hyperparameter tuning methods are referred to as tuners. We will not discuss all of the tuners implemented in NNI since there are too many of them. We will only discuss the...