Chapter 3: Model Evaluation in Comet
Before accepting a data science model, we need to evaluate it, to establish whether it is ready for production or not. Model evaluation is the process of assessing whether a trained model performs as expected. Usually, we perform model evaluation on a different dataset from the one on which the model was trained.
In this chapter, you will review the basic concepts behind model evaluation, such as data splitting, how to choose metrics for evaluation, and basic concepts behind error analysis. In addition, you will see the main model evaluation techniques for the different data science tasks (classification, regression, and clustering).
Finally, you will learn how to perform model evaluation in Comet by deepening some concepts that you already know, such as experiments, panels, and reports, as well as introducing new concepts, including hyperparameter tuning, model registry, and queries.
Throughout the chapter, you will also implement a practical...