Book Image

Designing Machine Learning Systems with Python

By : David Julian
Book Image

Designing Machine Learning Systems with Python

By: David Julian

Overview of this book

Machine learning is one of the fastest growing trends in modern computing. It has applications in a wide range of fields, including economics, the natural sciences, web development, and business modeling. In order to harness the power of these systems, it is essential that the practitioner develops a solid understanding of the underlying design principles. There are many reasons why machine learning models may not give accurate results. By looking at these systems from a design perspective, we gain a deeper understanding of the underlying algorithms and the optimisational methods that are available. This book will give you a solid foundation in the machine learning design process, and enable you to build customised machine learning models to solve unique problems. You may already know about, or have worked with, some of the off-the-shelf machine learning models for solving common problems such as spam detection or movie classification, but to begin solving more complex problems, it is important to adapt these models to your own specific needs. This book will give you this understanding and more.
Table of Contents (16 chapters)
Designing Machine Learning Systems with Python
Credits
About the Author
About the Reviewer
www.PacktPub.com
Preface
Free Chapter
1
Thinking in Machine Learning
Index

Evaluating model performance


Measuring a model's performance is an important machine learning task, and there are many varied parameters and heuristics for doing this. The importance of defining a scoring strategy should not be underestimated, and in Sklearn, there are basically three approaches:

  • Estimator score: This refers to using the estimator's inbuilt score() method, specific to each estimator

  • Scoring parameters: This refers to cross-validation tools relying on an internal scoring strategy

  • Metric functions: These are implemented in the metrics module

We have seen examples of the estimator score() method, for example, clf.score(). In the case of a linear classifier, the score() method returns the mean accuracy. It is a quick and easy way to gauge an individual estimator's performance. However, this method is usually insufficient in itself for a number of reasons.

If we remember, accuracy is the sum of the true positive and true negative cases divided by the number of samples. Using this...