Book Image

Python for Data Science For Dummies - Second Edition

By : John Paul Mueller, Luca Massaron
Book Image

Python for Data Science For Dummies - Second Edition

By: John Paul Mueller, Luca Massaron

Overview of this book

Python is a general-purpose programming language created in the late 1980s — and named after Monty Python — that's used by thousands of people to do things from testing microchips at Intel to powering Instagram to building video games with the PyGame library. The book begins by discussing how Python can make data science easy. You’ll learn how to work with the Anaconda tool suite that makes coding in Python easy. You’ll also learn to write code using Google Colab. As you progress, you'll discover how to perform interesting calculations and data manipulations using various Python libraries, such as pandas and NumPy. You’ll learn how to create data visualizations with MatPlotLib. While learning the advanced concepts, you’ll learn how to wrangle data by using techniques, such as hierarchical clustering. Finally, you’ll learn how to work with decision trees and use machine learning to make predictions. By the end of the book, you’ll have the skills and the knowledge that’s needed to write code in Python and extract information from data.
Table of Contents (13 chapters)
Free Chapter
About the Authors
Advertisement Page
Connect with Dummies
End User License Agreement

Chapter 18

Performing Cross-Validation, Selection, and Optimization


Bullet Learning about overfitting and underfitting

Bullet Choosing the right metric to monitor

Bullet Cross-validating the results

Bullet Selecting the best features for machine learning

Bullet Optimizing hyperparameters

Machine learning algorithms can indeed learn from data. For instance, the four algorithms presented in the previous chapter, although quite simple, can effectively estimate a class or a value after being presented with examples associated with outcomes. It is all a matter of learning by induction, which is the process of extracting general rules from specific examples. From childhood, humans commonly learn by seeing examples, deriving some general rules or ideas from them, and then successfully applying the derived rule to new situations as we grow up. For example, if we see someone being burned after touching fire, we understand that fire is dangerous, and we don’t need to touch it ourselves to know that...