Book Image

Supervised Machine Learning with Python

By : Taylor Smith
Book Image

Supervised Machine Learning with Python

By: Taylor Smith

Overview of this book

Supervised machine learning is used in a wide range of sectors, such as finance, online advertising, and analytics, to train systems to make pricing predictions, campaign adjustments, customer recommendations, and much more by learning from the data that is used to train it and making decisions on its own. This makes it crucial to know how a machine 'learns' under the hood. This book will guide you through the implementation and nuances of many popular supervised machine learning algorithms, and help you understand how they work. You’ll embark on this journey with a quick overview of supervised learning and see how it differs from unsupervised learning. You’ll then explore parametric models, such as linear and logistic regression, non-parametric methods, such as decision trees, and a variety of clustering techniques that facilitate decision-making and predictions. As you advance, you'll work hands-on with recommender systems, which are widely used by online companies to increase user interaction and enrich shopping potential. Finally, you’ll wrap up with a brief foray into neural networks and transfer learning. By the end of this book, you’ll be equipped with hands-on techniques and will have gained the practical know-how you need to quickly and effectively apply algorithms to solve new problems.
Table of Contents (11 chapters)
Title Page
Copyright and Credits
About Packt
Contributor
Preface
Index

Implementing a decision tree from scratch


We will start out by looking at the implementation of our splitting metrics. Then we'll cover some of our splitting logic, and finally, we'll see how we can wrap the tree so that we can generalize from classification and regression tasks.

Classification tree

Let's go ahead and walk through a classification tree example. We will be using the information gain criteria. In PyCharm there are three scripts open, two of which are metrics.py and cart.py, both of which are found inside of the packtml/decision_tree submodule. Then we have the example_classification_decision_tree.py file, which is in examples/decision_tree. Let's start with metrics.

 

 

If you open up the cart.py file, we have an order in which we should step through this so that you can understand how the decision tree class is going to work:

# 1. metrics.InformationGain & metrics.VarianceReduction
# 2. RandomSplitter
# 3. LeafNode
# 4. BaseCART

Starting with the metrics.py file from the top...