Book Image

Data Science Projects with Python - Second Edition

By : Stephen Klosterman
Book Image

Data Science Projects with Python - Second Edition

By: Stephen Klosterman

Overview of this book

If data is the new oil, then machine learning is the drill. As companies gain access to ever-increasing quantities of raw data, the ability to deliver state-of-the-art predictive models that support business decision-making becomes more and more valuable. In this book, you’ll work on an end-to-end project based around a realistic data set and split up into bite-sized practical exercises. This creates a case-study approach that simulates the working conditions you’ll experience in real-world data science projects. You’ll learn how to use key Python packages, including pandas, Matplotlib, and scikit-learn, and master the process of data exploration and data processing, before moving on to fitting, evaluating, and tuning algorithms such as regularized logistic regression and random forest. Now in its second edition, this book will take you through the end-to-end process of exploring data and delivering machine learning models. Updated for 2021, this edition includes brand new content on XGBoost, SHAP values, algorithmic fairness, and the ethical concerns of deploying a model in the real world. By the end of this data science book, you’ll have the skills, understanding, and confidence to build your own machine learning models and gain insights from real data.
Table of Contents (9 chapters)
Preface

3. Details of Logistic Regression and Feature Exploration

Activity 3.01: Fitting a Logistic Regression Model and Directly Using the Coefficients

Solution:

The first few steps are similar to things we've done in previous activities:

  1. Create a train/test split (80/20) with PAY_1 and LIMIT_BAL as features:
    from sklearn.model_selection import train_test_split
    X_train, X_test, y_train, y_test = train_test_split(
        df[['PAY_1', 'LIMIT_BAL']].values,
        df['default payment next month'].values,
        test_size=0.2, random_state=24)
  2. Import LogisticRegression, with the default options, but set the solver to 'liblinear':
    from sklearn.linear_model import LogisticRegression
    lr_model = LogisticRegression(solver='liblinear')
  3. Train on the training data and obtain predicted classes, as well as class probabilities, using the test data:
    lr_model.fit(X_train, y_train...