Book Image

Data Science with Python

By : Rohan Chopra, Aaron England, Mohamed Noordeen Alaudeen
Book Image

Data Science with Python

By: Rohan Chopra, Aaron England, Mohamed Noordeen Alaudeen

Overview of this book

Data Science with Python begins by introducing you to data science and teaches you to install the packages you need to create a data science coding environment. You will learn three major techniques in machine learning: unsupervised learning, supervised learning, and reinforcement learning. You will also explore basic classification and regression techniques, such as support vector machines, decision trees, and logistic regression. As you make your way through the book, you will understand the basic functions, data structures, and syntax of the Python language that are used to handle large datasets with ease. You will learn about NumPy and pandas libraries for matrix calculations and data manipulation, discover how to use Matplotlib to create highly customizable visualizations, and apply the boosting algorithm XGBoost to make predictions. In the concluding chapters, you will explore convolutional neural networks (CNNs), deep learning algorithms used to predict what is in an image. You will also understand how to feed human sentences to a neural network, make the model process contextual information, and create human language processing systems to predict the outcome. By the end of this book, you will be able to understand and implement any new data science algorithm and have the confidence to experiment with tools or libraries other than those covered in the book.
Table of Contents (10 chapters)

Chapter 3: Introduction to Machine Learning via Scikit-Learn

Activity 5: Generating Predictions and Evaluating the Performance of a Multiple Linear Regression Model

Solution:

  1. Generate predictions on the test data using the following:

    predictions = model.predict(X_test)

    2.    Plot the predicted versus actual values on a scatterplot using the following code:

    import matplotlib.pyplot as plt

    from scipy.stats import pearsonr

     

    plt.scatter(y_test, predictions)

    plt.xlabel('Y Test (True Values)')

    plt.ylabel('Predicted Values')

    plt.title('Predicted vs. Actual Values (r = {0:0.2f})'.format(pearsonr(y_test, predictions)[0], 2))

    plt.show()

    Refer to the resultant output here:

    Figure 3.33: A scatterplot of predicted versus actual values from a multiple linear regression model
    Figure 3.33: A scatterplot of predicted versus actual values from a multiple linear regression model

    Note

    There is a much stronger linear correlation between the predicted and actual values in the multiple linear regression model (r = 0.93) relative to the simple linear regression model (r = 0.62).

  2. To plot the distribution of the residuals, refer to the code here:

    import seaborn as sns

    from scipy.stats import shapiro

     

    sns.distplot((y_test - predictions), bins = 50)

    plt.xlabel('Residuals')

    plt.ylabel('Density')

    plt.title('Histogram of Residuals (Shapiro W p-value = {0:0.3f})'.format(shapiro(y_test - predictions)[1]))

    plt.show()

    Refer to the resultant output here:

    Figure 3.34: The distribution of the residuals from a multiple linear regression model
    Figure 3.34: The distribution of the residuals from a multiple linear regression model

    Note

    Our residuals are negatively skewed and non-normal, but this is less skewed than in the simple linear model.

  3. Calculate the metrics for mean absolute error, mean squared error, root mean squared error, and R-squared, and put them into a DataFrame as follows:

    from sklearn import metrics

    import numpy as np

    metrics_df = pd.DataFrame({'Metric': ['MAE',

                                          'MSE',

                                          'RMSE',

                                          'R-Squared'],

                              'Value': [metrics.mean_absolute_error(y_test, predictions),

                                          metrics.mean_squared_error(y_test, predictions),

                                        np.sqrt(metrics.mean_squared_error(y_test, predictions)),

                                        metrics.explained_variance_score(y_test, predictions)]}).round(3)

    print(metrics_df)

    Please refer to the resultant output:

    Figure 3.35: Model evaluation metrics from a multiple linear regression model
Figure 3.35: Model evaluation metrics from a multiple linear regression model

The multiple linear regression model performed better on every metric relative to the simple linear regression model.

Activity 6: Generating Predictions and Evaluating Performance of a Tuned Logistic Regression Model

Solution:

  1. Generate the predicted probabilities of rain using the following code:

    predicted_prob = model.predict_proba(X_test)[:,1]

  2. Generate the predicted class of rain using predicted_class = model.predict(X_test).
  3. Evaluate performance using a confusion matrix and save it as a DataFrame using the following code:

    from sklearn.metrics import confusion_matrix

    import numpy as np

     

    cm = pd.DataFrame(confusion_matrix(y_test, predicted_class))

    cm['Total'] = np.sum(cm, axis=1)

    cm = cm.append(np.sum(cm, axis=0), ignore_index=True)

    cm.columns = ['Predicted No', 'Predicted Yes', 'Total']

    cm = cm.set_index([['Actual No', 'Actual Yes', 'Total']])

    print(cm)

    Figure 3.36: The confusion matrix from our logistic regression grid search model
    Figure 3.36: The confusion matrix from our logistic regression grid search model

    Note

    Nice! We have decreased our number of false positives from 6 to 2. Additionally, our false negatives were lowered from 10 to 4 (see in Exercise 26). Be aware that results may vary slightly.

  4. For further evaluation, print a classification report as follows:

    from sklearn.metrics import classification_report

     

    print(classification_report(y_test, predicted_class))

Figure 3.37: The classification report from our logistic regression grid search model
Figure 3.37: The classification report from our logistic regression grid search model

By tuning the hyperparameters of the logistic regression model, we were able to improve upon a logistic regression model that was already performing very well.

Activity 7: Generating Predictions and Evaluating the Performance of the SVC Grid Search Model

Solution:

  1. Extract predicted classes of rain using the following code:

    predicted_class = model.predict(X_test)

  2. Create and print a confusion matrix using the code here:

    from sklearn.metrics import confusion_matrix

    import numpy as np

     

    cm = pd.DataFrame(confusion_matrix(y_test, predicted_class))

    cm['Total'] = np.sum(cm, axis=1)

    cm = cm.append(np.sum(cm, axis=0), ignore_index=True)

    cm.columns = ['Predicted No', 'Predicted Yes', 'Total']

    cm = cm.set_index([['Actual No', 'Actual Yes', 'Total']])

    print(cm)

    See the resultant output here:

    Figure 3.38: The confusion matrix from our SVC grid search model
    Figure 3.38: The confusion matrix from our SVC grid search model
  3. Generate and print a classification report as follows:

    from sklearn.metrics import classification_report

     

    print(classification_report(y_test, predicted_class))

    See the resultant output here:

Figure 3.39: The classification report from our SVC grid search model
Figure 3.39: The classification report from our SVC grid search model

Here, we demonstrated how to tune the hyperparameters of an SVC model using grid search.

Activity 8: Preparing Data for a Decision Tree Classifier

Solution:

  1. Import weather.csv and store it as a DataFrame using the following:

    import pandas as pd

     

    df = pd.read_csv('weather.csv')

  2. Dummy code the Description column as follows:

    import pandas as pd

     

    df_dummies = pd.get_dummies(df, drop_first=True)

  3. Shuffle df_dummies using the following code:

    from sklearn.utils import shuffle

     

    df_shuffled = shuffle(df_dummies, random_state=42)

  4. Split df_shuffled into X and y as follows:

    DV = 'Rain'

    X = df_shuffled.drop(DV, axis=1)

    y = df_shuffled[DV]

  5. Split X and y into testing and training data:

    from sklearn.model_selection import train_test_split

     

    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

  6. Scale X_train and X_test using the following code:

    from sklearn.preprocessing import StandardScaler

     

    model = StandardScaler()

    X_train_scaled = model.fit_transform(X_train)

    X_test_scaled = model.transform(X_test)

Activity 9: Generating Predictions and Evaluating the Performance of a Decision Tree Classifier Model

Solution:

  1. Generate the predicted probabilities of rain using the following:

    predicted_prob = model.predict_proba(X_test_scaled)[:,1]

  2. Generate the predicted classes of rain using the following:

    predicted_class = model.predict(X_test)

  3. Generate and print a confusion matrix with the code here:

    from sklearn.metrics import confusion_matrix

    import numpy as np

     

    cm = pd.DataFrame(confusion_matrix(y_test, predicted_class))

    cm['Total'] = np.sum(cm, axis=1)

    cm = cm.append(np.sum(cm, axis=0), ignore_index=True)

    cm.columns = ['Predicted No', 'Predicted Yes', 'Total']

    cm = cm.set_index([['Actual No', 'Actual Yes', 'Total']])

    print(cm)

    Refer to the resultant output here:

    Figure 3.40: The confusion matrix from our tuned decision tree classifier model
    Figure 3.40: The confusion matrix from our tuned decision tree classifier model
  4. Print a classification report as follows:

    from sklearn.metrics import classification_report

    print(classification_report(y_test, predicted_class))

    Refer to the resultant output here:

Figure 3.41: The classification report from our tuned decision tree classifier model
Figure 3.41: The classification report from our tuned decision tree classifier model

There was only one misclassified observation. Thus, by tuning a decision tree classifier model on our weather.csv dataset, we were able to predict rain (or snow) with great accuracy. We can see that the sole driving feature was temperature in Celsius. This makes sense due to the way in which decision trees use recursive partitioning to make predictions.

Activity 10: Tuning a Random Forest Regressor

Solution:

  1. Specify the hyperparameter space as follows:

    import numpy as np

     

    grid = {'criterion': ['mse','mae'],

            'max_features': ['auto', 'sqrt', 'log2', None],

            'min_impurity_decrease': np.linspace(0.0, 1.0, 10),

            'bootstrap': [True, False],

            'warm_start': [True, False]}

  2. Instantiate the GridSearchCV model, optimizing the explained variance using the following code:

    from sklearn.model_selection import GridSearchCV

    from sklearn.ensemble import RandomForestRegressor

     

    model = GridSearchCV(RandomForestRegressor(), grid, scoring='explained_variance', cv=5)

  3. Fit the grid search model to the training set using the following (note that this may take a while):

    model.fit(X_train_scaled, y_train)

    See the output here:

    Figure 3.42: The output from our tuned random forest regressor grid search model
    Figure 3.42: The output from our tuned random forest regressor grid search model
  4. Print the tuned parameters as follows:

    best_parameters = model.best_params_

    print(best_parameters)

    See the resultant output below:

Figure 3.43: The tuned hyperparameters from our random forest regressor grid search model
Figure 3.43: The tuned hyperparameters from our random forest regressor grid search model

Activity 11: Generating Predictions and Evaluating the Performance of a Tuned Random Forest Regressor Model

Solution:

  1. Generate predictions on the test data using the following:

    predictions = model.predict(X_test_scaled)

  2. Plot the correlation of predicted and actual values using the following code:

    import matplotlib.pyplot as plt

    from scipy.stats import pearsonr

     

    plt.scatter(y_test, predictions)

    plt.xlabel('Y Test (True Values)')

    plt.ylabel('Predicted Values')

    plt.title('Predicted vs. Actual Values (r = {0:0.2f})'.format(pearsonr(y_test, predictions)[0], 2))

    plt.show()

    Refer to the resultant output here:

    Figure 3.44: A scatterplot of predicted and actual values from our random forest regression model with tuned hyperparameters

    Figure 3.44: A scatterplot of predicted and actual values from our random forest regression model with tuned hyperparameters
  3. Plot the distribution of residuals as follows:

    import seaborn as sns

    from scipy.stats import shapiro

     

    sns.distplot((y_test - predictions), bins = 50)

    plt.xlabel('Residuals')

    plt.ylabel('Density')

    plt.title('Histogram of Residuals (Shapiro W p-value = {0:0.3f})'.format(shapiro(y_test - predictions)[1]))

    plt.show()

    Refer to the resultant output here:

    Figure 3.45: A histogram of residuals from a random forest regression model with 
tuned hyperparameters
    Figure 3.45: A histogram of residuals from a random forest regression model with tuned hyperparameters
  4. Compute metrics, place them in a DataFrame, and print it using the code here:

    from sklearn import metrics

    import numpy as np

    metrics_df = pd.DataFrame({'Metric': ['MAE',

                                          'MSE',

                                          'RMSE',

                                          'R-Squared'],

                              'Value': [metrics.mean_absolute_error(y_test, predictions),

                                        metrics.mean_squared_error(y_test, predictions),

                                        np.sqrt(metrics.mean_squared_error(y_test, predictions)),

                                        metrics.explained_variance_score(y_test, predictions)]}).round(3)

    print(metrics_df)

    Find the resultant output here:

Figure 3.46: Model evaluation metrics from our random forest regression model with tuned hyperparameters
Figure 3.46: Model evaluation metrics from our random forest regression model with tuned hyperparameters

The random forest regressor model seems to underperform compared to the multiple linear regression, as evidenced by greater MAE, MSE, and RMSE values, as well as less explained variance. Additionally, there was a weaker correlation between the predicted and actual values, and the residuals were further from being normally distributed. Nevertheless, by leveraging ensemble methods using a random forest regressor, we constructed a model that explains 75.8% of the variance in temperature and predicts temperature in Celsius + 3.781 degrees.