Create a train/test split (80/20) with PAY_1 and LIMIT_BAL as features:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
df[['PAY_1', 'LIMIT_BAL']].values, df['default payment next month'].values,
test_size=0.2, random_state=24)

Import LogisticRegression, with the default options, but set the solver to 'liblinear'.

from sklearn.linear_model import LogisticRegression
lr_model = LogisticRegression(solver='liblinear')

Train on the training data and obtain predicted classes, as well as class probabilities, using the testing data:

lr_model.fit(X_train, y_train)
y_pred = lr_model.predict(X_test)
y_pred_proba = lr_model.predict_proba(X_test)

Pull out the coefficients and intercept from the trained model and manually calculate predicted probabilities. You'll need to add a column of 1s to your features, to multiply by the intercept.

First, let's create the array of features, with a column of 1s added, using horizontal stacking:

ones_and_features = np.hstack([np.ones((X_test.shape[0],1)), X_test])

Now we need the intercept and coefficients, which we reshape and concatenate from scikit-learn output:

intercept_and_coefs = np.concatenate([lr_model.intercept_.reshape(1,1), lr_model.coef_], axis=1)

To repeatedly multiply the intercept and coefficients by the all the rows of ones_and_features, and take the sum of each row (that is, find the linear combination), you could write this all out using multiplication and addition. However, it's much faster to use the dot product:

X_lin_comb = np.dot(intercept_and_coefs, np.transpose(ones_and_features))

Now X_lin_comb has the argument we need to pass to the sigmoid function we defined, in order to calculate predicted probabilities:

y_pred_proba_manual = sigmoid(X_lin_comb)

Using a threshold of 0.5, manually calculate predicted classes. Compare this to the class predictions output by scikit-learn.

The manually predicted probabilities, y_pred_proba_manual, should be the same as y_pred_proba; we'll check that momentarily. First, manually predict the classes with the threshold:

y_pred_manual = y_pred_proba_manual >= 0.5

This array will have a different shape than y_pred, but it should contain the same values. We can check whether all the elements of two arrays are equal like this:

Figure 6.52: Equality of NumPy arrays

Calculate ROC AUC using both scikit-learn's predicted probabilities, and your manually predicted probabilities, and compare.

First, import the following:

from sklearn.metrics import roc_auc_score

Then, calculate this metric on both versions, taking care to access the correct column, or reshape as necessary:

Figure 6.53: Calculating the ROC AUC's from predicted probabilities