Transformers and the transform() method

Another common shortcut that we saw previously, fit_transform(), allows users to perform both steps in one command, making preprocessing workflows more efficient. Again, when to use fit_transform() and fit() with transform() separately depends on the task at hand. Typically, we should apply the fit_transform() method to our training data if we want to transform our data immediately based on the calculated transformation, something the fit() method can’t achieve by itself. However, when applying transformations to our test dataset, we wouldn’t want to reapply the fit() method; this would impose a potentially different data transformation, as our test data will be slightly different from our training data. Remember, our test dataset is meant to be treated exactly like our training data for model consistency purposes, so implementing a separate fit() method on it could potentially alter our test data and make our model predictions unreliable when applied in a real-world scenario:

from sklearn.preprocessing import StandardScaler
import numpy as np
# Example data
X = np.array([[1, 2], [3, 4], [5, 6]])
# Create a StandardScaler instance and
# fit_transform the data in one step
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(X_scaled)
# Output:
[[-1.22474487 -1.22474487]
[ 0.           0.        ]
[ 1.22474487   1.22474487]]

This consistency across all transformers allows them to be integrated seamlessly into ML pipelines, ensuring that the same transformation is applied to both the training and test data, something that becomes significantly important when implementing production-level models.

We will explore various transformers, including StandardScaler(), MinMaxScaler(), and OneHotEncoder(), in Chapter 2 to demonstrate how they can be used to prepare data for ML models using the fit(), transform(), and fit_transform() methods. Practical examples will be provided to illustrate how you can integrate transformers into workflows to ensure your data is preprocessed consistently.