When the dataset is fully available at the beginning of a supervised task, and doesn't exceed the quantity of RAM on your machine, you can train the classifier or the regression using batch learning. As seen in previous chapters, during training the learner scans the full dataset. This also happens when stochastic gradient descent (SGD)-based methods are used (see Chapter 2, Approaching Simple Linear Regression and Chapter 3, Multiple Regression in Action). Let's now compare how much time is needed to train a linear regressor and relate its performance with the number of observations in the dataset (that is, the number of rows of the feature matrix X) and the number of features (that is, the number of columns of X). In this first experiment, we will use the plain vanilla LinearRegression() and SGDRegressor() classes provided by Scikit-learn, and we will store the actual time taken to fit a classifier, without any parallelization.
Let's first create a function to create fake...