Chapter 1, Regression – The Workhorse of Data Science, introduces why regression is indeed useful for data science, how to quickly set up Python for data science and provides an overview of the packages used throughout the book with the help of examples. At the end of this chapter, we will be able to run all the examples contained in the following chapters. You will have clear ideas and motivations why regression analysis is not just an underrated technique taken from statistics but a powerful and effective data science algorithm.

Chapter 2, Approaching Simple Linear Regression, presents the simple linear regression by first describing a regression problem, where to fit a regressor, and then giving some intuitions underneath the math formulation of its algorithm. Then, you will learn how to tune the model for higher performances and understand every parameter of it, deeply. Finally, the engine under the hood the gradient descent will be described.

Chapter 3, Multiple Regression in Action, extends the simple linear regression to extract predictive information from more than a feature and create models that can solve real-life prediction tasks. The stochastic gradient descent technique, explained in the previous chapter, will be powered up to cope with a matrix of features and to complete the overview, you will be shown multi-collinearity, interactions, and polynomial regression topics.

Chapter 4, Logistic Regression, continues laying down the foundations of your knowledge of linear model. Starting from the necessary mathematical definitions, it demonstrates how to furthermore extend the linear regression to classification problems, both binary and multiclass.

Chapter 5, Data Preparation, discusses about the data feeding the model, describing what can be done to prepare the data in the best way and how to deal with unusual situations, especially when data is missing and outliers are present.

Chapter 6, Achieving Generalization, will introduce you to the key data science recipes for testing your model thoroughly, tune it at its best, make it parsimonious, and to put it against real fresh data, before proceeding to more complex techniques.

Chapter 7, Online and Batch Learning, illustrates the best practices to train classifiers on Big Data; it first focuses on batch learning and its limitations and then introduces online learning. Finally, you will be showed an example of Big Data, combining the benefits of online learning and the power of the hashing trick.

Chapter 8, Advanced Regression Methods, introduces some advanced methods for regression. Without getting too deep into their mathematical formulation, but always keeping an eye on practical applications, we will discuss the ideas underneath Least Angle Regression, Bayesian Regression, and stochastic gradient descent with hinge loss, and also touch upon bagging and boosting techniques.

Chapter 9, Real-world Applications for Regression Models, comprises of four practical examples of real-world data science problems solved by linear models. The ultimate goal is to demonstrate how to approach such problems and how develop the reasoning around their resolution, so that they can be used as blueprints for similar challenges you'll encounter.

The execution of the code examples provided in this book requires an installation of Python 3.4.3 or newer on Mac OS X, Linux, or Microsoft Windows.

The code presented throughout the book will also make frequent use of Python's essential libraries for scientific and statistical computing such as SciPy, NumPy, Scikit-learn, Statsmodels, to a minor extent, matplotlib, and pandas.

Apart from taking advantage of Scientific distributions (such as Anaconda from Continuum Analytics) that can save you a lot of time from the time-consuming operations in order to create a working environment, we also suggest you to adopt Jupyter and its IPython Notebooks, as a more productive and scientific-friendly way to code your linear models in Python.

The first chapter will provide you with the step-by-step instructions and some useful tips to set up your Python environment, these core libraries and all the necessary tools.

First, this book is addressed to Python developers with at least a basic understanding of data science, statistics, and math. Though the book doesn't require a previous background in data science or statistics, it can well address data scientists of any seniority who intend to learn how to best do regression analysis on a dataset. We imagine that you want to put intelligence in your data products but you don't want to depend on any black box. Therefore, you prefer a technique which is simple, understandable, and yet effective to be production grade. Through the book, we will provide you with the knowledge to use Python for building fast and better linear models and to deploy the resulting models in Python or in any computer language you prefer.

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "When inspecting the linear model, first check the coef_ attribute."

A block of code is set as follows:

from sklearn import datasets 
iris = datasets.load_iris()  

Since we will be using IPython Notebooks along most of the examples, expect to have always an input (marked as In:) and often an output (marked Out:) from the cell containing the block of code. On your computer you have just to input the code after the In: and check if results correspond to the Out: content:

In: clf.fit(X, y)
Out: SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0, degree=3, gamma=0.0, kernel='rbf', max_iter=-1, probability=False, random_state=None, shrinking=True, tol=0.001, verbose=False) 

When a command should be given in the terminal command line, you'll find the command with the prefix $>, otherwise, if it's for the Python REPL it will be preceded by >>>:

    $>python
    >>> import sys

>>> print sys.version_info

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Go to the Databases section and create a new database using the UTF collation."

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.