Book Image

Learning Predictive Analytics with Python

By : Ashish Kumar, Gary Dougan
Book Image

Learning Predictive Analytics with Python

By: Ashish Kumar, Gary Dougan

Overview of this book

Social Media and the Internet of Things have resulted in an avalanche of data. Data is powerful but not in its raw form - It needs to be processed and modeled, and Python is one of the most robust tools out there to do so. It has an array of packages for predictive modeling and a suite of IDEs to choose from. Learning to predict who would win, lose, buy, lie, or die with Python is an indispensable skill set to have in this data age. This book is your guide to getting started with Predictive Analytics using Python. You will see how to process data and make predictive models from it. We balance both statistical and mathematical concepts, and implement them in Python using libraries such as pandas, scikit-learn, and numpy. You’ll start by getting an understanding of the basics of predictive modeling, then you will see how to cleanse your data of impurities and get it ready it for predictive modeling. You will also learn more about the best predictive modeling algorithms such as Linear Regression, Decision Trees, and Logistic Regression. Finally, you will see the best practices in predictive modeling, as well as the different applications of predictive modeling in the modern world.

Related Content you might be interested in

Current Title:

Small book image
Learning Predictive Analytics with Python
Ashish Kumar | Gary Dougan
Feb, 2016 | 354 pages
No titles found
Table of Contents (19 chapters)
Learning Predictive Analytics with Python
Learning Predictive Analytics with Python
Credits
Credits
Foreword
Foreword
About the Author
About the Author
Acknowledgments
Acknowledgments
About the Reviewer
About the Reviewer
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Getting Started with Predictive Modelling
Getting Started with Predictive Modelling
Introducing predictive modelling
Applications and examples of predictive modelling
Python and its packages – download and installation
Python and its packages for predictive modelling
IDEs for Python
Summary
2
Data Cleaning
Data Cleaning
Reading the data – variations and examples
Various methods of importing data in Python
The read_csv method
Use cases of the read_csv method
Case 2 – reading a dataset using the open method of Python
Case 3 – reading data from a URL
Case 4 – miscellaneous cases
Basics – summary, dimensions, and structure
Handling missing values
Creating dummy variables
Visualizing a dataset by basic plotting
Summary
3
Data Wrangling
Data Wrangling
Subsetting a dataset
Generating random numbers and their usage
Grouping the data – aggregation, filtering, and transformation
Random sampling – splitting a dataset in training and testing datasets
Concatenating and appending data
Merging/joining datasets
Summary
4
Statistical Concepts for Predictive Modelling
Statistical Concepts for Predictive Modelling
Random sampling and the central limit theorem
Hypothesis testing
Chi-square tests
Correlation
Summary
5
Linear Regression with Python
Linear Regression with Python
Understanding the maths behind linear regression
Making sense of result parameters
Implementing linear regression with Python
Model validation
Handling other issues in linear regression
Summary
6
Logistic Regression with Python
Logistic Regression with Python
Linear regression versus logistic regression
Understanding the math behind logistic regression
Implementing logistic regression with Python
Model validation and evaluation
Model validation
Summary
7
Clustering with Python
Clustering with Python
Introduction to clustering – what, why, and how?
Mathematics behind clustering
Implementing clustering using Python
Fine-tuning the clustering
Summary
8
Trees and Random Forests with Python
Trees and Random Forests with Python
Introducing decision trees
Understanding the mathematics behind decision trees
Implementing a decision tree with scikit-learn
Understanding and implementing regression trees
Understanding and implementing random forests
Summary
9
Best Practices for Predictive Modelling
Best Practices for Predictive Modelling
Best practices for coding
Best practices for data handling
Best practices for algorithms
Best practices for statistics
Best practices for business contexts
Summary
A List of Links
A List of Links
Index
Index

Python and its packages – download and installation

There are various ways in which one can access and install Python and its packages. Here we will discuss a couple of them.

Anaconda

Anaconda is a popular Python distribution consisting of more than 195 popular Python packages. Installing Anaconda automatically installs many of the packages discussed in the preceding section, but they can be accessed only through an IDE called Spyder (more on this later in this chapter), which itself is installed on Anaconda installation. Anaconda also installs IPython Notebook and when you click on the IPython Notebook icon, it opens a browser tab and a Command Prompt.

Note

Anaconda can be downloaded and installed from the following web address: http://continuum.io/downloads

Download the suitable installer and double click on the .exe file and it will install Anaconda. Two of the features that you must check after the installation are:

  • IPython Notebook

  • Spyder IDE

Search for them in the "Start" icon's search, if it doesn't appear in the list of programs and files by default. We will be using IPython Notebook extensively and the codes in this book will work the best when run in IPython Notebook.

IPython Notebook can be opened by clicking on the icon. Alternatively, you can use the Command Prompt to open IPython Notebook. Just navigate to the directory where you have installed Anaconda and then write ipython notebook, as shown in the following screenshot:

Fig. 1.3: Opening IPython Notebook

Note

On the system used for this book, Anaconda was installed in the C:\Users\ashish directory. One can open a new Notebook in IPython by clicking on the New Notebook button on the dashboard, which opens up. In this book, we have used IPython Notebook extensively.

Standalone Python

You can download a Python version that is stable and is compatible to the OS on your system. The most stable version of Python is 2.7.0. So, installing this version is highly recommended. You can download it from https://www.python.org/ and install it.

There are some Python packages that you need to install on your machine before you start predictive analytics and modelling. This section consists of a demo of installation of one such library and a brief description of all such libraries.

Installing a Python package

There are several ways to install a Python package. The easiest and the most effective is the one using pip. As you might be aware, pip is a package management system that is used to install and manage software packages written in Python. To be able to use it to install other packages, pip needs to be installed first.

Installing pip

The following steps demonstrate how to install pip. Follow closely!

  1. Navigate to the webpage shown in the following screenshot. The URL address is https://pypi.python.org/pypi/pip:

    Downloading pip from the Python's official website

  2. Download the pip-7.0.3.tar.gz file and unzip in the folder where Python is installed. If you have Python v2.7.0 installed, this folder should be C:\Python27:

    Unzipping the .zar file for pip in the correct folder

  3. On unzipping the previously mentioned file, a folder called pip-7.0.3 is created. Opening that folder will take you to the screen similar to the one in the preceding screenshot.

  4. Open the CMD on your computer and change the current directory to the current directory in the preceding screenshot that is C:\Python27\pip-7.0.3 using the following command:

    cd C:\Python27\pip-7.0.3.

  5. The result of the preceding command is shown in the following screenshot:

    Navigating to the directory where pip is installed

  6. Now, the current directory is set to the directory where setup file for pip (setup.py) resides. Write the following command to install pip: 

    python setup.py install

  7. The result of the preceding command is shown in the following screenshot:

    Installing pip using a command line

Once pip is installed, it is very easy to install all the required Python packages to get started.

Installing Python packages with pip

The following are the steps to install Python packages using pip, which we just installed in the preceding section:

  1. Change the current directory in the command prompt to the directory where the Python v2.7.0 is installed that is: C:\Python27.

  2. Write the following command to install the package:

    pip install package-name

  3. For example, to install pandas, you can proceed as follows:

    Installing a Python package using a command line and pip

  4. Finally, to confirm that the package has installed successfully, write the following command:

    python  -c "import pandas"

  5. The result of the preceding command is shown in the following screenshot:

    Checking whether the package has installed correctly or not

If this doesn't throw up an error, then the package has been installed successfully.

Previous Section
End of Section 4
Next Section