Book Image

Data Science with Python

By : Rohan Chopra, Aaron England, Mohamed Noordeen Alaudeen
Book Image

Data Science with Python

By: Rohan Chopra, Aaron England, Mohamed Noordeen Alaudeen

Overview of this book

Data Science with Python begins by introducing you to data science and teaches you to install the packages you need to create a data science coding environment. You will learn three major techniques in machine learning: unsupervised learning, supervised learning, and reinforcement learning. You will also explore basic classification and regression techniques, such as support vector machines, decision trees, and logistic regression. As you make your way through the book, you will understand the basic functions, data structures, and syntax of the Python language that are used to handle large datasets with ease. You will learn about NumPy and pandas libraries for matrix calculations and data manipulation, discover how to use Matplotlib to create highly customizable visualizations, and apply the boosting algorithm XGBoost to make predictions. In the concluding chapters, you will explore convolutional neural networks (CNNs), deep learning algorithms used to predict what is in an image. You will also understand how to feed human sentences to a neural network, make the model process contextual information, and create human language processing systems to predict the outcome. By the end of this book, you will be able to understand and implement any new data science algorithm and have the confidence to experiment with tools or libraries other than those covered in the book.

Related Content you might be interested in

Current Title:

Small book image
Data Science with Python
Rohan Chopra | Aaron England | Mohamed Noordeen Alaudeen
Jul, 2019 | 426 pages
Small book image
Applied Deep Learning with Keras
Matthew Moocarme | Ritesh Bhagwat | Mahla Abdolahnejad
Apr, 2019 | 412 pages
Small book image
Machine Learning Fundamentals
Hyatt Saleh
Nov, 2018 | 240 pages
Small book image
Ensemble Machine Learning Cookbook
Vijayalakshmi Natarajan | Dipayan Sarkar
Jan, 2019 | 336 pages
Table of Contents (10 chapters)
About the Book
About the Book
About the Authors
Learning Objectives
Audience
Approach
Minimum Hardware Requirements
Software Requirements
Installation and Setup
Using Kaggle for Faster Experimentation
Conventions
Installing the Code Bundle
Free Chapter
1
Introduction to Data Science and Data Pre-Processing
Introduction to Data Science and Data Pre-Processing
Introduction
Python Libraries
Roadmap for Building Machine Learning Models
Data Representation
Data Cleaning
Data Integration
Data Transformation
Data in Different Scales
Data Discretization
Train and Test Data
Supervised Learning
Unsupervised Learning
Reinforcement Learning
Performance Metrics
Summary
2
Data Visualization
Data Visualization
Introduction
Functional Approach
Object-Oriented Approach Using Subplots
Summary
3
Introduction to Machine Learning via Scikit-Learn
Introduction to Machine Learning via Scikit-Learn
Introduction
Introduction to Linear and Logistic Regression
Multiple Linear Regression
Logistic Regression
Max Margin Classification Using SVMs
Decision Trees
Random Forests
Summary
4
Dimensionality Reduction and Unsupervised Learning
Dimensionality Reduction and Unsupervised Learning
Introduction
Hierarchical Cluster Analysis (HCA)
K-means Clustering
Principal Component Analysis (PCA)
Supervised Data Compression using Linear Discriminant Analysis (LDA)
Summary
5
Mastering Structured Data
Mastering Structured Data
Introduction
Boosting Algorithms
XGBoost Library
External Memory Usage
Cross-validation
Saving and Loading a Model
Neural Networks
Keras
Categorical Variables
Summary
6
Decoding Images
Decoding Images
Introduction
Images
Convolutional Neural Networks
Pooling Layer
Adam Optimizer
Cross-entropy Loss
Regularization
Image Data Preprocessing
Data Augmentation
Generators
Summary
7
Processing Human Language
Processing Human Language
Introduction
Text Data Processing
Recurrent Neural Networks (RNNs)
Summary
8
Tips and Tricks of the Trade
Tips and Tricks of the Trade
Introduction
Transfer Learning
Useful Tools and Tips
AutoML
Summary
Appendix
Appendix
Chapter 1: Introduction to Data Science and Data Preprocessing
Chapter 2: Data Visualization
Chapter 3: Introduction to Machine Learning via Scikit-Learn
Chapter 4: Dimensionality Reduction and Unsupervised Learning
Chapter 5: Mastering Structured Data
Chapter 6: Decoding Images
Chapter 7: Processing Human Language
Chapter 8: Tips and Tricks of the Trade

Data Integration

So far, we've made sure to remove the impurities in data and make it clean. Now, the next step is to combine data from different sources to get a unified structure with more meaningful and valuable information. This is mostly used if the data is segregated into different sources. To make it simple, let's assume we have data in CSV format in different places, all talking about the same scenario. Say we have some data about an employee in a database. We can't expect all the data about the employee to reside in the same table. It's possible that the employee's personal data will be located in one table, the employee's project history will be in a second table, the employee's time-in and time-out details will be in another table, and so on. So, if we want to do some analysis about the employee, we need to get all the employee data in one common place. This process of bringing data together in one place is called data integration. To do data integration, we can merge multiple pandas DataFrames using the merge function.

Let's solve an exercise based on data integration to get a clear understanding of it.

Exercise 5: Integrating Data

In this exercise, we'll merge the details of students from two datasets, namely student.csv and marks.csv. The student dataset contains columns such as Age, Gender, Grade, and Employed. The marks.csv dataset contains columns such as Mark and City. The Student_id column is common between the two datasets. Follow these steps to complete this exercise:

Note

The student.csv dataset can be found at this location: https://github.com/TrainingByPackt/Data-Science-with-Python/blob/master/Chapter01/Data/student.csv.

The marks.csv dataset can be found at this location: https://github.com/TrainingByPackt/Data-Science-with-Python/blob/master/Chapter01/Data/mark.csv.

  1. Open a Jupyter notebook and add a new cell. Write the following code to import pandas and load the student.csv and marks.csv datasets into the df1 and df2 pandas DataFrames:

    import pandas as pd

    dataset1 = "https://github.com/TrainingByPackt/Data-Science-with-Python/blob/master/Chapter01/Data/student.csv"

    dataset2 = "https://github.com/TrainingByPackt/Data-Science-with-Python/blob/master/Chapter01/Data/mark.csv"

    df1 = pd.read_csv(dataset1, header = 0)

    df2 = pd.read_csv(dataset2, header = 0)

  2. To print the first five rows of the first DataFrame, add the following code:

    df1.head()

    The preceding code generates the following output:

    Figure 1.25: The first five rows of the first DataFrame
    Figure 1.25: The first five rows of the first DataFrame
  3. To print the first five rows of the second DataFrame, add the following code:

    df2.head()

    The preceding code generates the following output:

    Figure 1.26: The first five rows of the second DataFrame
    Figure 1.26: The first five rows of the second DataFrame
  4. Student_id is common to both datasets. Perform data integration on both the DataFrames with respect to the Student_id column using the pd.merge() function, and then print the first 10 values of the new DataFrame:

    df = pd.merge(df1, df2, on = 'Student_id')

    df.head(10)

Figure 1.27: First 10 rows of the merged DataFrame
Figure 1.27: First 10 rows of the merged DataFrame

Here, the data of the df1 DataFrame is merged with the data of the df2 DataFrame. The merged data is stored inside a new DataFrame called df.

We have now learned how to perform data integration. In the next section, we'll explore another pre-processing task, data transformation.

Previous Section
End of Section 7
Next Section