Book Image

Practical Data Science with Python

By : Nathan George

Book Image

Practical Data Science with Python

By: Nathan George

Overview of this book

Practical Data Science with Python teaches you core data science concepts, with real-world and realistic examples, and strengthens your grip on the basic as well as advanced principles of data preparation and storage, statistics, probability theory, machine learning, and Python programming, helping you build a solid foundation to gain proficiency in data science. The book starts with an overview of basic Python skills and then introduces foundational data science techniques, followed by a thorough explanation of the Python code needed to execute the techniques. You'll understand the code by working through the examples. The code has been broken down into small chunks (a few lines or a function at a time) to enable thorough discussion. As you progress, you will learn how to perform data analysis while exploring the functionalities of key data science Python packages, including pandas, SciPy, and scikit-learn. Finally, the book covers ethics and privacy concerns in data science and suggests resources for improving data science skills, as well as ways to stay up to date on new data science developments. By the end of the book, you should be able to comfortably use Python for basic data science projects and should have the skills to execute the data science process on any data source.

Preface

Who this book is for

What this book covers

To get the most out of this book

Part I - An Introduction and the Basics

Part I - An Introduction and the Basics

Free Chapter

Introduction to Data Science

Introduction to Data Science

The data science origin story

The top data science tools and skills

Specializations in and around data science

Data science project methodologies

Test your knowledge

Getting Started with Python

Getting Started with Python

Installing Python with Anaconda and getting started

Software engineering best practices

Test your knowledge

Part II - Dealing with Data

Part II - Dealing with Data

SQL and Built-in File Handling Modules in Python

Loading, reading, and writing files with base Python

Using SQLite and SQL

Using the SQLAlchemy package in Python

Test your knowledge

Loading and Wrangling Data with Pandas and NumPy

Loading and Wrangling Data with Pandas and NumPy

Data wrangling and analyzing iTunes data

Understanding NumPy basics

Using NumPy mathematical functions

Test your knowledge

Exploratory Data Analysis and Visualization

Exploratory Data Analysis and Visualization

EDA and visualization libraries in Python

Performing EDA with Seaborn and pandas

Using EDA Python packages

Using visualization best practices

Making plots with Plotly

Test your knowledge

Data Wrangling Documents and Spreadsheets

Data Wrangling Documents and Spreadsheets

Parsing and processing Word and PDF documents

Reading and writing data with Excel files

Test your knowledge

Web Scraping

Understanding the structure of the internet

Performing simple web scraping

Parsing HTML from scraped pages

Using APIs to collect data

The ethics and legality of web scraping

Test your knowledge

Part III - Statistics for Data Science

Part III - Statistics for Data Science

Probability, Distributions, and Sampling

Probability, Distributions, and Sampling

Probability basics

Sampling from data

Test your knowledge

Statistical Testing for Data Science

Statistical Testing for Data Science

Statistical testing basics and sample comparison tests

Other statistical tests

Test your knowledge

Part IV - Machine Learning

Part IV - Machine Learning

Preparing Data for Machine Learning: Feature Selection, Feature Engineering, and Dimensionality Reduction

Preparing Data for Machine Learning: Feature Selection, Feature Engineering, and Dimensionality Reduction

Types of machine learning

Feature selection

Feature engineering

Dimensionality reduction

Test your knowledge

Machine Learning for Classification

Machine Learning for Classification

Machine learning classification algorithms

Test your knowledge

Evaluating Machine Learning Classification Models and Sampling for Classification

Evaluating Machine Learning Classification Models and Sampling for Classification

Evaluating classification algorithm performance with metrics

Sampling and balancing classification data

Test your knowledge

Machine Learning with Regression

Machine Learning with Regression

Linear regression

Regression models on big data

Test your knowledge

Optimizing Models and Using AutoML

Optimizing Models and Using AutoML

Hyperparameter optimization with search methods

Using learning curves

Optimizing the number of features with ML models

Using AutoML with PyCaret

Test your knowledge

Tree-Based Machine Learning Models

Tree-Based Machine Learning Models

Feature importance from tree-based methods

Boosted trees: AdaBoost, XGboost, LightGBM, and CatBoost

Test your knowledge

Support Vector Machine (SVM) Machine Learning Models

Support Vector Machine (SVM) Machine Learning Models

Test your knowledge

Part V - Text Analysis and Reporting

Part V - Text Analysis and Reporting

Clustering with Machine Learning

Clustering with Machine Learning

Using k-means clustering

Hierarchical clustering

Other unsupervised methods

Test your knowledge

Working with Text

Working with Text

Text preprocessing

Basic text analysis

Unsupervised learning

Supervised learning

Sentiment analysis

Test your knowledge

Part VI - Wrapping Up

Part VI - Wrapping Up

Data Storytelling and Automated Reporting/Dashboarding

Data Storytelling and Automated Reporting/Dashboarding

Data storytelling

Automated reporting and dashboarding

Test your knowledge

Ethics and Privacy

Ethics and Privacy

The ethics of machine learning algorithms

Data privacy regulations and laws

k-anonymity, l-diversity, and t-closeness

Differential privacy

Using data science for the common good

Other ethical considerations

Test your knowledge

Staying Up to Date and the Future of Data Science

Staying Up to Date and the Future of Data Science

Blogs, newsletters, books, and academic sources

Data science competition websites

Online learning platforms

Other places to keep an eye on

Strategies for staying up to date

Other data science topics we didn't cover

The future of data science

Other Books You May Enjoy

Other Books You May Enjoy

Index

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Linear regression

Linear regression has been around since the 1800s but is still used today. It is an easy-to-use and -interpret method that generally works for many datasets as long as we have relationships between our features and target that are somewhat linear and a few other assumptions are met.

With linear regression, we predict continuous values based on our features, and our results may look something like this:

Figure 13.1: An example of a linear regression fit to data

Here, we have a scatter plot of square feet of the first floor of a house on the x-axis and the sale price on the y-axis. We can see a generally linear relationship holds, with higher prices corresponding to bigger square footage. The line shows a linear fit to the data.

To fit a simple 1-D linear model as shown above, we use the equation:

Where m is the coefficient for our input feature and b is the value at which the line intersects the y-axis (the y-intercept). We can generalize...