Book Image

Practical Data Science with Python

By : Nathan George

Book Image

Practical Data Science with Python

By: Nathan George

Overview of this book

Practical Data Science with Python teaches you core data science concepts, with real-world and realistic examples, and strengthens your grip on the basic as well as advanced principles of data preparation and storage, statistics, probability theory, machine learning, and Python programming, helping you build a solid foundation to gain proficiency in data science. The book starts with an overview of basic Python skills and then introduces foundational data science techniques, followed by a thorough explanation of the Python code needed to execute the techniques. You'll understand the code by working through the examples. The code has been broken down into small chunks (a few lines or a function at a time) to enable thorough discussion. As you progress, you will learn how to perform data analysis while exploring the functionalities of key data science Python packages, including pandas, SciPy, and scikit-learn. Finally, the book covers ethics and privacy concerns in data science and suggests resources for improving data science skills, as well as ways to stay up to date on new data science developments. By the end of the book, you should be able to comfortably use Python for basic data science projects and should have the skills to execute the data science process on any data source.

Preface

Who this book is for

What this book covers

To get the most out of this book

Part I - An Introduction and the Basics

Part I - An Introduction and the Basics

Free Chapter

Introduction to Data Science

Introduction to Data Science

The data science origin story

The top data science tools and skills

Specializations in and around data science

Data science project methodologies

Test your knowledge

Getting Started with Python

Getting Started with Python

Installing Python with Anaconda and getting started

Software engineering best practices

Test your knowledge

Part II - Dealing with Data

Part II - Dealing with Data

SQL and Built-in File Handling Modules in Python

Loading, reading, and writing files with base Python

Using SQLite and SQL

Using the SQLAlchemy package in Python

Test your knowledge

Loading and Wrangling Data with Pandas and NumPy

Loading and Wrangling Data with Pandas and NumPy

Data wrangling and analyzing iTunes data

Understanding NumPy basics

Using NumPy mathematical functions

Test your knowledge

Exploratory Data Analysis and Visualization

Exploratory Data Analysis and Visualization

EDA and visualization libraries in Python

Performing EDA with Seaborn and pandas

Using EDA Python packages

Using visualization best practices

Making plots with Plotly

Test your knowledge

Data Wrangling Documents and Spreadsheets

Data Wrangling Documents and Spreadsheets

Parsing and processing Word and PDF documents

Reading and writing data with Excel files

Test your knowledge

Web Scraping

Understanding the structure of the internet

Performing simple web scraping

Parsing HTML from scraped pages

Using APIs to collect data

The ethics and legality of web scraping

Test your knowledge

Part III - Statistics for Data Science

Part III - Statistics for Data Science

Probability, Distributions, and Sampling

Probability, Distributions, and Sampling

Probability basics

Sampling from data

Test your knowledge

Statistical Testing for Data Science

Statistical Testing for Data Science

Statistical testing basics and sample comparison tests

Other statistical tests

Test your knowledge

Part IV - Machine Learning

Part IV - Machine Learning

Preparing Data for Machine Learning: Feature Selection, Feature Engineering, and Dimensionality Reduction

Preparing Data for Machine Learning: Feature Selection, Feature Engineering, and Dimensionality Reduction

Types of machine learning

Feature selection

Feature engineering

Dimensionality reduction

Test your knowledge

Machine Learning for Classification

Machine Learning for Classification

Machine learning classification algorithms

Test your knowledge

Evaluating Machine Learning Classification Models and Sampling for Classification

Evaluating Machine Learning Classification Models and Sampling for Classification

Evaluating classification algorithm performance with metrics

Sampling and balancing classification data

Test your knowledge

Machine Learning with Regression

Machine Learning with Regression

Linear regression

Regression models on big data

Test your knowledge

Optimizing Models and Using AutoML

Optimizing Models and Using AutoML

Hyperparameter optimization with search methods

Using learning curves

Optimizing the number of features with ML models

Using AutoML with PyCaret

Test your knowledge

Tree-Based Machine Learning Models

Tree-Based Machine Learning Models

Feature importance from tree-based methods

Boosted trees: AdaBoost, XGboost, LightGBM, and CatBoost

Test your knowledge

Support Vector Machine (SVM) Machine Learning Models

Support Vector Machine (SVM) Machine Learning Models

Test your knowledge

Part V - Text Analysis and Reporting

Part V - Text Analysis and Reporting

Clustering with Machine Learning

Clustering with Machine Learning

Using k-means clustering

Hierarchical clustering

Other unsupervised methods

Test your knowledge

Working with Text

Working with Text

Text preprocessing

Basic text analysis

Unsupervised learning

Supervised learning

Sentiment analysis

Test your knowledge

Part VI - Wrapping Up

Part VI - Wrapping Up

Data Storytelling and Automated Reporting/Dashboarding

Data Storytelling and Automated Reporting/Dashboarding

Data storytelling

Automated reporting and dashboarding

Test your knowledge

Ethics and Privacy

Ethics and Privacy

The ethics of machine learning algorithms

Data privacy regulations and laws

k-anonymity, l-diversity, and t-closeness

Differential privacy

Using data science for the common good

Other ethical considerations

Test your knowledge

Staying Up to Date and the Future of Data Science

Staying Up to Date and the Future of Data Science

Blogs, newsletters, books, and academic sources

Data science competition websites

Online learning platforms

Other places to keep an eye on

Strategies for staying up to date

Other data science topics we didn't cover

The future of data science

Other Books You May Enjoy

Other Books You May Enjoy

Index

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Feature importance from tree-based methods

Feature importance, also called variable importance, can be calculated from tree-based methods by summing the reduction in Gini or entropy over all the trees for each variable.

So, if a particular variable is used to split the data and reduces the Gini or entropy value by a large amount, that feature is important for making predictions. This is a nice contrast to using coefficient-based feature importance from logistic or linear regression, because tree-based feature importances are non-linear. There are other ways of calculating feature importance as well, such as permutation feature importance and SHAP (SHapley Additive exPlanations).

Using H2O for feature importance

We can easily get the importances with drf.varimp(), or plot them with drf.varimp_plot(server=True). The server=True argument uses matplotlib, which allows us to do things such as directly saving the figure with plt.savefig(). The result looks like this:

...