Book Image

Numerical Computing with Python

By : Pratap Dangeti, Allen Yu, Claire Chung, Aldrin Yim, Theodore Petrou
Book Image

Numerical Computing with Python

By: Pratap Dangeti, Allen Yu, Claire Chung, Aldrin Yim, Theodore Petrou

Overview of this book

Data mining, or parsing the data to extract useful insights, is a niche skill that can transform your career as a data scientist Python is a flexible programming language that is equipped with a strong suite of libraries and toolkits, and gives you the perfect platform to sift through your data and mine the insights you seek. This Learning Path is designed to familiarize you with the Python libraries and the underlying statistics that you need to get comfortable with data mining. You will learn how to use Pandas, Python's popular library to analyze different kinds of data, and leverage the power of Matplotlib to generate appealing and impressive visualizations for the insights you have derived. You will also explore different machine learning techniques and statistics that enable you to build powerful predictive models. By the end of this Learning Path, you will have the perfect foundation to take your data mining skills to the next level and set yourself on the path to become a sought-after data science professional. This Learning Path includes content from the following Packt products: • Statistics for Machine Learning by Pratap Dangeti • Matplotlib 2.x By Example by Allen Yu, Claire Chung, Aldrin Yim • Pandas Cookbook by Theodore Petrou

Related Content you might be interested in

Current Title:

Small book image
Numerical Computing with Python
Pratap Dangeti | Allen Yu | Claire Chung | Aldrin Yim | Theodore Petrou
Dec, 2018 | 11 hours 22 minutes
No titles found
Table of Contents (21 chapters)
Title Page
Title Page
Contributors
Contributors
About Packt
About Packt
Preface
Preface
Free Chapter
1
Journey from Statistics to Machine Learning
Journey from Statistics to Machine Learning
Statistical terminology for model building and validation
Summary
2
Tree-Based Machine Learning Models
Tree-Based Machine Learning Models
Introducing decision tree classifiers
Comparison between logistic regression and decision trees
Comparison of error components across various styles of models
Remedial actions to push the model towards the ideal region
HR attrition data example
Decision tree classifier
Tuning class weights in decision tree classifier
Bagging classifier
Random forest classifier
Random forest classifier - grid search
AdaBoost classifier
Gradient boosting classifier
Comparison between AdaBoosting versus gradient boosting
Extreme gradient boosting - XGBoost classifier
Ensemble of ensembles - model stacking
Ensemble of ensembles with different types of classifiers
Ensemble of ensembles with bootstrap samples using a single type of classifier
Summary
3
K-Nearest Neighbors and Naive Bayes
K-Nearest Neighbors and Naive Bayes
K-nearest neighbors
KNN classifier with breast cancer Wisconsin data example
Tuning of k-value in KNN classifier
Naive Bayes
Probability fundamentals
Understanding Bayes theorem with conditional probability
Naive Bayes classification
Laplace estimator
Naive Bayes SMS spam classification example
Summary
4
Unsupervised Learning
Unsupervised Learning
K-means clustering
Principal Component Analysis - PCA
Singular value decomposition - SVD
Deep auto encoders
Model building technique using encoder-decoder architecture
Deep auto encoders applied on handwritten digits using Keras
Summary
5
Reinforcement Learning
Reinforcement Learning
Reinforcement learning basics
Markov decision processes and Bellman equations
Dynamic programming
Grid world example using value and policy iteration algorithms with basic Python
Monte Carlo methods
Temporal difference learning
SARSA on-policy TD control
Q-learning - off-policy TD control
Cliff walking example of on-policy and off-policy of TD control
Further reading
Summary
6
Hello Plotting World!
Hello Plotting World!
Hello Matplotlib!
Plotting our first graph
Summary
7
Visualizing Online Data
Visualizing Online Data
Typical API data formats
Introducing pandas
Visualizing the trend of data
Introducing Seaborn
Visualizing univariate distribution
Visualizing a bivariate distribution
Visualizing categorical data
Controlling Seaborn figure aesthetics
Summary
8
Visualizing Multivariate Data
Visualizing Multivariate Data
Getting End-of-Day (EOD) stock data from Quandl
Two-dimensional faceted plots
Other two-dimensional multivariate plots
Three-dimensional (3D) plots
Summary
9
Adding Interactivity and Animating Plots
Adding Interactivity and Animating Plots
Scraping information from websites
Non-interactive backends
Interactive backends
Creating animated plots
Summary
10
Selecting Subsets of Data
Selecting Subsets of Data
Selecting Series data
Selecting DataFrame rows
Selecting DataFrame rows and columns simultaneously
Selecting data with both integers and labels
Speeding up scalar selection
Slicing rows lazily
Slicing lexicographically
11
Boolean Indexing
Boolean Indexing
Calculating boolean statistics
Constructing multiple boolean conditions
Filtering with boolean indexing
Replicating boolean indexing with index selection
Selecting with unique and sorted indexes
Gaining perspective on stock prices
Translating SQL WHERE clauses
Determining the normality of stock market returns
Improving readability of boolean indexing with the query method
Preserving Series with the where method
Masking DataFrame rows
Selecting with booleans, integer location, and labels
12
Index Alignment
Index Alignment
Examining the Index object
Producing Cartesian products
Exploding indexes
Filling values with unequal indexes
Appending columns from different DataFrames
Highlighting the maximum value from each column
Replicating idxmax with method chaining
Finding the most common maximum
13
Grouping for Aggregation, Filtration, and Transformation
Grouping for Aggregation, Filtration, and Transformation
Defining an aggregation
Grouping and aggregating with multiple columns and functions
Removing the MultiIndex after grouping
Customizing an aggregation function
Customizing aggregating functions with *args and **kwargs
Examining the groupby object
Filtering for states with a minority majority
Transforming through a weight loss bet
Calculating weighted mean SAT scores per state with apply
Grouping by continuous variables
Counting the total number of flights between cities
Finding the longest streak of on-time flights
14
Restructuring Data into a Tidy Form
Restructuring Data into a Tidy Form
Tidying variable values as column names with stack
Tidying variable values as column names with melt
Stacking multiple groups of variables simultaneously
Inverting stacked data
Unstacking after a groupby aggregation
Replicating pivot_table with a groupby aggregation
Renaming axis levels for easy reshaping
Tidying when multiple variables are stored as column names
Tidying when multiple variables are stored as column values
Tidying when two or more values are stored in the same cell
Tidying when variables are stored in column names and values
Tidying when multiple observational units are stored in the same table
15
Combining Pandas Objects
Combining Pandas Objects
Appending new rows to DataFrames
Concatenating multiple DataFrames together
Comparing President Trump's and Obama's approval ratings
Understanding the differences between concat, join, and merge
Connecting to SQL databases
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index
Index

SARSA on-policy TD control

State-action-reward-state-action (SARSA) is an on-policy TD control problem, in which policy will be optimized using policy iteration (GPI), only time TD methods used for evaluation of predicted policy. In the first step, the algorithm learns a SARSA function. In particular, for an on-policy method we estimate qπ (s, a) for the current behavior policy π and for all states (s) and actions (a), using the TD method for learning vπ.

 

Now, we consider transitions from state-action pair to state-action pair, and learn the values of state-action pairs:

This update is done after every transition from a non-terminal state St. If St+1 is terminal, then Q (St+1, At+1) is defined as zero. This rule uses every element of the quintuple of events (St, At, Rt, St+1, At+1), which make up a transition from one state-action pair to the next. This quintuple gives rise to the name SARSA for the algorithm.

As in all on-policy methods, we continually estimate qπ for the behavior policy...

Previous Section
End of Section 8
Next Section