Book Image

Statistics for Machine Learning

By : Pratap Dangeti
Book Image

Statistics for Machine Learning

By: Pratap Dangeti

Overview of this book

Complex statistics in machine learning worry a lot of developers. Knowing statistics helps you build strong machine learning models that are optimized for a given problem statement. This book will teach you all it takes to perform the complex statistical computations that are required for machine learning. You will gain information on the statistics behind supervised learning, unsupervised learning, reinforcement learning, and more. You will see real-world examples that discuss the statistical side of machine learning and familiarize yourself with it. You will come across programs for performing tasks such as modeling, parameter fitting, regression, classification, density collection, working with vectors, matrices, and more. By the end of the book, you will have mastered the statistics required for machine learning and will be able to apply your new skills to any sort of industry problem.
Table of Contents (16 chapters)
Title Page
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

SARSA on-policy TD control


State-action-reward-state-action (SARSA) is an on-policy TD control problem, in which policy will be optimized using policy iteration (GPI), only time TD methods used for evaluation of predicted policy. In the first step, the algorithm learns an SARSA function. In particular, for an on-policy method we estimate qπ (s, a) for the current behavior policy π and for all states (s) and actions (a), using the TD method for learning vπ. Now, we consider transitions from state-action pair to state-action pair, and learn the values of state-action pairs:

This update is done after every transition from a non-terminal state St. If St+1 is terminal, then Q (St+1, At+1) is defined as zero. This rule uses every element of the quintuple of events (St, At, Rt, St+1, At+1), which make up a transition from one state-action pair to the next. This quintuple gives rise to the name SARSA for the algorithm.

As in all on-policy methods, we continually estimate qπ for the behavior policy...