Book Image

Reinforcement Learning with TensorFlow

By : Sayon Dutta
Book Image

Reinforcement Learning with TensorFlow

By: Sayon Dutta

Overview of this book

Reinforcement learning (RL) allows you to develop smart, quick and self-learning systems in your business surroundings. It's an effective method for training learning agents and solving a variety of problems in Artificial Intelligence - from games, self-driving cars and robots, to enterprise applications such as data center energy saving (cooling data centers) and smart warehousing solutions. The book covers major advancements and successes achieved in deep reinforcement learning by synergizing deep neural network architectures with reinforcement learning. You'll also be introduced to the concept of reinforcement learning, its advantages and the reasons why it's gaining so much popularity. You'll explore MDPs, Monte Carlo tree searches, dynamic programming such as policy and value iteration, and temporal difference learning such as Q-learning and SARSA. You will use TensorFlow and OpenAI Gym to build simple neural network models that learn from their own actions. You will also see how reinforcement learning algorithms play a role in games, image processing and NLP. By the end of this book, you will have gained a firm understanding of what reinforcement learning is and understand how to put your knowledge to practical use by leveraging the power of TensorFlow and OpenAI Gym.
Table of Contents (21 chapters)
Title Page
Packt Upsell
Contributors
Preface
Index

Partially observable Markov decision processes


In an MDP, the observable quantities are action, set A, the state, set S, transition model, T, and rewards, set R. This is not in case of Partially observable MDP, also known as POMDP. In a POMDP, there's an MDP inside that is not directly observable to the agent and takes the decision from whatever observations made. 

In POMDP, there's an observation set, Z, containing different observable states and a observation function, O, which takes the s state and the z observation as inputs and outputs the probability of seeing that z observation in the s state.

POMDPs are basically a generalization of MDPs:

  • MDP: {S,A,T,R}

  • POMDP: {S,A,Z,T,R,O}

  • where, S, A, T ,and R are the same. Therefore, for a POMDP to be a true MDP, following condition:

, that is, fully observe all states

POMDP are hugely intractable to solve optimally.

State estimation

If we expand the state spaces, this helps us to convert the POMDP into an MDP where Z contains fully observable state space...