Book Image

Mastering Reinforcement Learning with Python

By : Enes Bilgin
Book Image

Mastering Reinforcement Learning with Python

By: Enes Bilgin

Overview of this book

Reinforcement learning (RL) is a field of artificial intelligence (AI) used for creating self-learning autonomous agents. Building on a strong theoretical foundation, this book takes a practical approach and uses examples inspired by real-world industry problems to teach you about state-of-the-art RL. Starting with bandit problems, Markov decision processes, and dynamic programming, the book provides an in-depth review of the classical RL techniques, such as Monte Carlo methods and temporal-difference learning. After that, you will learn about deep Q-learning, policy gradient algorithms, actor-critic methods, model-based methods, and multi-agent reinforcement learning. Then, you'll be introduced to some of the key approaches behind the most successful RL implementations, such as domain randomization and curiosity-driven learning. As you advance, you’ll explore many novel algorithms with advanced implementations using modern Python libraries such as TensorFlow and Ray’s RLlib package. You’ll also find out how to implement RL in areas such as robotics, supply chain management, marketing, finance, smart cities, and cybersecurity while assessing the trade-offs between different approaches and avoiding common pitfalls. By the end of this book, you’ll have mastered how to train and deploy your own RL agents for solving RL problems.
Table of Contents (24 chapters)
Section 1: Reinforcement Learning Foundations
Section 2: Deep Reinforcement Learning
Section 3: Advanced Topics in RL
Section 4: Applications of RL

Partially observable Markov decision process

The definition of policy we have used in this chapter so far is that it is a mapping from the states of an environment to actions. Now, the question we should ask is whether the state is really known to the agent in all types of environments.

Remember the definition of the state: it describes everything in an environment related to the agent's decision-making (in the grid world example, if the color of the walls is not important, for instance, so it will not be part of the state).

If you think about it: This is a very strong definition. Consider the situation when someone is driving a car. Does the driver know everything about the world around them while making their driving decisions? Of course not! To begin with, the cars would be blocking each other in their sight more often than not. Not knowing the precise state of the world does not stop anyone from driving, though. In such cases, we base our decision on our observations...