# What this book covers

*Chapter 1*, *What Is Reinforcement Learning?*, contains an introduction to RL ideas and the main formal models.

*Chapter 2*, *OpenAI Gym*, introduces the practical aspects of RL, using the open source library Gym.

*Chapter 3*, *Deep Learning with PyTorch*, gives a quick overview of the PyTorch library.

*Chapter 4*, *The Cross-Entropy Method*, introduces one of the simplest methods in RL to give you an impression of RL methods and problems.

*Chapter 5*, *Tabular Learning and the Bellman Equation*, introduces the value-based family of RL methods.

*Chapter 6*, *Deep Q-Networks*, describes deep Q-networks (DQNs), an extension of the basic value-based methods, allowing us to solve a complicated environment.

*Chapter 7*, *Higher-Level RL Libraries*, describes the library PTAN, which we will use in the book to simplify the implementations of RL methods.

*Chapter 8*, *DQN Extensions*, gives a detailed overview of a modern extension to the DQN method, to improve its stability and convergence in complex environments.

*Chapter 9*, *Ways to Speed up RL Methods*, provides an overview of ways to make the execution of RL code faster.

*Chapter 10*, *Stocks Trading Using RL*, is the first practical project and focuses on applying the DQN method to stock trading.

*Chapter 11*, *Policy Gradients—an Alternative*, introduces another family of RL methods that is based on policy learning.

*Chapter 12*, *The Actor-Critic Method*, describes one of the most widely used methods in RL.

*Chapter 13*, *Asynchronous Advantage Actor-Critic*, extends the actor-critic method with parallel environment communication, which improves stability and convergence.

*Chapter 14*, *Training Chatbots with RL*, is the second project and shows how to apply RL methods to natural language processing problems.

*Chapter 15*, *The TextWorld Environment*, covers the application of RL methods to interactive fiction games.

*Chapter 16*, *Web Navigation*, is another long project that applies RL to web page navigation using the MiniWoB set of tasks.

*Chapter 17*, *Continuous Action Space*, describes the specifics of environments using continuous action spaces and various methods.

*Chapter 18*, *RL in Robotics*, covers the application of RL methods to robotics problems. In this chapter, I describe the process of building and training a small hardware robot with RL methods.

*Chapter 19*, *Trust Regions – PPO, TRPO, ACKTR, and SAC*, is yet another chapter about continuous action spaces describing the trust region set of methods.

*Chapter 20*, *Black-Box Optimization in RL*, shows another set of methods that don’t use gradients in their explicit form.

*Chapter 21*, *Advanced Exploration*, covers different approaches that can be used for better exploration of the environment.

*Chapter 22*, *Beyond Model-Free – Imagination*, introduces the model-based approach to RL and uses recent research results about imagination in RL.

*Chapter 23*, *AlphaGo Zero*, describes the AlphaGo Zero method and applies it to the game Connect 4.

*Chapter 24*, *RL in Discrete Optimization*, describes the application of RL methods to the domain of discrete optimization, using the Rubik’s Cube as an environment.

*Chapter 25*, *Multi-agent RL*, introduces a relatively new direction of RL methods for situations with multiple agents.