Chapter 1, Brushing Up on Reinforcement Learning Concepts, covers the building-block concepts of agents, environments, states, actions, rewards, policies, and values. The reader will be introduced to stochastic and deterministic environments, learning rates, and exploration versus exploitation.
Chapter 2, Getting Started with the Q-Learning Algorithm, covers in great depth how a Markov decision process works and the way a Q-learning algorithm is designed to solve it. We will discuss what kinds of problems can and can't be solved with a model-free algorithm, as well as the types of problems that Q-learning is especially well suited to.
Chapter 3, Setting Up Your First Environment with OpenAI Gym, covers using OpenAI Gym to set up environments and begin building our first randomly-acting RL agent. We will set up a sample environment and become familiar with the basic tools and functionality of Gym.
Chapter 4, Teaching a Smartcab to Drive Using Q-Learning, covers building your first model-free Q-learning agent in OpenAI Gym, a simulation of a self-driving vehicle dropping a passenger off at a destination. You will build and test your agent's decision-making algorithm and observe the conditions under which its value function converges.
Chapter 5, Building Q-Networks with TensorFlow, covers learning how to use a neural network with a Q-learning algorithm to solve an environment, such as in cases where the state space becomes too large to be modeled with a Q-table. We will learn the challenges of solving a reinforcement learning task in an environment with sparse data.
Chapter 6, Digging Deeper into Deep Q-Networks with Keras and TensorFlow, explains how to build more advanced Q-learning models by combining Q-learning with deep learning and giving the agent an existing model of a problem to work from. This section contains an implementation for a solution to the CartPole problem from OpenAI Gym.
Chapter 7, Decoupling Exploration and Exploitation in Multi-Armed Bandits, covers the problem of multi-armed bandits and extends the concept of exploration versus exploitation. We will explore well-known examples of these problems and talk about why bandit problems are scientifically interesting.
Chapter 8, Further Q-Learning Research and Future Projects, covers a wide range of problems to consider for future projects in Q-learning. You will leave the chapter with a wealth of knowledge on how to continue your research as an RL practitioner.