This book offers a practical guide for those eager to learn about reinforcement learning. We will take a hands-on approach toward learning about reinforcement learning by going through numerous examples of algorithms and their applications. Each chapter focuses on a particular use case and introduces reinforcement learning algorithms that are used to solve the given problem. Some of these use cases rely on state-of-the-art algorithms; hence through this book, we will learn about and implement some of the best-performing algorithms and techniques in the industry.
The projects increase in difficulty/complexity as you go through the book. The following table describes what you will learn from each chapter:
Chapter name | The use case/problem | Concepts/algorithms/technologies discussed and used |
Balancing Cart Pole | Control horizontal movement of a cart to balance a vertical bar | OpenAI Gym framework, Q-Learning |
Playing Atari Games | Play various Atari games at human-level proficiency | Deep Q-Networks |
Simulating Control Tasks | Control agents in a continuous action space as opposed to a discrete one | Deterministic policy gradients (DPG), Trust Region Policy Optimization (TRPO), multi-tasking |
Building Virtual Worlds in Minecraft | Navigate a character in the virtual world of Minecraft | Asynchronous Advantage Actor-Critic (A3C) |
Learning to Play Go | Go, one of the oldest and most complex board games in the world | Monte Carlo tree search, policy and value networks |
Creating a Chatbot | Generating natural language in a conversational setting | Policy gradient methods, Long Short-Term Memory (LSTM) |
Auto Generating a Deep Learning Image Classifier | Create an agent that generates neural networks to solve a given task | Recurrent neural networks, policy gradient methods (REINFORCE) |
Predicting Future Stock Prices | Predict stock prices and make buy and sell decisions | Actor-Critic methods, time-series analysis, experience replay |
This book is best suited for the reader who:
- Has intermediate proficiency in Python
- Possesses a basic understanding of machine learning and deep learning, especially for the following topics:
- Neural networks
- Backpropagation
- Convolution
- Techniques for better generalization and reduced overfitting
- Enjoys a hands-on, practical approach toward learning
Since this book serves as a practical introduction to the field, we try to keep theoretical content to a minimum. However, it is advisable for the reader to have basic knowledge of some of the fundamental mathematical and statistical concepts on which the field of machine learning depends. These include the following:
- Calculus (single and multivariate)
- Linear algebra
- Probability theory
- Graph theory
Having some experience with these subjects would greatly assist the reader in understanding the concepts and algorithms we will cover throughout this book.
The ensuing chapters will require you to implement various reinforcement learning algorithms. Hence a proper development environment is necessary for a smooth learning journey. In particular, you should have the following:
- A computer running either macOS or the Linux operating system (for those on Windows, try setting up a Virtual Machine with a Linux image)
- A stable internet connection
- A GPU (preferably)
We will exclusively use the Python programming language to implement our reinforcement learning and deep learning algorithms. Moreover, we will be using Python 3.6. A list of libraries we will be using can be found on the official GitHub repository, located at (https://github.com/PacktPublishing/Python-Reinforcement-Learning-Projects). You will also find the implementations of every algorithm we will cover in this book.
Assuming you have a working Python installation, you can install all the required packages using the requirements.txt file found in our repository. We also recommend you create a virtualenv to isolate your development environment from your main OS system. The following steps will help you construct an environment and install the packages:
# Install virtualenv using pip $ pip install virtualenv # Create a virtualenv $ virtualenv rl_projects # Activate virtualenv $ source rl_projects/bin/activate # cd into the directory with our requirements.txt (rl_projects) $ cd /path/to/requirements.txt # pip install the required packages (rl_projects) $ pip install -r requirements.txt
And now you are all set and ready to start! The next few sections of this chapter will introduce the field of reinforcement learning and will also provide a refresher on deep learning.