Book Image

Hands-On Reinforcement Learning with Python

By : Sudharsan Ravichandiran
Book Image

Hands-On Reinforcement Learning with Python

By: Sudharsan Ravichandiran

Overview of this book

Reinforcement Learning (RL) is the trending and most promising branch of artificial intelligence. Hands-On Reinforcement learning with Python will help you master not only the basic reinforcement learning algorithms but also the advanced deep reinforcement learning algorithms. The book starts with an introduction to Reinforcement Learning followed by OpenAI Gym, and TensorFlow. You will then explore various RL algorithms and concepts, such as Markov Decision Process, Monte Carlo methods, and dynamic programming, including value and policy iteration. This example-rich guide will introduce you to deep reinforcement learning algorithms, such as Dueling DQN, DRQN, A3C, PPO, and TRPO. You will also learn about imagination-augmented agents, learning from human preference, DQfD, HER, and many more of the recent advancements in reinforcement learning. By the end of the book, you will have all the knowledge and experience needed to implement reinforcement learning and deep reinforcement learning in your projects, and you will be all set to enter the world of artificial intelligence.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Reinforcement Learning with Python
Sudharsan Ravichandiran
Jun, 2018 | 318 pages
Small book image
PyTorch 1.x Reinforcement Learning Cookbook
Yuxi Hayden Liu
Oct, 2019 | 340 pages
Small book image
Reinforcement Learning with TensorFlow
Sayon Dutta
Apr, 2018 | 334 pages
Small book image
Hands-On Deep Learning Algorithms with Python
Sudharsan Ravichandiran
Jul, 2019 | 512 pages
Table of Contents (16 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
What is RL?
RL algorithm
How RL differs from other ML paradigms
Elements of RL
Agent environment interface
Types of RL environment
RL platforms
Applications of RL
Summary
Questions
Further reading
2
Getting Started with OpenAI and TensorFlow
Getting Started with OpenAI and TensorFlow
Setting up your machine
OpenAI Gym
OpenAI Universe
TensorFlow
Summary
Questions
Further reading
3
The Markov Decision Process and Dynamic Programming
The Markov Decision Process and Dynamic Programming
The Markov chain and Markov process
Markov Decision Process
The Bellman equation and optimality
Solving the Bellman equation
Solving the frozen lake problem
Summary
Questions
Further reading
4
Gaming with Monte Carlo Methods
Gaming with Monte Carlo Methods
Monte Carlo methods
Monte Carlo prediction
Monte Carlo control
Summary
Questions
Further reading
5
Temporal Difference Learning
Temporal Difference Learning
TD learning
TD prediction
TD control
The difference between Q learning and SARSA
Summary
Questions
Further reading
6
Multi-Armed Bandit Problem
Multi-Armed Bandit Problem
The MAB problem
Applications of MAB
Identifying the right advertisement banner using MAB
Contextual bandits
Summary
Questions
Further reading
7
Deep Learning Fundamentals
Deep Learning Fundamentals
Artificial neurons
ANNs
Deep diving into ANN
Neural networks in TensorFlow
RNN
Long Short-Term Memory RNN
Convolutional neural networks
Classifying fashion products using CNN
Summary
Questions
Further reading
8
Atari Games with Deep Q Network
Atari Games with Deep Q Network
What is a Deep Q Network?
Architecture of DQN
Building an agent to play Atari games
Double DQN
Prioritized experience replay
Dueling network architecture
Summary
Questions
Further reading
9
Playing Doom with a Deep Recurrent Q Network
Playing Doom with a Deep Recurrent Q Network
DRQN
Training an agent to play Doom
DARQN
Summary
Questions
Further reading
10
The Asynchronous Advantage Actor Critic Network
The Asynchronous Advantage Actor Critic Network
The Asynchronous Advantage Actor Critic
Driving up a mountain with A3C
Summary
Questions
Further reading
11
Policy Gradients and Optimization
Policy Gradients and Optimization
Policy gradient
Deep deterministic policy gradient
Trust Region Policy Optimization
Proximal Policy Optimization
Summary
Questions
Further reading
12
Capstone Project – Car Racing Using DQN
Capstone Project – Car Racing Using DQN
Environment wrapper functions
Dueling network
Replay memory
Training the network
Car racing
Summary
Questions
Further reading
13
Recent Advancements and Next Steps
Recent Advancements and Next Steps
Imagination augmented agents
Learning from human preference
Deep Q learning from demonstrations
Hindsight experience replay
Hierarchical reinforcement learning
Inverse reinforcement learning
Summary
Questions
Further reading
14
Assessments
Assessments
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
15
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

To get the most out of this book

You need the following software for this book:

  • Anaconda
  • Python
  • Any web browser
  • Docker

Download the example code files

You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

  1. Log in or register at www.packtpub.com.
  2. Select the SUPPORT tab.
  3. Click on Code Downloads & Errata.
  4. Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

  • WinRAR/7-Zip for Windows
  • Zipeg/iZip/UnRarX for Mac
  • 7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Hands-On-Reinforcement-Learning-with-Python. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Mount the downloaded WebStorm-10*.dmg disk image file as another disk in your system."

A block of code is set as follows:

policy_iteration():
    Initialize random policy
    for i in no_of_iterations: 
        Q_value = value_function(random_policy)
        new_policy = Maximum state action pair from Q value

Any command-line input or output is written as follows:

bash Anaconda3-5.0.1-Linux-x86_64.sh

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this.

Warnings or important notes appear like this.
Tips and tricks appear like this.
Previous Section
Next Section