Book Image

Python Reinforcement Learning

By : Sudharsan Ravichandiran, Sean Saito, Rajalingappaa Shanmugamani, Yang Wenzhuo
Book Image

Python Reinforcement Learning

By: Sudharsan Ravichandiran, Sean Saito, Rajalingappaa Shanmugamani, Yang Wenzhuo

Overview of this book

Reinforcement Learning (RL) is the trending and most promising branch of artificial intelligence. This Learning Path will help you master not only the basic reinforcement learning algorithms but also the advanced deep reinforcement learning algorithms. The Learning Path starts with an introduction to RL followed by OpenAI Gym, and TensorFlow. You will then explore various RL algorithms, such as Markov Decision Process, Monte Carlo methods, and dynamic programming, including value and policy iteration. You'll also work on various datasets including image, text, and video. This example-rich guide will introduce you to deep RL algorithms, such as Dueling DQN, DRQN, A3C, PPO, and TRPO. You will gain experience in several domains, including gaming, image processing, and physical simulations. You'll explore TensorFlow and OpenAI Gym to implement algorithms that also predict stock prices, generate natural language, and even build other neural networks. You will also learn about imagination-augmented agents, learning from human preference, DQfD, HER, and many of the recent advancements in RL. By the end of the Learning Path, you will have all the knowledge and experience needed to implement RL and deep RL in your projects, and you enter the world of artificial intelligence to solve various real-life problems. This Learning Path includes content from the following Packt products: • Hands-On Reinforcement Learning with Python by Sudharsan Ravichandiran • Python Reinforcement Learning Projects by Sean Saito, Yang Wenzhuo, and Rajalingappaa Shanmugamani

Related Content you might be interested in

Current Title:

Small book image
Python Reinforcement Learning
Sudharsan Ravichandiran | Sean Saito | Rajalingappaa Shanmugamani | Yang Wenzhuo
Apr, 2019 | 8 hours 16 minutes
No titles found
Table of Contents (27 chapters)
Title Page
Title Page
About Packt
About Packt
Contributors
Contributors
Preface
Preface
Free Chapter
1
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
What is RL?
RL algorithm
How RL differs from other ML paradigms
Elements of RL
Agent environment interface
Types of RL environment
RL platforms
Applications of RL
Summary
Questions
Further reading
2
Getting Started with OpenAI and TensorFlow
Getting Started with OpenAI and TensorFlow
Setting up your machine
OpenAI Gym
OpenAI Universe
TensorFlow
Summary
Questions
Further reading
3
The Markov Decision Process and Dynamic Programming
The Markov Decision Process and Dynamic Programming
The Markov chain and Markov process
Markov Decision Process
The Bellman equation and optimality
Solving the Bellman equation
Solving the frozen lake problem
Summary
Questions
Further reading
4
Gaming with Monte Carlo Methods
Gaming with Monte Carlo Methods
Monte Carlo methods
Monte Carlo prediction
Monte Carlo control
Summary
Questions
Further reading
5
Temporal Difference Learning
Temporal Difference Learning
TD learning
TD prediction
TD control
The difference between Q learning and SARSA
Summary
Questions
Further reading
6
Multi-Armed Bandit Problem
Multi-Armed Bandit Problem
The MAB problem
Applications of MAB
Identifying the right advertisement banner using MAB
Contextual bandits
Summary
Questions
Further reading
7
Playing Atari Games
Playing Atari Games
Introduction to Atari games
Building an Atari emulator
Data preparation
Deep Q-learning
Implementation of DQN
Experiments
Summary
8
Atari Games with Deep Q Network
Atari Games with Deep Q Network
What is a Deep Q Network?
Architecture of DQN
Building an agent to play Atari games
Double DQN
Prioritized experience replay
Dueling network architecture
Summary
Questions
Further reading
9
Playing Doom with a Deep Recurrent Q Network
Playing Doom with a Deep Recurrent Q Network
DRQN
Training an agent to play Doom 
DARQN
Summary
Questions
Further reading
10
The Asynchronous Advantage Actor Critic Network
The Asynchronous Advantage Actor Critic Network
The Asynchronous Advantage Actor Critic
Driving up a mountain with A3C
Summary
Questions
Further reading
11
Policy Gradients and Optimization
Policy Gradients and Optimization
Policy gradient
Deep deterministic policy gradient
Trust Region Policy Optimization
Proximal Policy Optimization
Summary
Questions
Further reading
12
Balancing CartPole
Balancing CartPole
OpenAI Gym
Markov models
Summary
13
Simulating Control Tasks
Simulating Control Tasks
Introduction to control tasks
Deterministic policy gradient
Trust region policy optimization
Summary
14
Building Virtual Worlds in Minecraft
Building Virtual Worlds in Minecraft
Introduction to the Minecraft environment
Data preparation
Asynchronous advantage actor-critic algorithm
Implementation of A3C
Experiments
Summary
15
Learning to Play Go
Learning to Play Go
A brief introduction to Go
Monte Carlo tree search
AlphaGo
AlphaGo Zero
Implementing AlphaGo Zero
Summary
References
16
Creating a Chatbot
Creating a Chatbot
The background problem
Step-by-step guide
Summary
17
Generating a Deep Learning Image Classifier
Generating a Deep Learning Image Classifier
Neural Architecture Search
Implementing NAS
Advantages of NAS
Summary
18
Predicting Future Stock Prices
Predicting Future Stock Prices
Background problem
Data used
Step-by-step guide
Summary
19
Capstone Project - Car Racing Using DQN
Capstone Project - Car Racing Using DQN
Environment wrapper functions
Dueling network
Replay memory
Training the network
Car racing
Summary
Questions
Further reading
20
Looking Ahead
Looking Ahead
The shortcomings of reinforcement learning
Upcoming developments in reinforcement learning
Summary
References
Assessments
Assessments
Chapter 1: Introduction to Reinforcement Learning
Chapter 2: Getting Started with OpenAI and TensorFlow
Chapter 3: The Markov Decision Process and Dynamic Programming
Chapter 4: Gaming with Monte Carlo Methods
Chapter 5: Temporal Difference Learning
Chapter 6: Multi-Armed Bandit Problem
Chapter 8: Atari Games with Deep Q Network
Chapter 9: Playing Doom with a Deep Recurrent Q Network
Chapter 10: The Asynchronous Advantage Actor Critic Network
Chapter 11: Policy Gradients and Optimization
Chapter 19: Capstone Project – Car Racing Using DQN
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index
Index

Building an agent to play Atari games

Now we will see how to build an agent to play any Atari game. You can get the complete code as a Jupyter notebook with the explanation here (https://github.com/sudharsan13296/Hands-On-Reinforcement-Learning-With-Python/blob/master/08.%20Atari%20Games%20with%20DQN/8.8%20Building%20an%20Agent%20to%20Play%20Atari%20Games.ipynb).

First, we import all the necessary libraries:

import numpy as np
import gym
import tensorflow as tf
from tensorflow.contrib.layers import flatten, conv2d, fully_connected
from collections import deque, Counter
import random
from datetime import datetime

We can use any of the Atari gaming environments given here: http://gym.openai.com/envs/#atari.

In this example, we use the Pac-Man game environment:

env = gym.make("MsPacman-v0")
n_outputs = env.action_space.n

The Pac-Man environment is shown here:

Now we define a preprocess_observation function for preprocessing our input game screen. We reduce the image size and convert the image to grayscale...

Previous Section
End of Section 4
Next Section