Book Image

Hands-On Reinforcement Learning with Python

By : Sudharsan Ravichandiran
Book Image

Hands-On Reinforcement Learning with Python

By: Sudharsan Ravichandiran

Overview of this book

Reinforcement Learning (RL) is the trending and most promising branch of artificial intelligence. Hands-On Reinforcement learning with Python will help you master not only the basic reinforcement learning algorithms but also the advanced deep reinforcement learning algorithms. The book starts with an introduction to Reinforcement Learning followed by OpenAI Gym, and TensorFlow. You will then explore various RL algorithms and concepts, such as Markov Decision Process, Monte Carlo methods, and dynamic programming, including value and policy iteration. This example-rich guide will introduce you to deep reinforcement learning algorithms, such as Dueling DQN, DRQN, A3C, PPO, and TRPO. You will also learn about imagination-augmented agents, learning from human preference, DQfD, HER, and many more of the recent advancements in reinforcement learning. By the end of the book, you will have all the knowledge and experience needed to implement reinforcement learning and deep reinforcement learning in your projects, and you will be all set to enter the world of artificial intelligence.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Reinforcement Learning with Python
Sudharsan Ravichandiran
Jun, 2018 | 318 pages
Small book image
PyTorch 1.x Reinforcement Learning Cookbook
Yuxi Hayden Liu
Oct, 2019 | 340 pages
Small book image
Reinforcement Learning with TensorFlow
Sayon Dutta
Apr, 2018 | 334 pages
Small book image
Hands-On Deep Learning Algorithms with Python
Sudharsan Ravichandiran
Jul, 2019 | 512 pages
Table of Contents (16 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
What is RL?
RL algorithm
How RL differs from other ML paradigms
Elements of RL
Agent environment interface
Types of RL environment
RL platforms
Applications of RL
Summary
Questions
Further reading
2
Getting Started with OpenAI and TensorFlow
Getting Started with OpenAI and TensorFlow
Setting up your machine
OpenAI Gym
OpenAI Universe
TensorFlow
Summary
Questions
Further reading
3
The Markov Decision Process and Dynamic Programming
The Markov Decision Process and Dynamic Programming
The Markov chain and Markov process
Markov Decision Process
The Bellman equation and optimality
Solving the Bellman equation
Solving the frozen lake problem
Summary
Questions
Further reading
4
Gaming with Monte Carlo Methods
Gaming with Monte Carlo Methods
Monte Carlo methods
Monte Carlo prediction
Monte Carlo control
Summary
Questions
Further reading
5
Temporal Difference Learning
Temporal Difference Learning
TD learning
TD prediction
TD control
The difference between Q learning and SARSA
Summary
Questions
Further reading
6
Multi-Armed Bandit Problem
Multi-Armed Bandit Problem
The MAB problem
Applications of MAB
Identifying the right advertisement banner using MAB
Contextual bandits
Summary
Questions
Further reading
7
Deep Learning Fundamentals
Deep Learning Fundamentals
Artificial neurons
ANNs
Deep diving into ANN
Neural networks in TensorFlow
RNN
Long Short-Term Memory RNN
Convolutional neural networks
Classifying fashion products using CNN
Summary
Questions
Further reading
8
Atari Games with Deep Q Network
Atari Games with Deep Q Network
What is a Deep Q Network?
Architecture of DQN
Building an agent to play Atari games
Double DQN
Prioritized experience replay
Dueling network architecture
Summary
Questions
Further reading
9
Playing Doom with a Deep Recurrent Q Network
Playing Doom with a Deep Recurrent Q Network
DRQN
Training an agent to play Doom
DARQN
Summary
Questions
Further reading
10
The Asynchronous Advantage Actor Critic Network
The Asynchronous Advantage Actor Critic Network
The Asynchronous Advantage Actor Critic
Driving up a mountain with A3C
Summary
Questions
Further reading
11
Policy Gradients and Optimization
Policy Gradients and Optimization
Policy gradient
Deep deterministic policy gradient
Trust Region Policy Optimization
Proximal Policy Optimization
Summary
Questions
Further reading
12
Capstone Project – Car Racing Using DQN
Capstone Project – Car Racing Using DQN
Environment wrapper functions
Dueling network
Replay memory
Training the network
Car racing
Summary
Questions
Further reading
13
Recent Advancements and Next Steps
Recent Advancements and Next Steps
Imagination augmented agents
Learning from human preference
Deep Q learning from demonstrations
Hindsight experience replay
Hierarchical reinforcement learning
Inverse reinforcement learning
Summary
Questions
Further reading
14
Assessments
Assessments
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
15
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Applications of RL

With greater advancements and research, RL has rapidly evolved everyday applications in several fields ranging from playing computer games to automating a car. Some of the RL applications are listed in the following sections.

Education

Many online education platforms are using RL for providing personalized content for each and every student. Some students may learn better from video content, some may learn better by doing projects, and some may learn better from notes. RL is used to tune educational content personalized for each student according to their learning style and that can be changed dynamically according to the behavior of the user.

Medicine and healthcare

RL has endless applications in medicine and health care; some of them include personalized medical treatment, diagnosis based on a medical image, obtaining treatment strategies in clinical decision making, medical image segmentation, and so on.

Manufacturing

In manufacturing, intelligent robots are used to place objects in the right position. If it fails or succeeds in placing the object at the right position, it remembers the object and trains itself to do this with greater accuracy. The use of intelligent agents will reduce labor costs and result in better performance.

Inventory management

RL is extensively used in inventory management, which is a crucial business activity. Some of these activities include supply chain management, demand forecasting, and handling several warehouse operations (such as placing products in warehouses for managing space efficiently). Google researchers in DeepMind have developed RL algorithms for efficiently reducing the energy consumption in their own data center.

Finance

RL is widely used in financial portfolio management, which is the process of constant redistribution of a fund into different financial products and also in predicting and trading in commercial transactions markets. JP Morgan has successfully used RL to provide better trade execution results for large orders.

Natural Language Processing and Computer Vision

With the unified power of deep learning and RL, Deep Reinforcement Learning (DRL) has been greatly evolving in the fields of Natural Language Processing (NLP) and Computer Vision (CV). DRL has been used for text summarization, information extraction, machine translation, and image recognition, providing greater accuracy than current systems.

Previous Section
End of Section 9
Next Section