Book Image

Hands-On Intelligent Agents with OpenAI Gym

By : Palanisamy P
Book Image

Hands-On Intelligent Agents with OpenAI Gym

By: Palanisamy P

Overview of this book

Many real-world problems can be broken down into tasks that require a series of decisions to be made or actions to be taken. The ability to solve such tasks without a machine being programmed requires a machine to be artificially intelligent and capable of learning to adapt. This book is an easy-to-follow guide to implementing learning algorithms for machine software agents in order to solve discrete or continuous sequential decision making and control tasks. Hands-On Intelligent Agents with OpenAI Gym takes you through the process of building intelligent agent algorithms using deep reinforcement learning starting from the implementation of the building blocks for configuring, training, logging, visualizing, testing, and monitoring the agent. You will walk through the process of building intelligent agents from scratch to perform a variety of tasks. In the closing chapters, the book provides an overview of the latest learning environments and learning algorithms, along with pointers to more resources that will help you take your deep reinforcement learning skills to the next level.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Intelligent Agents with OpenAI Gym
Palanisamy P
Jul, 2018 | 254 pages
Small book image
TensorFlow 2 Reinforcement Learning Cookbook
Palanisamy P
Jan, 2021 | 472 pages
Small book image
TensorFlow Reinforcement Learning Quick Start Guide
Kaushik Balakrishnan
Mar, 2019 | 184 pages
Small book image
Reinforcement Learning Algorithms with Python
Andrea Lonza
Oct, 2019 | 366 pages
Table of Contents (12 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Introduction to Intelligent Agents and Learning Environments
Introduction to Intelligent Agents and Learning Environments
What is an intelligent agent?
Learning environments
What is OpenAI Gym?
Understanding the features of OpenAI Gym
What can you do with the OpenAI Gym toolkit?
Creating your first OpenAI Gym environment
Summary
2
Reinforcement Learning and Deep Reinforcement Learning
Reinforcement Learning and Deep Reinforcement Learning
What is reinforcement learning?
Understanding what AI means and what's in it in an intuitive way
Practical reinforcement learning
Markov Decision Process
Planning with dynamic programming
Monte Carlo learning and temporal difference learning
SARSA and Q-learning
Deep reinforcement learning
Practical applications of reinforcement and deep reinforcement learning algorithms
Summary
3
Getting Started with OpenAI Gym and Deep Reinforcement Learning
Getting Started with OpenAI Gym and Deep Reinforcement Learning
Code repository, setup, and configuration
Installing tools and libraries needed for deep reinforcement learning
Summary
4
Exploring the Gym and its Features
Exploring the Gym and its Features
Exploring the list of environments and nomenclature
Understanding the Gym interface
Spaces in the Gym
Summary
5
Implementing your First Learning Agent - Solving the Mountain Car problem
Implementing your First Learning Agent - Solving the Mountain Car problem
Understanding the Mountain Car problem
Implementing a Q-learning agent from scratch
Training the reinforcement learning agent at the Gym
Testing and recording the performance of the agent
A simple and complete Q-Learner implementation for solving the Mountain Car problem
Summary
6
Implementing an Intelligent Agent for Optimal Control using Deep Q-Learning
Implementing an Intelligent Agent for Optimal Control using Deep Q-Learning
Improving the Q-learning agent
Implementing a deep Q-learning agent
The Atari Gym environment
Training the deep Q-learner to play Atari games
Summary
7
Creating Custom OpenAI Gym Environments - CARLA Driving Simulator
Creating Custom OpenAI Gym Environments - CARLA Driving Simulator
Understanding the anatomy of Gym environments
Creating an OpenAI Gym-compatible CARLA driving simulator environment
Summary
8
Implementing an Intelligent - Autonomous Car Driving Agent using Deep Actor-Critic Algorithm
Implementing an Intelligent - Autonomous Car Driving Agent using Deep Actor-Critic Algorithm
The deep n-step advantage actor-critic algorithm
Implementing a deep n-step advantage actor critic agent
Training an intelligent and autonomous driving agent
Summary
9
Exploring the Learning Environment Landscape - Roboschool, Gym-Retro, StarCraft-II, DeepMindLab
Exploring the Learning Environment Landscape - Roboschool, Gym-Retro, StarCraft-II, DeepMindLab
Gym interface-compatible environments
Other open source Python-based learning environments
Summary
10
Exploring the Learning Algorithm Landscape - DDPG (Actor-Critic), PPO (Policy-Gradient), Rainbow (Value-Based)
Exploring the Learning Algorithm Landscape - DDPG (Actor-Critic), PPO (Policy-Gradient), Rainbow (Value-Based)
Deep Deterministic Policy Gradients
Proximal Policy Optimization
Rainbow
Summary
11
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Implementing a deep n-step advantage actor critic agent

We have prepared ourselves with all the background information required to implement the deep n-step advantage actor-critic (A2C) agent. Let's look at an overview of the agent implementation process and then jump right into the hands-on implementation.

The following is the high-level flow of our A2C agent:

  1. Initialize the actor's and critic's networks.
  2. Use the current policy of the actor to gather n-step experiences from the environment and calculate the n-step return.
  1. Calculate the actor's and critic's losses.
  2. Perform the stochastic gradent descent optimization step to update the actor and critic parameters.
  3. Repeat from step 2.

We will implement the agent in a Python class named DeepActorCriticAgent. You will find the full implementation in this book's code repository under 8th chapter: ch8...

Previous Section
End of Section 3
Next Section