Book Image

TensorFlow 2 Reinforcement Learning Cookbook

By : Palanisamy P
Book Image

TensorFlow 2 Reinforcement Learning Cookbook

By: Palanisamy P

Overview of this book

With deep reinforcement learning, you can build intelligent agents, products, and services that can go beyond computer vision or perception to perform actions. TensorFlow 2.x is the latest major release of the most popular deep learning framework used to develop and train deep neural networks (DNNs). This book contains easy-to-follow recipes for leveraging TensorFlow 2.x to develop artificial intelligence applications. Starting with an introduction to the fundamentals of deep reinforcement learning and TensorFlow 2.x, the book covers OpenAI Gym, model-based RL, model-free RL, and how to develop basic agents. You'll discover how to implement advanced deep reinforcement learning algorithms such as actor-critic, deep deterministic policy gradients, deep-Q networks, proximal policy optimization, and deep recurrent Q-networks for training your RL agents. As you advance, you’ll explore the applications of reinforcement learning by building cryptocurrency trading agents, stock/share trading agents, and intelligent agents for automating task completion. Finally, you'll find out how to deploy deep reinforcement learning agents to the cloud and build cross-platform apps using TensorFlow 2.x. By the end of this TensorFlow book, you'll have gained a solid understanding of deep reinforcement learning algorithms and their implementations from scratch.

Related Content you might be interested in

Current Title:

Small book image
TensorFlow 2 Reinforcement Learning Cookbook
Palanisamy P
Jan, 2021 | 472 pages
Small book image
Hands-On Intelligent Agents with OpenAI Gym
Palanisamy P
Jul, 2018 | 254 pages
Small book image
TensorFlow Reinforcement Learning Quick Start Guide
Kaushik Balakrishnan
Mar, 2019 | 184 pages
Small book image
Python Reinforcement Learning Projects
Rajalingappaa Shanmugamani | Sean Saito | Wenzhuo Yang
Sep, 2018 | 296 pages
Table of Contents (11 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
1
Chapter 1: Developing Building Blocks for Deep Reinforcement Learning Using Tensorflow 2.x
Chapter 1: Developing Building Blocks for Deep Reinforcement Learning Using Tensorflow 2.x
Technical requirements
Building an environment and reward mechanism for training RL agents
Implementing neural network-based RL policies for discrete action spaces and decision-making problems
Implementing neural network-based RL policies for continuous action spaces and continuous-control problems
Working with OpenAI Gym for RL training environments
Building a neural agent
Building a neural evolutionary agent
Free Chapter
2
Chapter 2: Implementing Value-Based, Policy-Based, and Actor-Critic Deep RL Algorithms
Chapter 2: Implementing Value-Based, Policy-Based, and Actor-Critic Deep RL Algorithms
Technical requirements
Building stochastic environments for training RL agents
Building value-based reinforcement learning agent algorithms
Implementing temporal difference learning
Building Monte Carlo prediction and control algorithms for RL
Implementing the SARSA algorithm and an RL agent
Building a Q-learning agent
Implementing policy gradients
Implementing actor-critic RL algorithms
3
Chapter 3: Implementing Advanced RL Algorithms
Chapter 3: Implementing Advanced RL Algorithms
Technical requirements
Implementing the Deep Q-Learning algorithm, DQN, and Double-DQN agent
Implementing the Dueling DQN agent
Implementing the Dueling Double DQN algorithm and DDDQN agent
Implementing the Deep Recurrent Q-Learning algorithm and DRQN agent
Implementing the Asynchronous Advantage Actor-Critic algorithm and A3C agent
Implementing the Proximal Policy Optimization algorithm and PPO agent
Implementing the Deep Deterministic Policy Gradient algorithm and DDPG agent
4
Chapter 4: Reinforcement Learning in the Real World – Building Cryptocurrency Trading Agents
Chapter 4: Reinforcement Learning in the Real World – Building Cryptocurrency Trading Agents
Technical requirements
Building a Bitcoin trading RL platform using real market data
Building an Ethereum trading RL platform using price charts
Building an advanced cryptocurrency trading platform for RL agents
Training a cryptocurrency trading bot using RL
5
Chapter 5: Reinforcement Learning in the Real World – Building Stock/Share Trading Agents
Chapter 5: Reinforcement Learning in the Real World – Building Stock/Share Trading Agents
Technical requirements
Building a stock market trading RL platform using real stock exchange data
Building a stock market trading RL platform using price charts
Building an advanced stock trading RL platform to train agents to mimic professional traders
6
Chapter 6: Reinforcement Learning in the Real World – Building Intelligent Agents to Complete Your To-Dos
Chapter 6: Reinforcement Learning in the Real World – Building Intelligent Agents to Complete Your To-Dos
Technical requirements
Building learning environments for real-world RL
Building an RL Agent to complete tasks on the web – Call to Action
Building a visual auto-login bot
Training an RL Agent to automate flight booking for your travel
Training an RL Agent to manage your emails
Training an RL Agent to automate your social media account management
7
Chapter 7: Deploying Deep RL Agents to the Cloud
Chapter 7: Deploying Deep RL Agents to the Cloud
Technical requirements
Implementing the RL agent’s runtime components
Building RL environment simulators as a service
Training RL agents using a remote simulator service
Testing/evaluating RL agents
Packaging RL agents for deployment – a trading bot
Deploying RL agents to the cloud – a trading Bot-as-a-Service
8
Chapter 8: Distributed Training for Accelerated Development of Deep RL Agents
Chapter 8: Distributed Training for Accelerated Development of Deep RL Agents
Technical requirements
Distributed deep learning models using TensorFlow 2.x – Multi-GPU training
Scaling up and out – Multi-machine, multi-GPU training
Training Deep RL agents at scale – Multi-GPU PPO agent
Building blocks for distributed Deep Reinforcement Learning for accelerated training
Large-scale Deep RL agent training using Ray, Tune, and RLLib
9
Chapter 9: Deploying Deep RL Agents on Multiple Platforms
Chapter 9: Deploying Deep RL Agents on Multiple Platforms
Technical requirements
Packaging Deep RL agents for mobile and IoT devices using TensorFlow Lite
Deploying RL agents on mobile devices
Packaging Deep RL agents for the web and Node.js using TensorFlow.js
Deploying a Deep RL agent as a service
Packaging Deep RL agents for cross-platform deployment
10
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Implementing the Dueling Double DQN algorithm and DDDQN agent

Dueling Double DQN (DDDQN) combines the benefits of both Double Q-learning and Dueling architecture. Double Q-learning corrects DQN from overestimating the action values. The Dueling architecture uses a modified architecture to separately learn the state value function (V) and the advantage function (A). This explicit separation allows the algorithm to learn faster, especially when there are many actions to choose from and when the actions are very similar to each other. The dueling architecture enables the agent to learn even when only one action in a state has been taken, as it can update and estimate the state value function, unlike the DQN agent, which cannot learn from actions that were not taken yet. By the end of this recipe, you will have a complete implementation of the DDDQN agent.

Getting ready

To complete this recipe, you will first need to activate the tf2rl-cookbook Conda Python virtual environment and...

Previous Section
End of Section 5
Next Section