- DDPG is an off-policy algorithm, as it uses a replay buffer.
- In general, the same number of hidden layers and the number of neurons per hidden layer is used for the actor and the critic, but this is not required. Note that the output layer will be different for the actor and the critic, with the actor having the number of outputs equal to the number of actions; the critic will have only one output.
- DDPG is used for continuous control, that is, when the actions are continuous and real-valued. Atari Breakout has discrete actions, and so DDPG is not suitable for Atari Breakout.
- We use the relu activation function, and so the biases are initialized to small positive values so that they fire at the beginning of the training and allow gradients to back-propagate.
- This is an exercise. See https://gym.openai.com/envs/InvertedDoublePendulum-v2/.
- This is also an exercise. Notice...
TensorFlow Reinforcement Learning Quick Start Guide
By :
TensorFlow Reinforcement Learning Quick Start Guide
By:
Overview of this book
Advances in reinforcement learning algorithms have made it possible to use them for optimal control in several different industrial applications. With this book, you will apply Reinforcement Learning to a range of problems, from computer games to autonomous driving.
The book starts by introducing you to essential Reinforcement Learning concepts such as agents, environments, rewards, and advantage functions. You will also master the distinctions between on-policy and off-policy algorithms, as well as model-free and model-based algorithms. You will also learn about several Reinforcement Learning algorithms, such as SARSA, Deep Q-Networks (DQN), Deep Deterministic Policy Gradients (DDPG), Asynchronous Advantage Actor-Critic (A3C), Trust Region Policy Optimization (TRPO), and Proximal Policy Optimization (PPO). The book will also show you how to code these algorithms in TensorFlow and Python and apply them to solve computer games from OpenAI Gym. Finally, you will also learn how to train a car to drive autonomously in the Torcs racing car simulator.
By the end of the book, you will be able to design, build, train, and evaluate feed-forward neural networks and convolutional neural networks. You will also have mastered coding state-of-the-art algorithms and also training agents for various control problems.
Related Content you might be interested in
Current Title:
TensorFlow Reinforcement Learning Quick Start Guide
Table of Contents (11 chapters)
Preface
Free Chapter
Up and Running with Reinforcement Learning
Temporal Difference, SARSA, and Q-Learning
Deep Q-Network
Double DQN, Dueling Architectures, and Rainbow
Deep Deterministic Policy Gradient
Asynchronous Methods - A3C and A2C
Trust Region Policy Optimization and Proximal Policy Optimization
Deep RL Applied to Autonomous Driving
Other Books You May Enjoy
Customer Reviews