We will now delve into the DDPG algorithm, which is a state-of-the-art RL algorithm for continuous control. It was originally published by Google DeepMind in 2016 and has gained a lot of interest in the community, with several new variants proposed thereafter. As was the case in DQN, DDPG also uses target networks for stability. It also uses a replay buffer to reuse past data, and therefore, it is an off-policy RL algorithm.
The ddpg.py file is the main file from which we start the training and testing. It will call the training or testing functions, which are present in TrainOrTest.py. The AandC.py file has the TensorFlow code for the actor and the critic networks. Finally, replay_buffer.py stores the samples in a replay buffer by using a deque data structure. We will train the DDPG to learn to hold an inverted pendulum vertically, using OpenAI...