The PyTorch Agent Net library

In Chapter 6, Deep Q-Networks, we implemented a DQN from scratch, using only PyTorch, OpenAI Gym, and pytorch-tensorboard. It suited our needs to demonstrate how things work, but now we're going to extend the basic DQN with extra tweaks. Some tweaks are quite simple and trivial, but some will require a major code modification. To be able to focus only on the significant parts, it would be useful to have as small and concise version of a DQN as possible, preferably with reusable code pieces. This will be extremely helpful when you're experimenting with some methods published in papers or your own ideas. In that case, you don't need to reimplement the same functionality again and again, fighting with the inevitable bugs.

With this in mind, some time ago I started to implement my own toolkit for the deep RL domain. I called it PTAN, which stands for PyTorch Agent Net, as it was inspired by another open-source library called AgentNet...