We are using the TensorFlow framework to build a Q-network that will solve the Taxi-v2 task. Note that this is a single-layer network, so it does not qualify as a deep Q-network. We'll be building a deep Q-network implementation in the next chapter.
Many people use the term "deep learning" in association with any machine learning model that uses neural networks, and, in fact, some incorrectly generalize the term "deep Q-network" to include any Q-learning implementation that uses a neural network. The main distinction is that deep learning structures contain many hierarchical neural network layers that are constructed into various architectures.
The primary difference here from the model-free version that we built in Chapter 4, Teaching a Smartcab to Drive Using Q-Learning, is that, instead of updating a Q-table with the exact...