Book Image

Hands-On Q-Learning with Python

By : Nazia Habib
Book Image

Hands-On Q-Learning with Python

By: Nazia Habib

Overview of this book

Q-learning is a machine learning algorithm used to solve optimization problems in artificial intelligence (AI). It is one of the most popular fields of study among AI researchers. This book starts off by introducing you to reinforcement learning and Q-learning, in addition to helping you become familiar with OpenAI Gym as well as libraries such as Keras and TensorFlow. A few chapters into the book, you will gain insights into model-free Q-learning and use deep Q-networks and double deep Q-networks to solve complex problems. This book will guide you in exploring use cases such as self-driving vehicles and OpenAI Gym’s CartPole problem. You will also learn how to tune and optimize Q-networks and their hyperparameters. As you progress, you will understand the reinforcement learning approach to solving real-world problems. You will also explore how to use Q-learning and related algorithms in scientific research. Toward the end, you’ll gain insight into what’s in store for reinforcement learning. By the end of this book, you will be equipped with the skills you need to solve reinforcement learning problems using Q-learning algorithms with OpenAI Gym, Keras, and TensorFlow.
Table of Contents (14 chapters)
Free Chapter
1
Section 1: Q-Learning: A Roadmap
6
Section 2: Building and Optimizing Q-Learning Agents
9
Section 3: Advanced Q-Learning Challenges with Keras, TensorFlow, and OpenAI Gym

Questions

  1. Define a control process.
  2. What is the difference between a Markov chain and an MDP?
  3. What does it mean for a system to have the Markov property? Explain this in the context of memorylessness.
  4. Explain why the Taxi-v2 environment has 500 states. Describe the three state variables and enumerate the state space.
  5. Why are some states unreachable and why do we include them in our description of the state space?
  6. Describe a systematic way to choose the optimal hyperparameters for a Q-learning model.
  7. Why do we choose to decay epsilon, and how do we refer to the decision-making phenomenon that results?
  8. What type of environment will an alpha value of 1 be ideal for? What will an alpha value of 0 result in?
  9. What is one good reason to decay gamma? Why might you want a lower value for gamma toward the end of a simulation?
  1. Briefly describe the greedy strategy and give an example...