Book Image

Reinforcement Learning with TensorFlow

By : Sayon Dutta
Book Image

Reinforcement Learning with TensorFlow

By: Sayon Dutta

Overview of this book

Reinforcement learning (RL) allows you to develop smart, quick and self-learning systems in your business surroundings. It's an effective method for training learning agents and solving a variety of problems in Artificial Intelligence - from games, self-driving cars and robots, to enterprise applications such as data center energy saving (cooling data centers) and smart warehousing solutions. The book covers major advancements and successes achieved in deep reinforcement learning by synergizing deep neural network architectures with reinforcement learning. You'll also be introduced to the concept of reinforcement learning, its advantages and the reasons why it's gaining so much popularity. You'll explore MDPs, Monte Carlo tree searches, dynamic programming such as policy and value iteration, and temporal difference learning such as Q-learning and SARSA. You will use TensorFlow and OpenAI Gym to build simple neural network models that learn from their own actions. You will also see how reinforcement learning algorithms play a role in games, image processing and NLP. By the end of this book, you will have gained a firm understanding of what reinforcement learning is and understand how to put your knowledge to practical use by leveraging the power of TensorFlow and OpenAI Gym.

Related Content you might be interested in

Current Title:

Small book image
Reinforcement Learning with TensorFlow
Sayon Dutta
Apr, 2018 | 334 pages
Small book image
Hands-On Reinforcement Learning with Python
Sudharsan Ravichandiran
Jun, 2018 | 318 pages
Small book image
Python Reinforcement Learning Projects
Rajalingappaa Shanmugamani | Sean Saito | Wenzhuo Yang
Sep, 2018 | 296 pages
Small book image
Hands-On Intelligent Agents with OpenAI Gym
Palanisamy P
Jul, 2018 | 254 pages
Table of Contents (21 chapters)
Title Page
Title Page
Packt Upsell
Packt Upsell
Contributors
Contributors
Preface
Preface
Free Chapter
1
Deep Learning – Architectures and Frameworks
Deep Learning – Architectures and Frameworks
Deep learning
Reinforcement learning
Introduction to TensorFlow and OpenAI Gym
The pioneers and breakthroughs in reinforcement learning
Summary
2
Training Reinforcement Learning Agents Using OpenAI Gym
Training Reinforcement Learning Agents Using OpenAI Gym
The OpenAI Gym
Programming an agent using an OpenAI Gym environment
Summary
3
Markov Decision Process
Markov Decision Process
Markov decision processes
Partially observable Markov decision processes
Training the FrozenLake-v0 environment using MDP
Summary
4
Policy Gradients
Policy Gradients
The policy optimization method
Why policy optimization methods?
Policy objective functions
Temporal difference rule
Policy gradients
Agent learning pong using policy gradients
Summary
5
Q-Learning and Deep Q-Networks
Q-Learning and Deep Q-Networks
Why reinforcement learning?
Model based learning and model free learning
Q-learning
Deep Q-networks
The Monte Carlo tree search algorithm
The SARSA algorithm
Summary
6
Asynchronous Methods
Asynchronous Methods
Why asynchronous methods?
Asynchronous one-step Q-learning
Asynchronous one-step SARSA
Asynchronous n-step Q-learning
Asynchronous advantage actor critic
A3C for Pong-v0 in OpenAI gym
Summary
7
Robo Everything – Real Strategy Gaming
Robo Everything – Real Strategy Gaming
Real-time strategy games
Reinforcement learning and other approaches
Reinforcement learning in RTS gaming
Summary
8
AlphaGo – Reinforcement Learning at Its Best
AlphaGo – Reinforcement Learning at Its Best
What is Go?
AlphaGo – mastering Go
AlphaGo Zero
Summary
9
Reinforcement Learning in Autonomous Driving
Reinforcement Learning in Autonomous Driving
Machine learning for autonomous driving
Reinforcement learning for autonomous driving
Proposed frameworks for autonomous driving
DeepTraffic – MIT simulator for autonomous driving 
Summary
10
Financial Portfolio Management
Financial Portfolio Management
Introduction
Problem definition
Data preparation
Reinforcement learning
Further improvements
Summary
11
Reinforcement Learning in Robotics
Reinforcement Learning in Robotics
Reinforcement learning in robotics
Challenges in robot reinforcement learning
Open questions and practical challenges
Key takeaways
Summary
12
Deep Reinforcement Learning in Ad Tech
Deep Reinforcement Learning in Ad Tech
Computational advertising challenges and bidding strategies
Real-time bidding by reinforcement learning in display advertising
Summary
13
Reinforcement Learning in Image Processing
Reinforcement Learning in Image Processing
Hierarchical object detection with deep reinforcement learning
Summary
14
Deep Reinforcement Learning in NLP
Deep Reinforcement Learning in NLP
Text summarization
Text question answering
Summary
Further topics in Reinforcement Learning
Further topics in Reinforcement Learning
Continuous action space algorithms
Scoring mechanism in sequential models in NLP
Summary
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index
Index

The pioneers and breakthroughs in reinforcement learning

Before going on floor with all the coding, let's shed some light on some of the pioneers, industrial leaders, and research breakthroughs in the field of deep reinforcement learning.

David Silver

Dr. David Silver, with an h-index of 30, heads the research team of reinforcement learning at Google DeepMind and is the lead researcher on AlphaGo. David co-founded Elixir Studios and then completed his PhD in reinforcement learning from the University of Alberta, where he co-introduced the algorithms used in the first master-level 9x9 Go programs. After this, he became a lecturer at University College London. He used to consult for DeepMind before joining full-time in 2013. David lead the AlphaGo project, which became the first program to defeat a top professional player in the game of Go.

Pieter Abbeel

Pieter Abbeel is a professor at UC Berkeley and was a Research Scientist at OpenAI. Pieter completed his PhD in Computer Science under Andrew Ng. His current research focuses on robotics and machine learning, with a particular focus on deep reinforcement learning, deep imitation learning, deep unsupervised learning, meta-learning, learning-to-learn, and AI safety. Pieter also won the NIPS 2016 Best Paper Award.

Google DeepMind

Google DeepMind is a British artificial intelligence company founded in September 2010 and acquired by Google in 2014. They are an industrial leader in the domains of deep reinforcement learning and a neural turing machine. They made news in 2016 when the AlphaGo program defeated Lee Sedol, 9th dan Go player. Google DeepMind has channelized its focus on two big sectors: energy and healthcare.

Here are some of its projects:

  • In July 2016, Google DeepMind and Moorfields Eye Hospital announced their collaboration to use eye scans to research early signs of diseases leading to blindness
  • In August 2016, Google DeepMind announced its collaboration with University College London Hospital to research and develop an algorithm to automatically differentiate between healthy and cancerous tissues in head and neck areas
  • Google DeepMind AI reduced the Google's data center cooling bill by 40%

The AlphaGo program

As mentioned previously in Google DeepMind, AlphaGo is a computer program that first defeated Lee Sedol and then Ke Jie, who at the time was the world No. 1 in Go. In 2017 an improved version, AlphaGo zero was launched that defeated AlphaGo 100 games to 0.

Libratus

Libratus is an artificial intelligence computer program designed by the team led by Professor Tuomas Sandholm at Carnegie Mellon University to play Poker. Libratus and its predecessor, Claudico, share the same meaning, balanced.

In January 2017, it made history by defeating four of the world's best professional poker players in a marathon 20-day poker competition.

Though Libratus focuses on playing poker, its designers mentioned its ability to learn any game that has incomplete information and where opponents are engaging in deception. As a result, they have proposed that the system can be applied to problems in cybersecurity, business negotiations, or medical planning domains.

Previous Section
End of Section 5
Next Section