Book Image

ROS Robotics Projects - Second Edition

By : Ramkumar Gandhinathan
Book Image

ROS Robotics Projects - Second Edition

By: Ramkumar Gandhinathan

Overview of this book

Nowadays, heavy industrial robots placed in workcells are being replaced by new age robots called cobots, which don't need workcells. They are used in manufacturing, retail, banks, energy, and healthcare, among other domains. One of the major reasons for this rapid growth in the robotics market is the introduction of an open source robotics framework called the Robot Operating System (ROS). This book covers projects in the latest ROS distribution, ROS Melodic Morenia with Ubuntu Bionic (18.04). Starting with the fundamentals, this updated edition of ROS Robotics Projects introduces you to ROS-2 and helps you understand how it is different from ROS-1. You'll be able to model and build an industrial mobile manipulator in ROS and simulate it in Gazebo 9. You'll then gain insights into handling complex robot applications using state machines and working with multiple robots at a time. This ROS book also introduces you to new and popular hardware such as Nvidia's Jetson Nano, Asus Tinker Board, and Beaglebone Black, and allows you to explore interfacing with ROS. You'll learn as you build interesting ROS projects such as self-driving cars, making use of deep learning, reinforcement learning, and other key AI concepts. By the end of the book, you'll have gained the confidence to build interesting and intricate projects with ROS.
Table of Contents (14 chapters)

Reinforcement learning algorithms

MDP models can be solved in many ways. One of the methods to solve MDP is using Monte Carlo prediction, which helps in predicting value functions and control methods for further optimization of those value functions. This method is only for time-bound tasks or episodic tasks. The problem with this method is that if the environment is large or the episodes are long, the time taken to optimize the value functions takes a while as well. However, we won't be discussing Monte Carlo methods in this section. Instead, we shall look at a more interesting model-free learning technique that is actually a combination of Monte Carlo methods and dynamic programming. This technique is called temporal difference learning. This learning can be applied to non-episodic tasks as well and doesn't need any model information to be known in advance. Let&apos...