Book Image

Applied Deep Learning and Computer Vision for Self-Driving Cars

By : Sumit Ranjan, Dr. S. Senthamilarasu
Book Image

Applied Deep Learning and Computer Vision for Self-Driving Cars

By: Sumit Ranjan, Dr. S. Senthamilarasu

Overview of this book

Thanks to a number of recent breakthroughs, self-driving car technology is now an emerging subject in the field of artificial intelligence and has shifted data scientists' focus to building autonomous cars that will transform the automotive industry. This book is a comprehensive guide to use deep learning and computer vision techniques to develop autonomous cars. Starting with the basics of self-driving cars (SDCs), this book will take you through the deep neural network techniques required to get up and running with building your autonomous vehicle. Once you are comfortable with the basics, you'll delve into advanced computer vision techniques and learn how to use deep learning methods to perform a variety of computer vision tasks such as finding lane lines, improving image classification, and so on. You will explore the basic structure and working of a semantic segmentation model and get to grips with detecting cars using semantic segmentation. The book also covers advanced applications such as behavior-cloning and vehicle detection using OpenCV, transfer learning, and deep learning methodologies to train SDCs to mimic human driving. By the end of this book, you'll have learned how to implement a variety of neural networks to develop your own autonomous vehicle using modern Python libraries.

Related Content you might be interested in

Current Title:

Small book image
Applied Deep Learning and Computer Vision for Self-Driving Cars
Sumit Ranjan | Dr. S. Senthamilarasu
Aug, 2020 | 332 pages
Small book image
Computer Vision with Python 3
Salil Kapur | Saurabh Kapur
Aug, 2017 | 206 pages
Small book image
Hands-On Vision and Behavior for Self-Driving Cars
Luca Venturi | Krishtof Korda
Oct, 2020 | 374 pages
Small book image
Python Image Processing Cookbook
Sandipan Dey
Apr, 2020 | 438 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
1
Section 1: Deep Learning Foundation and SDC Basics
Section 1: Deep Learning Foundation and SDC Basics
Free Chapter
2
The Foundation of Self-Driving Cars
The Foundation of Self-Driving Cars
Introduction to SDCs
Benefits of SDCs
Advancements in SDCs
Challenges in current deployments
Building safe systems
The cheapest computer and hardware
Software programming
Fast internet
Levels of autonomy
Level 0 – manual cars
Level 1 – driver support
Level 2 – partial automation
Level 3 – conditional automation
Level 4 – high automation
Level 5 – complete automation
Deep learning and computer vision approaches for SDCs
LIDAR and computer vision for SDC vision
Summary
3
Dive Deep into Deep Neural Networks
Dive Deep into Deep Neural Networks
Diving deep into neural networks
Introduction to neurons
Understanding neurons and perceptrons
The workings of ANNs
Understanding activation functions
The threshold function
The sigmoid function
The rectifier linear function
The hyperbolic tangent activation function
The cost function of neural networks
Optimizers
Understanding hyperparameters
Model training-specific hyperparameters
Learning rate
Batch size
Number of epochs
Network architecture-specific hyperparameters
Number of hidden layers
Regularization
L1 and L2 regularization
Dropout
Activation functions as hyperparameters
TensorFlow versus Keras
Summary
4
Implementing a Deep Learning Model Using Keras
Implementing a Deep Learning Model Using Keras
Starting work with Keras
Advantages of Keras
The working principle behind Keras
Building Keras models
The sequential model
The functional model
Types of Keras execution
Keras for deep learning
Building your first deep learning model
Description of the Auto-Mpg dataset
Importing the data
Splitting the data
Standardizing the data
Building and compiling the model
Training the model
Predicting new, unseen data
Evaluating the model's performance
Saving and loading models
Summary
5
Section 2: Deep Learning and Computer Vision Techniques for SDC
Section 2: Deep Learning and Computer Vision Techniques for SDC
6
Computer Vision for Self-Driving Cars
Computer Vision for Self-Driving Cars
Introduction to computer vision
Challenges in computer vision
Artificial eyes versus human eyes
Building blocks of an image
Digital representation of an image
Converting images from RGB to grayscale
Road-marking detection
Detection with the grayscale image
Detection with the RGB image
Challenges in color selection techniques
Color space techniques
Introducing the RGB space
HSV space
Color space manipulation
Introduction to convolution
Sharpening and blurring
Edge detection and gradient calculation
Introducing Sobel
Introducing the Laplacian edge detector
Canny edge detection
Image transformation
Affine transformation
Projective transformation
Image rotation
Image translation
Image resizing
Perspective transformation
Cropping, dilating, and eroding an image
Masking regions of interest
The Hough transform
Summary
7
Finding Road Markings Using OpenCV
Finding Road Markings Using OpenCV
Finding road markings in an image
Loading the image using OpenCV
Converting the image into grayscale
Smoothing the image
Canny edge detection
Masking the region of interest
Applying bitwise_and
Applying the Hough transform
Optimizing the detected road markings
Detecting road markings in a video
Summary
8
Improving the Image Classifier with CNN
Improving the Image Classifier with CNN
Images in computer format
The need for CNNs
The intuition behind CNNs
Introducing CNNs
Why 3D layers?
Understanding the convolution layer
Depth, stride, and padding
Depth
Stride
Zero-padding
ReLU
Fully connected layers
The softmax function
Introduction to handwritten digit recognition
Problem and aim
Loading the data
Reshaping the data
The transformation of data
One-hot encoding the output
Building and compiling our model
Compiling the model
Training the model
Validation versus train loss
Validation versus test accuracy
Saving the model
Visualizing the model architecture
Confusion matrix
The accuracy report
Summary
9
Road Sign Detection Using Deep Learning
Road Sign Detection Using Deep Learning
Dataset overview
Dataset structure
Image format
Loading the data
Image exploration
Data preparation
Model training
Model accuracy
Summary
10
Section 3: Semantic Segmentation for Self-Driving Cars
Section 3: Semantic Segmentation for Self-Driving Cars
11
The Principles and Foundations of Semantic Segmentation
The Principles and Foundations of Semantic Segmentation
Introduction to semantic segmentation
Understanding the semantic segmentation architecture
Overview of different semantic segmentation architectures
U-Net
SegNet
Encoder
Decoder
PSPNet
DeepLabv3+
E-Net
Summary
12
Implementing Semantic Segmentation
Implementing Semantic Segmentation
Semantic segmentation in images
Semantic segmentation in videos
Summary
13
Section 4: Advanced Implementations
Section 4: Advanced Implementations
14
Behavioral Cloning Using Deep Learning
Behavioral Cloning Using Deep Learning
Neural network for regression
Behavior cloning using deep learning
Data collection
Data preparation
Model development
Evaluating the simulator
Summary
15
Vehicle Detection Using OpenCV and Deep Learning
Vehicle Detection Using OpenCV and Deep Learning
What makes YOLO different?
The YOLO loss function
The YOLO architecture
Fast YOLO
YOLO v2
YOLO v3
Implementation of YOLO object detection
Importing the libraries
Processing the image function
The get class function
Draw box function
Detect image function
Detect video function
Importing YOLO
Detecting objects in images
Detecting objects in videos
Summary
16
Next Steps
Next Steps
SDC sensors
Camera
RADAR
Ultrasonic sensors
Odometric sensors
LIDAR
Introduction to sensor fusion
Kalman filter
Summary
17
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Semantic segmentation in images

In this section, we are going to implement one project on semantic segmentation using a popular network called ENet. 

Efficient Neural Network (ENet) is one of the more popular networks out there due to its ability to perform real-time, pixel-wise semantic segmentation. ENet is up to 18x faster, requires 75x fewer FLOPs, and has 79x fewer parameters than other networks. This means ENet provides better accuracy than the existing models, such as U-Net and SegNet. ENet networks are typically tested on CamVid, CityScapes, and SUN datasets. The model's size is 3.2 MB.

The model we are using has been trained on 20 classes: 

  • Road
  • Sidewalk
  • Building
  • Wall
  • Fence
  • Pole
  • TrafficLight
  • TrafficSign
  • Vegetation
  • Terrain
  • Sky
  • Person
  • Rider
  • Car
  • Truck
  • Bus
  • Train
  • Motorcycle
  • Bicycle
  • Unlabeled 

We will start with the semantic segmentation project: 

  1. First, we will import the necessary packages and libraries, such as numpy, openCV, and...
Previous Section
End of Section 2
Next Section