Book Image

Modern Computer Vision with PyTorch

By : V Kishore Ayyadevara, Yeshwanth Reddy
Book Image

Modern Computer Vision with PyTorch

By: V Kishore Ayyadevara, Yeshwanth Reddy

Overview of this book

Deep learning is the driving force behind many recent advances in various computer vision (CV) applications. This book takes a hands-on approach to help you to solve over 50 CV problems using PyTorch1.x on real-world datasets. You’ll start by building a neural network (NN) from scratch using NumPy and PyTorch and discover best practices for tweaking its hyperparameters. You’ll then perform image classification using convolutional neural networks and transfer learning and understand how they work. As you progress, you’ll implement multiple use cases of 2D and 3D multi-object detection, segmentation, human-pose-estimation by learning about the R-CNN family, SSD, YOLO, U-Net architectures, and the Detectron2 platform. The book will also guide you in performing facial expression swapping, generating new faces, and manipulating facial expressions as you explore autoencoders and modern generative adversarial networks. You’ll learn how to combine CV with NLP techniques, such as LSTM and transformer, and RL techniques, such as Deep Q-learning, to implement OCR, image captioning, object detection, and a self-driving car agent. Finally, you'll move your NN model to production on the AWS Cloud. By the end of this book, you’ll be able to leverage modern NN architectures to solve over 50 real-world CV problems confidently.

Related Content you might be interested in

Current Title:

Small book image
Modern Computer Vision with PyTorch
V Kishore Ayyadevara | Yeshwanth Reddy
Nov, 2020 | 824 pages
Small book image
Neural Networks with Keras Cookbook
V Kishore Ayyadevara
Feb, 2019 | 568 pages
Small book image
PyTorch Artificial Intelligence Fundamentals
Jibin Mathew
Feb, 2020 | 200 pages
Small book image
PyTorch Computer Vision Cookbook
Michael Avendi
Mar, 2020 | 364 pages
Table of Contents (25 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
1
Section 1 - Fundamentals of Deep Learning for Computer Vision
Section 1 - Fundamentals of Deep Learning for Computer Vision
Free Chapter
2
Artificial Neural Network Fundamentals
Artificial Neural Network Fundamentals
Comparing AI and traditional machine learning
Learning about the artificial neural network building blocks
Implementing feedforward propagation
Implementing backpropagation
Putting feedforward propagation and backpropagation together
Understanding the impact of the learning rate
Summarizing the training process of a neural network
Summary
Questions
3
PyTorch Fundamentals
PyTorch Fundamentals
Installing PyTorch
PyTorch tensors
Building a neural network using PyTorch
Using a sequential method to build a neural network
Saving and loading a PyTorch model
Summary
Questions
4
Building a Deep Neural Network with PyTorch
Building a Deep Neural Network with PyTorch
Representing an image
Why leverage neural networks for image analysis?
Preparing our data for image classification
Training a neural network
Scaling a dataset to improve model accuracy
Understanding the impact of varying the batch size
Understanding the impact of varying the loss optimizer
Understanding the impact of varying the learning rate
Understanding the impact of learning rate annealing
Building a deeper neural network
Understanding the impact of batch normalization
The concept of overfitting
Summary
Questions
5
Section 2 - Object Classification and Detection
Section 2 - Object Classification and Detection
6
Introducing Convolutional Neural Networks
Introducing Convolutional Neural Networks
The problem with traditional deep neural networks
Building blocks of a CNN
Implementing a CNN
Classifying images using deep CNNs
Implementing data augmentation
Visualizing the outcome of feature learning
Building a CNN for classifying real-world images
Summary
Questions
7
Transfer Learning for Image Classification
Transfer Learning for Image Classification
Introducing transfer learning
Understanding VGG16 architecture
Understanding ResNet architecture
Implementing facial key point detection
Multi-task learning – Implementing age estimation and gender classification
Introducing the torch_snippets library
Summary
Questions
8
Practical Aspects of Image Classification
Practical Aspects of Image Classification
Generating CAMs
Understanding the impact of data augmentation and batch normalization
Practical aspects to take care of during model implementation
Summary
Questions
9
Basics of Object Detection
Basics of Object Detection
Introducing object detection
Creating a bounding box ground truth for training
Understanding region proposals
Understanding IoU
Non-max suppression
Mean average precision
Training R-CNN-based custom object detectors
Training Fast R-CNN-based custom object detectors
Summary
Questions
10
Advanced Object Detection
Advanced Object Detection
Components of modern object detection algorithms
Training Faster R-CNN on a custom dataset
Working details of YOLO
Training YOLO on a custom dataset
Working details of SSD
Training SSD on a custom dataset
Summary
Test your understanding
11
Image Segmentation
Image Segmentation
Exploring the U-Net architecture
Implementing semantic segmentation using U-Net
Exploring the Mask R-CNN architecture
Implementing instance segmentation using Mask R-CNN
Summary
Questions
12
Applications of Object Detection and Segmentation
Applications of Object Detection and Segmentation
Multi-object instance segmentation
Human pose detection
Crowd counting
Image colorization
3D object detection with point clouds
Summary
13
Section 3 - Image Manipulation
Section 3 - Image Manipulation
14
Autoencoders and Image Manipulation
Autoencoders and Image Manipulation
Understanding autoencoders
Understanding convolutional autoencoders
Understanding variational autoencoders
Performing an adversarial attack on images
Performing neural style transfer
Generating deep fakes
Summary
Questions
15
Image Generation Using GANs
Image Generation Using GANs
Introducing GANs
Using GANs to generate handwritten digits
Using DCGANs to generate face images
Implementing conditional GANs
Summary
Questions
16
Advanced GANs to Manipulate Images
Advanced GANs to Manipulate Images
Leveraging the Pix2Pix GAN
Leveraging CycleGAN
Leveraging StyleGAN on custom images
Super-resolution GAN
Summary
Questions
17
Section 4 - Combining Computer Vision with Other Techniques
Section 4 - Combining Computer Vision with Other Techniques
18
Training with Minimal Data Points
Training with Minimal Data Points
Implementing zero-shot learning
Implementing few-shot learning
Summary
Questions
19
Combining Computer Vision and NLP Techniques
Combining Computer Vision and NLP Techniques
Introducing RNNs
Introducing LSTM architecture
Implementing image captioning
Transcribing handwritten images
Object detection using DETR
Summary
Questions
20
Combining Computer Vision and Reinforcement Learning
Combining Computer Vision and Reinforcement Learning
Learning the basics of reinforcement learning
Implementing Q-learning
Implementing deep Q-learning
Implementing deep Q-learning with the fixed targets model
Implementing an agent to perform autonomous driving
Summary
Questions
21
Moving a Model to Production
Moving a Model to Production
Understanding the basics of an API
Creating an API and making predictions on a local server
Moving the API to the cloud
Summary
22
Using OpenCV Utilities for Image Analysis
Using OpenCV Utilities for Image Analysis
Drawing bounding boxes around words in an image
Detecting lanes in an image of a road
Detecting objects based on color
Building a panoramic view of images
Detecting the number plate of a car
Summary
23
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think
Appendix
Chapter 1 - Artificial Neural Network Fundamentals
Chapter 2 - PyTorch Fundamentals
Chapter 3 - Building a Deep Neural Network with PyTorch
Chapter 4 - Introducing Convolutional Neural Networks
Chapter 5 - Transfer Learning for Image Classification
Chapter 6 - Practical Aspects of Image Classification
Chapter 7 - Basics of Object Detection
Chapter 8 - Advanced Object Detection
Chapter 9 - Image Segmentation
Chapter 11 - Autoencoders and Image Manipulation
Chapter 12 - Image Generation Using GANs
Chapter 13 - Advanced GANs to Manipulate Images
Chapter 14 - Training with Minimal Data Points
Chapter 15 - Combining Computer Vision and NLP Techniques
Chapter 16 - Combining Computer Vision and Reinforcement Learning

Training Fast R-CNN-based custom object detectors

One of the major drawbacks of R-CNN is that it takes considerable time to generate predictions, as generating region proposals for each image, resizing the crops of regions, and extracting features corresponding to each crop (region proposal), constitute the bottleneck.

Fast R-CNN gets around this problem by passing the entire image through the pretrained model to extract features and then fetching the region of features that correspond to the region proposals (which are obtained from selectivesearch) of the original image. In the following sections, we will learn about the working details of Fast R-CNN before training it on our custom dataset.

Working details of Fast R-CNN

Let's understand Fast R-CNN through the following diagram:

Let's understand the preceding diagram through the following steps:

  1. Pass the image through a pretrained model to extract features prior to the flattening layer; let's call the output as feature...
Previous Section
End of Section 9
Next Section