Book Image

Deep Learning with PyTorch Lightning

By : Kunal Sawarkar
3.5 (2)
Book Image

Deep Learning with PyTorch Lightning

3.5 (2)
By: Kunal Sawarkar

Overview of this book

Building and implementing deep learning (DL) is becoming a key skill for those who want to be at the forefront of progress.But with so much information and complex study materials out there, getting started with DL can feel quite overwhelming. Written by an AI thought leader, Deep Learning with PyTorch Lightning helps researchers build their first DL models quickly and easily without getting stuck on the complexities. With its help, you’ll be able to maximize productivity for DL projects while ensuring full flexibility – from model formulation to implementation. Throughout this book, you’ll learn how to configure PyTorch Lightning on a cloud platform, understand the architectural components, and explore how they are configured to build various industry solutions. You’ll build a neural network architecture, deploy an application from scratch, and see how you can expand it based on your specific needs, beyond what the framework can provide. In the later chapters, you’ll also learn how to implement capabilities to build and train various models like Convolutional Neural Nets (CNN), Natural Language Processing (NLP), Time Series, Self-Supervised Learning, Semi-Supervised Learning, Generative Adversarial Network (GAN) using PyTorch Lightning. By the end of this book, you’ll be able to build and deploy DL models with confidence.

Related Content you might be interested in

Current Title:

Small book image
Deep Learning with PyTorch Lightning
Kunal Sawarkar
Apr, 2022 | 366 pages
Small book image
Spring 5.0 Projects
Nilang Patel
Feb, 2019 | 442 pages
Table of Contents (15 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the colour images
Conventions used
Get in touch
Share Your Thoughts
1
Section 1: Kickstarting with PyTorch Lightning
Section 1: Kickstarting with PyTorch Lightning
Free Chapter
2
Chapter 1: PyTorch Lightning Adventure
Chapter 1: PyTorch Lightning Adventure
What makes PyTorch Lightning so special?
<pip install> – My Lightning adventure
Understanding the key components of PyTorch Lightning
Crafting AI applications using PyTorch Lightning
Further reading
Summary
3
Chapter 2: Getting off the Ground with the First Deep Learning Model
Chapter 2: Getting off the Ground with the First Deep Learning Model
Technical requirements
Getting started with Neural Networks
Building a Hello World MLP model
Building our first Deep Learning model
Building a CNN model for image recognition
Summary
4
Chapter 3: Transfer Learning Using Pre-Trained Models
Chapter 3: Transfer Learning Using Pre-Trained Models
Technical requirements
Getting started with transfer learning
An image classifier using a pre-trained ResNet-50 architecture
Text classification using BERT transformers
Summary
5
Chapter 4: Ready-to-Cook Models from Lightning Flash
Chapter 4: Ready-to-Cook Models from Lightning Flash
Technical requirements
Getting started with Lightning Flash
Flash is as simple as 1-2-3
Video classification using Flash
Automatic speech recognition using Flash
Further learning
Summary
6
Section 2: Solving using PyTorch Lightning
Section 2: Solving using PyTorch Lightning
7
Chapter 5: Time Series Models
Chapter 5: Time Series Models
Technical requirements
Introduction to time series
Getting started with time series models
Traffic volume forecasting using the LSTM time series model
Summary
8
Chapter 6: Deep Generative Models
Chapter 6: Deep Generative Models
Technical requirements
Getting started with GAN models
Creating new food items using a GAN
Creating new butterfly species using a GAN
GAN training challenges
Creating images using DCGAN
Summary
9
Chapter 7: Semi-Supervised Learning
Chapter 7: Semi-Supervised Learning
Technical requirements
Getting started with semi-supervised learning
Going through the CNN–RNN architecture
Generating captions for images
Summary
10
Chapter 8: Self-Supervised Learning
Chapter 8: Self-Supervised Learning
Technical requirements
Getting started with Self-Supervised Learning
What is Contrastive Learning?
SimCLR architecture
SimCLR model for image recognition
Summary
11
Section 3: Advanced Topics
Section 3: Advanced Topics
12
Chapter 9: Deploying and Scoring Models
Chapter 9: Deploying and Scoring Models
Technical requirements
Deploying and scoring a Deep Learning model natively
Deploying and scoring inter-portable models
Next steps
Further reading
Summary
13
Chapter 10: Scaling and Managing Training
Chapter 10: Scaling and Managing Training
Technical Requirements
Managing training
Scaling up training
Controlling training
Further reading
Summary
Why subscribe?
14
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

GAN training challenges

A GAN model requires a lot of compute resources for training a model in order to get a good result, especially when a dataset is not very clean and representations in an image are not very easy to learn. In order to get a very clean output with sharp representations in our fake generated image, we need to pass a higher resolution image as input to our GAN model. However, the higher resolution means a lot more parameters are needed in the model, which in turn requires much more memory to train the model.

Here is an example scenario. We have trained our models using the image size of 64 pixels, but if we increase the image size to 128 pixels, then the number of parameters in the GAN model increases drastically from 15.9 M to 93.4 M. This, in turn, requires much more compute power to train the model, and with the limited resources in the Google Collab environment, you might get an error similar to this after 20–25 epochs:

RuntimeError: CUDA out of...
Previous Section
End of Section 6
Next Section