Book Image

Hands-On GPU Programming with Python and CUDA

By : Dr. Brian Tuomanen
Book Image

Hands-On GPU Programming with Python and CUDA

By: Dr. Brian Tuomanen

Overview of this book

Hands-On GPU Programming with Python and CUDA hits the ground running: you’ll start by learning how to apply Amdahl’s Law, use a code profiler to identify bottlenecks in your Python code, and set up an appropriate GPU programming environment. You’ll then see how to “query” the GPU’s features and copy arrays of data to and from the GPU’s own memory. As you make your way through the book, you’ll launch code directly onto the GPU and write full blown GPU kernels and device functions in CUDA C. You’ll get to grips with profiling GPU code effectively and fully test and debug your code using Nsight IDE. Next, you’ll explore some of the more well-known NVIDIA libraries, such as cuFFT and cuBLAS. With a solid background in place, you will now apply your new-found knowledge to develop your very own GPU-based deep neural network from scratch. You’ll then explore advanced topics, such as warp shuffling, dynamic parallelism, and PTX assembly. In the final chapter, you’ll see some topics and applications related to GPU programming that you may wish to pursue, including AI, graphics, and blockchain. By the end of this book, you will be able to apply GPU programming to problems related to data science and high-performance computing.

Related Content you might be interested in

Current Title:

Small book image
Hands-On GPU Programming with Python and CUDA
Dr. Brian Tuomanen
Nov, 2018 | 310 pages
Small book image
CUDA Cookbook
Bharatkumar Sharma | Jack Han
Sep, 2019 | 508 pages
Small book image
Hands-On GPU-Accelerated Computer Vision with OpenCV and CUDA
Bhaumik Vaidya
Sep, 2018 | 380 pages
Small book image
Hands-On GPU Computing with Python
Avimanyu Bandyopadhyay
May, 2019 | 452 pages
Table of Contents (15 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Why GPU Programming?
Why GPU Programming?
Technical requirements
Parallelization and Amdahl's Law
Profiling your code
Summary
Questions
2
Setting Up Your GPU Programming Environment
Setting Up Your GPU Programming Environment
Technical requirements
Ensuring that we have the right hardware
Installing the GPU drivers
Setting up a C++ programming environment
Setting up our Python environment for GPU programming
Summary
Questions
3
Getting Started with PyCUDA
Getting Started with PyCUDA
Technical requirements
Querying your GPU
Using PyCUDA's gpuarray class
Using PyCUDA's ElementWiseKernel for performing pointwise computations
Summary
Questions
4
Kernels, Threads, Blocks, and Grids
Kernels, Threads, Blocks, and Grids
Technical requirements
Kernels
Threads, blocks, and grids
Thread synchronization and intercommunication
The parallel prefix algorithm
Summary
Questions
5
Streams, Events, Contexts, and Concurrency
Streams, Events, Contexts, and Concurrency
Technical requirements
CUDA device synchronization
Events
Contexts
Summary
Questions
6
Debugging and Profiling Your CUDA Code
Debugging and Profiling Your CUDA Code
Technical requirements
Using printf from within CUDA kernels
Filling in the gaps with CUDA-C
Using the Nsight IDE for CUDA-C development and debugging
Using the NVIDIA nvprof profiler and Visual Profiler
Summary
Questions
7
Using the CUDA Libraries with Scikit-CUDA
Using the CUDA Libraries with Scikit-CUDA
Technical requirements
Installing Scikit-CUDA
Basic linear algebra with cuBLAS
Fast Fourier transforms with cuFFT
Using cuSolver from Scikit-CUDA
Summary
Questions
8
The CUDA Device Function Libraries and Thrust
The CUDA Device Function Libraries and Thrust
Technical requirements
The cuRAND device function library
The CUDA Math API
The CUDA Thrust library
Summary
Questions
9
Implementation of a Deep Neural Network
Implementation of a Deep Neural Network
Technical requirements
Artificial neurons and neural networks
Implementation of the softmax layer
Implementation of Cross-Entropy loss
Implementation of a sequential network
The Iris dataset
Summary
Questions
10
Working with Compiled GPU Code
Working with Compiled GPU Code
Launching compiled code with Ctypes
Compiling and launching pure PTX code
Writing wrappers for the CUDA Driver API
Summary
Questions
11
Performance Optimization in CUDA
Performance Optimization in CUDA
Dynamic parallelism
Vectorized data types and memory access
Thread-safe atomic operations
Warp shuffling
Inline PTX assembly
Performance-optimized array sum
Summary
Questions
12
Where to Go from Here
Where to Go from Here
Furthering your knowledge of CUDA and GPGPU programming
Graphics
Machine learning and computer vision
Blockchain technology
Summary
Questions
13
Assessment
Assessment
Chapter 1, Why GPU Programming?
Chapter 2, Setting Up Your GPU Programming Environment
Chapter 3, Getting Started with PyCUDA
Chapter 4, Kernels, Threads, Blocks, and Grids
Chapter 5, Streams, Events, Contexts, and Concurrency
Chapter 6, Debugging and Profiling Your CUDA Code
Chapter 7, Using the CUDA Libraries with Scikit-CUDA
Chapter 8, The CUDA Device Function Libraries and Thrust
Chapter 9, Implementation of a Deep Neural Network
Chapter 10, Working with Compiled GPU Code
Chapter 11, Performance Optimization in CUDA
Chapter 12, Where to Go from Here
14
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Querying your GPU

Before we begin to program our GPU, we should really know something about its technical capacities and limits. We can determine this by doing what is known as a GPU query. A GPU query is a very basic operation that will tell us the specific technical details of our GPU, such as available GPU memory and core count. NVIDIA includes a command-line example written in pure CUDA-C called deviceQuery in the samples directory (for both Windows and Linux) that we can run to perform this operation. Let's take a look at the output that is produced on the author's Windows 10 laptop (which is a Microsoft Surface Book 2 with a GTX 1050 GPU):

Let's look at some of the essentials of all of the technical information displayed here. First, we see that there is only one GPU installed, Device 0—it is possible that a host computer has multiple GPUs and makes...

Previous Section
End of Section 3
Next Section