Book Image

Hands-On GPU Programming with Python and CUDA

By : Dr. Brian Tuomanen

Book Image

Hands-On GPU Programming with Python and CUDA

By: Dr. Brian Tuomanen

Overview of this book

Hands-On GPU Programming with Python and CUDA hits the ground running: you’ll start by learning how to apply Amdahl’s Law, use a code profiler to identify bottlenecks in your Python code, and set up an appropriate GPU programming environment. You’ll then see how to “query” the GPU’s features and copy arrays of data to and from the GPU’s own memory. As you make your way through the book, you’ll launch code directly onto the GPU and write full blown GPU kernels and device functions in CUDA C. You’ll get to grips with profiling GPU code effectively and fully test and debug your code using Nsight IDE. Next, you’ll explore some of the more well-known NVIDIA libraries, such as cuFFT and cuBLAS. With a solid background in place, you will now apply your new-found knowledge to develop your very own GPU-based deep neural network from scratch. You’ll then explore advanced topics, such as warp shuffling, dynamic parallelism, and PTX assembly. In the final chapter, you’ll see some topics and applications related to GPU programming that you may wish to pursue, including AI, graphics, and blockchain. By the end of this book, you will be able to apply GPU programming to problems related to data science and high-performance computing.

Preface

Who this book is for

What this book covers

To get the most out of this book

Free Chapter

Why GPU Programming?

Why GPU Programming?

Technical requirements

Parallelization and Amdahl's Law

Profiling your code

Setting Up Your GPU Programming Environment

Setting Up Your GPU Programming Environment

Technical requirements

Ensuring that we have the right hardware

Installing the GPU drivers

Setting up a C++ programming environment

Setting up our Python environment for GPU programming

Getting Started with PyCUDA

Getting Started with PyCUDA

Technical requirements

Querying your GPU

Using PyCUDA's gpuarray class

Using PyCUDA's ElementWiseKernel for performing pointwise computations

Kernels, Threads, Blocks, and Grids

Kernels, Threads, Blocks, and Grids

Technical requirements

Threads, blocks, and grids

Thread synchronization and intercommunication

The parallel prefix algorithm

Streams, Events, Contexts, and Concurrency

Streams, Events, Contexts, and Concurrency

Technical requirements

CUDA device synchronization

Debugging and Profiling Your CUDA Code

Debugging and Profiling Your CUDA Code

Technical requirements

Using printf from within CUDA kernels

Filling in the gaps with CUDA-C

Using the Nsight IDE for CUDA-C development and debugging

Using the NVIDIA nvprof profiler and Visual Profiler

Using the CUDA Libraries with Scikit-CUDA

Using the CUDA Libraries with Scikit-CUDA

Technical requirements

Installing Scikit-CUDA

Basic linear algebra with cuBLAS

Fast Fourier transforms with cuFFT

Using cuSolver from Scikit-CUDA

The CUDA Device Function Libraries and Thrust

The CUDA Device Function Libraries and Thrust

Technical requirements

The cuRAND device function library

The CUDA Math API

The CUDA Thrust library

Implementation of a Deep Neural Network

Implementation of a Deep Neural Network

Technical requirements

Artificial neurons and neural networks

Implementation of the softmax layer

Implementation of Cross-Entropy loss

Implementation of a sequential network

The Iris dataset

Working with Compiled GPU Code

Working with Compiled GPU Code

Launching compiled code with Ctypes

Compiling and launching pure PTX code

Writing wrappers for the CUDA Driver API

Performance Optimization in CUDA

Performance Optimization in CUDA

Dynamic parallelism

Vectorized data types and memory access

Thread-safe atomic operations

Inline PTX assembly

Performance-optimized array sum

Where to Go from Here

Where to Go from Here

Furthering your knowledge of CUDA and GPGPU programming

Machine learning and computer vision

Blockchain technology

Assessment

Chapter 1, Why GPU Programming?

Chapter 2, Setting Up Your GPU Programming Environment

Chapter 3, Getting Started with PyCUDA

Chapter 4, Kernels, Threads, Blocks, and Grids

Chapter 5, Streams, Events, Contexts, and Concurrency

Chapter 6, Debugging and Profiling Your CUDA Code

Chapter 7, Using the CUDA Libraries with Scikit-CUDA

Chapter 8, The CUDA Device Function Libraries and Thrust

Chapter 9, Implementation of a Deep Neural Network

Chapter 10, Working with Compiled GPU Code

Chapter 11, Performance Optimization in CUDA

Chapter 12, Where to Go from Here

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

The Iris dataset

We will now construct our very own DNN for a real-life problem: classification of flower types based on the measurements of petals. We will be working with the well-known Iris dataset for this. This dataset is stored as a comma-separated value (CSV) text file, with each line containing four different numerical values (petal measurements), followed by the flower type (here, there are three classes—Irissetosa, Irisversicolor, and Irisvirginica). We will now design a small DNN that will classify the type of iris, based on this set.

Before we continue, please download the Iris dataset and put it into your working directory. This is available from the UC Irvine Machine Learning repository, which can be found here: https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data.

We will start by processing this file into appropriate data arrays that...