-
Book Overview & Buying
-
Table Of Contents
GPU-Accelerated Computing with Python 3 and CUDA
By :
A single GPU often becomes insufficient as computational problems grow in scale, for example, when the model is too large to fit into the memory of one GPU, or when fitting model parameters is so computationally demanding that distributing the load becomes necessary to achieve reasonable performance or time. In such cases, multi-GPU computing becomes unavoidable as it allows distributing the workload across several GPUs, enabling more data to be processed or simulations that would otherwise be impractical to run to be accelerated.
In the following two sections, we will briefly discuss the architecture of multi-GPU systems and elaborate on popular parallelism strategies for utilizing multiple GPUs.
A system with multiple GPUs is designed to utilize the combined computational power of several GPUs. The way these GPUs are arranged and connected within the system affects how efficiently they can share data and coordinate work...