Accelerating data loading

Accelerating data loading is crucial to get an efficient data pipeline. In general, the following two changes are enough to get the work done:

• Optimizing a data transfer between the CPU and GPU
• Increasing the number of workers in the data pipeline

Putting it that way, these changes may sound tougher to implement than they are. Making these changes is quite simple – we just need to add a couple of parameters when creating the DataLoader instance for the data pipeline. We will cover this in the following subsections.

Optimizing a data transfer to the GPU

To transfer data from main memory to the GPU, and vice versa, the device driver must ask the operating system to pin or lock a portion of memory. After receiving access to that pinned memory, the device driver starts to copy data from the original memory location to the GPU, but using the pinned memory as a staging area:

Figure 5.6 – Data transfer...