GPU Programming

What is the main purpose of using multiple CUDA streams?

  1. ATo increase cooling airflow through the GPU
  2. BTo overlap data transfers with kernel execution and shorten total time
  3. CTo improve video streaming quality
Show answer and explanation

Answer: B. To overlap data transfers with kernel execution and shorten total time

A stream is an ordered queue of work issued to the GPU, such as transfers and kernel launches. Work within a stream runs in order, but work in different streams can proceed concurrently. By splitting the data and transferring the second part while computing on the first, transfer and computation largely overlap, shortening the total time.

  • A. It has nothing to do with cooling.
  • B. Correct. Transfer time can be hidden.
  • C. It is unrelated to video streaming.
Practice this topic Back to topic

More GPU Programming questions

  1. A program that launches a very small kernel thousands of times in a row is slow. What is the most suitable improvement?
  2. Which describes CUDA unified memory?
  3. When a GPU performs element-wise addition of two large arrays (c[i] = a[i] + b[i]), what mainly determines performance?
  4. Why is double-precision (FP64) performance on many consumer gaming GPUs much lower than single-precision (FP32), for example one thirty-second?
  5. What is a characteristic of the tensor cores (matrix units) in recent GPUs?