GPU Programming

When a GPU performs element-wise addition of two large arrays (c[i] = a[i] + b[i]), what mainly determines performance?

  1. AMemory bandwidth (12 bytes are read and written per element for just one addition)
  2. BThe number of arithmetic units
  3. CClock frequency alone
Show answer and explanation

Answer: A. Memory bandwidth (12 bytes are read and written per element for just one addition)

Each float addition reads 4 bytes each from a and b and writes 4 bytes to c, so there is one operation per 12 bytes moved. GPUs can compute tens of trillions of operations per second, but memory bandwidth is only a few TB/s, so this kernel is capped by bandwidth. Speeding up such work means reducing memory traffic, for example by fusing operations so intermediate results are never written to memory.

  • A. Correct. Its low arithmetic intensity makes it memory-bound.
  • B. The arithmetic units spend most of their time waiting.
  • C. Raising frequency does not help memory keep up.
Practice this topic Back to topic

More GPU Programming questions

  1. Why is double-precision (FP64) performance on many consumer gaming GPUs much lower than single-precision (FP32), for example one thirty-second?
  2. What is a characteristic of the tensor cores (matrix units) in recent GPUs?
  3. In GPU mixed-precision computation, why are multiplications often done in a low precision such as FP16 while accumulation is done in FP32?
  4. Summing the same floating-point data on a GPU gives results whose last digits differ slightly from run to run. What is the most likely cause?
  5. What is a common guideline for choosing the number of threads per block in CUDA?