GPU Programming

Summing the same floating-point data on a GPU gives results whose last digits differ slightly from run to run. What is the most likely cause?

  1. AThe GPU's arithmetic units are broken
  2. BThe data is being altered inside the GPU each time
  3. CThe addition order varies between runs (for example with atomics), and floating-point addition rounds differently depending on order
Show answer and explanation

Answer: C. The addition order varies between runs (for example with atomics), and floating-point addition rounds differently depending on order

Floating-point addition rounds every time, so (a + b) + c and a + (b + c) can differ slightly. In a parallel sum, which thread's value is added in what order depends on thread scheduling and the arrival order of atomic operations. As a result, the last digits change between runs. When identical results are required, a reduction with a fixed addition order is used.

  • A. It is normal and happens on healthy hardware.
  • B. The data does not change.
  • C. Correct. Floating-point addition is not associative.
Practice this topic Back to topic

More GPU Programming questions

  1. What is a common guideline for choosing the number of threads per block in CUDA?
  2. What is the advantage of writing a kernel with a loop like the one above (a grid-stride loop)?
  3. For what kind of access does CUDA constant memory perform especially well?
  4. In general, how do GPU caches compare with CPU caches?
  5. Which is an example of a parallel-programming framework for GPUs that also works on GPUs other than NVIDIA's?