GPU Programming

What is the main way a GPU hides long memory latency (hundreds of cycles)?

  1. AIt gives each thread a cache many times larger than a CPU's
  2. BWhile one warp waits for memory, it switches to other ready warps to keep the execution units busy
  3. CIt guesses data values instead of waiting for memory
Show answer and explanation

Answer: B. While one warp waits for memory, it switches to other ready warps to keep the execution units busy

A CPU reduces one thread's waiting with large caches and complex circuitry, whereas a GPU keeps many warps resident, sets aside those waiting for memory and issues instructions from other warps that are ready. Because all registers stay on chip, switching costs nothing. With enough warps, the execution units stay busy throughout memory latency.

  • A. Cache per thread is actually small.
  • B. Correct. Many threads fill the waiting time.
  • C. It does not compute with guessed values.
Practice this topic Back to topic

More GPU Programming questions

  1. Which statement about GPU occupancy is correct?
  2. In CUDA, what is the relationship between blocks and SMs (streaming multiprocessors)?
  3. In CUDA, you want to wait mid-kernel for every other block to finish before moving to the next phase. What is the usual approach?
  4. Building a histogram of pixel values on a GPU can be slow if every thread does atomicAdd directly on 256 bins in global memory. What is a good improvement?
  5. When summing many values on a GPU, why use a tree-shaped reduction (repeatedly adding neighbors to halve the count)?