GPU Programming

In general, how do GPU caches compare with CPU caches?

  1. AGPUs have no caches at all
  2. BGPUs have L1 and L2 caches, but with so many concurrent threads the cache capacity per thread is far smaller
  3. CGPU caches are many times larger per thread than CPU caches
Show answer and explanation

Answer: B. GPUs have L1 and L2 caches, but with so many concurrent threads the cache capacity per thread is far smaller

GPUs have an L1 cache per SM and an L2 cache shared across the chip, but with tens of thousands of threads running simultaneously, the capacity available per thread is orders of magnitude smaller than for a CPU thread. Rather than relying on caches to reduce latency, GPUs hide latency by switching among many warps and depend on coalescing and shared memory to use memory well.

  • A. They do have L1 and L2 caches.
  • B. Correct. That is why coalescing and shared memory matter.
  • C. With so many threads, the per-thread share is small.
Practice this topic Back to topic

More GPU Programming questions

  1. Which is an example of a parallel-programming framework for GPUs that also works on GPUs other than NVIDIA's?
  2. On a GPU you want to compact only the elements meeting a condition into another array (each thread outputs 0 or 1 items). Which parallel computation is commonly used to determine each thread's write position?
  3. Which assumption must be respected when writing GPU kernels?
  4. When transposing a large matrix on a GPU (b[j][i] = a[i][j]), what is the main purpose of going through shared memory?
  5. In CUDA, what generally happens right after the CPU-side program launches a kernel?