GPU Programming

Which assumption must be respected when writing GPU kernels?

  1. AThreads always execute one at a time in increasing index order
  2. BBlock 1 never starts before block 0 finishes
  3. CThe execution order of threads and blocks is unspecified, so results must not depend on execution order
Show answer and explanation

Answer: C. The execution order of threads and blocks is unspecified, so results must not depend on execution order

A GPU assigns threads and blocks to free resources as they become available, so execution order varies with the GPU model and load. If one thread relies on a value written by another, the code may happen to work but fail elsewhere. When order matters, use __syncthreads() within a block, separate kernels between blocks, and atomics or reductions for accumulation.

  • A. Many threads run simultaneously with no order guarantee.
  • B. Blocks may run concurrently in any order.
  • C. Correct. Use synchronization when order matters.
Practice this topic Back to topic

More GPU Programming questions

  1. When transposing a large matrix on a GPU (b[j][i] = a[i][j]), what is the main purpose of going through shared memory?
  2. In CUDA, what generally happens right after the CPU-side program launches a kernel?
  3. How do the design philosophies of CPUs and GPUs differ?
  4. Which characteristic best describes work that GPUs process quickly?
  5. Which processing tends to run inefficiently on a GPU?