Assume an SM of a GPU has one issue unit that can issue instructions from one warp per cycle. Each warp repeatedly issues 20 cycles of instructions and then waits 400 cycles for the result of a memory load. If switching warps takes no time, at least how many warps are needed to eliminate idle time due to memory waits (latency hiding)?

Answer: A. 21

One warp can issue instructions for only 20 out of every 420 cycles (20 + 400). To keep the issue unit busy, 420 ÷ 20 = 21 warps must take turns: while one warp waits 400 cycles, the other 20 warps fill that time by issuing 20 cycles each. With too few warps, the wait shows up in performance and the execution units sit idle. On a real GPU the number needed depends on the wait length and the instruction mix, and waits can also be filled by adding mutually independent instructions within one warp (instruction-level parallelism).

• A. Correct. (20 + 400) ÷ 20 = 21.
• B. One other warp's 20 cycles cannot fill one warp's 400-cycle wait.
• C. 420 is the length in cycles of one warp's repeating period, not a number of warps.