AI Hardware & Software Stack
Which describes the 2:4 structured sparsity supported by GPU matrix units?
- AWeights are grouped in fours with two zeros in each group; this fixed pattern lets matrix units skip zeros for up to about 2× speedup
- BTwo of every four weights are chosen at random each time
- CMatrices are split into 2 × 4 blocks for computation
Show answer and explanation
Answer: A. Weights are grouped in fours with two zeros in each group; this fixed pattern lets matrix units skip zeros for up to about 2× speedup
With 2:4 sparsity, training or fine-tuning ensures that two of every four consecutive weights are zero. Only the two non-zero values and a small index of their positions need storing, so memory shrinks, and matrix units skip the zero multiplications for about twice the speed. The fixed pattern made hardware acceleration practical where scattered sparsity is difficult.
- A. Correct. Zero positions can be encoded with little metadata.
- B. The choice is not random.
- C. It is not about how matrices are partitioned.