AI Hardware & Software Stack

Which describes the "weight-stationary" dataflow used in AI accelerators?

  1. AWeights are reread from memory each time while inputs stay fixed
  2. BWeights are kept fixed inside each unit while input values stream through for successive multiply-accumulates, reducing weight reads
  3. CComputation proceeds without weights
Show answer and explanation

Answer: B. Weights are kept fixed inside each unit while input values stream through for successive multiply-accumulates, reducing weight reads

For matrix products, dataflows differ by which of weights, inputs or outputs stays inside the units for reuse. In weight-stationary dataflow, each unit loads one weight and holds it while many inputs stream through for continuous multiply-accumulates. For convolutions and large-batch work where the same weight serves many inputs, this greatly reduces the energy of reading weights.

  • A. Keeping inputs fixed is a different dataflow.
  • B. Correct. Many systolic arrays work this way.
  • C. Weights are needed for computation.
Practice this topic Back to topic

More AI Hardware & Software Stack questions

  1. Which is NOT an advantage of running AI inference on the device itself (at the edge), such as a smartphone, rather than on cloud servers?
  2. Why are the NPUs in smartphone SoCs more power-efficient than CPUs for AI inference?
  3. What should be kept in mind about AI accelerator performance figures quoted as "XX TOPS"?
  4. Which describes MFU (Model FLOPs Utilization), a metric of large-scale training efficiency?
  5. For a Transformer with N parameters trained on D tokens, which approximation is commonly used for the number of training operations?