AI Hardware & Software Stack
Why do some AI accelerators include large on-chip SRAM (tens to hundreds of MB)?
- ASRAM retains data with power off
- BSRAM is cheaper per bit than DRAM
- CTo keep weights and intermediate results on chip for repeated use, reducing accesses to slower, more power-hungry external memory
Show answer and explanation
Answer: C. To keep weights and intermediate results on chip for repeated use, reducing accesses to slower, more power-hungry external memory
Reading data once from off-chip DRAM costs tens to hundreds of times the energy of a single multiply-accumulate. Keeping data in on-chip SRAM for repeated use reduces external accesses, improving both speed and energy efficiency. SRAM consumes much area and is expensive, though, so how much to include is a balance between target model size and cost.
- A. SRAM is volatile and loses data at power-off.
- B. SRAM is far more expensive per bit than DRAM.
- C. Correct. Moving data costs more energy than computing.