What is Arithmetic intensity?
Operations performed divided by bytes moved to and from memory. High values tend toward compute-bound, low values toward memory-bound.
Field: AI Hardware & Software Stack
Arithmetic intensity in the explanations
The roofline model plots performance against arithmetic intensity (operations per byte read from memory).
Which model plots whether a program's performance is limite…Arithmetic intensity is 2N³ ÷ 6N² = N/3, about 340 for N = 1024.
For the product of two N × N FP16 (2-byte) matrices, assumi…In the Roofline model, attainable performance is the smaller of the peak and bandwidth × arithmetic intensity.
On an accelerator with 100 TFLOPS peak and 2 TB/s memory ba…
Questions that mention Arithmetic intensity (7)
- Which model plots whether a program's performance is limited by compute throughput or memory bandwidth, using arithmetic intensity (operations per byte) on the horizontal axis?
- For the product of two N × N FP16 (2-byte) matrices, assuming each input is read once and the result written once, roughly what is the arithmetic intensity (operations per byte)?
- On an accelerator with 100 TFLOPS peak and 2 TB/s memory bandwidth, a workload with arithmetic intensity 10 FLOP/byte runs. What performance does the Roofline model predict?
- Why does a fully connected layer (a weight-matrix-by-vector product) tend to be memory-bound with a single input (batch size 1)?
- Why does increasing the batch size (the number of inputs processed together) tend to improve inference efficiency?
- Why do element-wise operations such as ReLU or layer normalization tend to take a lot of time relative to their operation count in AI workloads?
- When the Roofline model shows a workload in the memory-bandwidth-limited region, which direction improves performance?
Related terms
Roofline model Batch size Throughput Operator fusion AI compiler AI inference All-reduce Attention mechanism Backpropagation Computational graph