What is FlashAttention?
A fast attention implementation that tiles the computation in on-chip memory and never writes the large intermediate table to HBM. Its results match standard attention.
Field: AI Hardware & Software Stack
FlashAttention in the explanations
FlashAttention splits queries, keys and values into blocks loaded into shared memory and carries the computation through in one pass, updating the softmax incrementally, never writing the table to HBM.
What is the main idea behind fast attention implementations…
Questions that mention FlashAttention (1)
Related terms
HBM AI compiler AI inference All-reduce Arithmetic intensity Attention mechanism Backpropagation Batch size Computational graph Data parallelism