Memory Hierarchy

Each miss fetches a 64-byte line and memory latency is 100 ns. With only one miss outstanding at a time, read bandwidth is about 0.64 GB/s. If 10 misses can be outstanding at once, what is the bandwidth approximately (assuming no other limits)?

  1. A0.064 GB/s
  2. B0.64 GB/s
  3. C6.4 GB/s
Show answer and explanation

Answer: C. 6.4 GB/s

Bandwidth equals the amount of data in flight divided by latency (Little's law). With one miss, 64 B ÷ 100 ns = 0.64 GB/s; with 10 outstanding, 10 × 64 B ÷ 100 ns = 6.4 GB/s. Saturating memory bandwidth requires not just lower latency but memory-level parallelism, servicing many misses at once. 0.64 GB/s is the single-miss value unchanged, and 0.064 GB/s divides by 10 instead of multiplying.

  • A. 0.064 GB/s divides by 10 instead of multiplying.
  • B. 0.64 GB/s is the value with only one outstanding miss.
  • C. Correct. 10 × 64 B ÷ 100 ns = 6.4 GB/s.
Practice this topic Back to topic

More Memory Hierarchy questions

  1. When writing out a large amount of data that will not be read back for a while, what is the main benefit of non-temporal stores that bypass the cache?
  2. If the cache hit rate is 95%, what is the miss rate?
  3. Which technique, when fetching a missed line from memory, sends the word the CPU needs first and the rest afterward?
  4. Why does fusing the two loops above into one (loop fusion) tend to be faster when N is large?
  5. Which pair gives the reason a large matrix transpose (b[j][i] = a[i][j]) tends to cause many cache misses, and a typical countermeasure?