English
Menu

L1 vs L2 vs L3 Cache: Key Differences Explained

L1, L2 and L3 caches form a multi-level hierarchy ordered by distance from the CPU core: the closer a level is, the smaller and faster it is, and the farther it is, the larger and slower. In typical PC and server CPUs, representative ranges are tens of KB and a few cycles for L1, hundreds of KB to a few MB and a dozen or so cycles for L2, and a few MB to tens of MB and tens of cycles for L3. L1 and L2 are usually private to each core, while L3 is shared by several cores. An access to main-memory DRAM takes hundreds of cycles, so this hierarchy absorbs as many accesses as possible.

AspectL1 cacheL2 cacheL3 cache
Relationship to coresPrivate to each coreUsually private to each core. Some designs share it among a few coresShared by multiple cores (the whole chip or a cluster of cores)
Capacity (representative range for PC and server CPUs)About 32 to 64 KB each for instructions and dataAbout 256 KB to a few MB per coreA few MB to tens of MB for the whole chip
Latency (representative range)About 3 to 5 cyclesAbout 10 to 20 cyclesAbout 30 to 70 cycles. Longer for parts farther from the core
Instructions and dataSplit (I-cache and D-cache)Shared (unified cache)Shared (unified cache)
Design priorityShort hit time. Used every cycle within the pipelineAbsorbing L1 misses and reducing the miss penaltyReducing main-memory accesses and sharing data among cores
Where a miss goesL2L3Main memory (DRAM)
AddressingOften virtually indexed, physically tagged (VIPT)Physical addressesPhysical addresses
InclusionTopmost level. Whether it is contained in the lower levels depends on their policyDepends on the design, such as inclusive or non-inclusiveMaking it inclusive makes it easier to narrow down coherence checks. Exclusive designs that eliminate duplication and non-inclusive designs also exist
Physical placementInside the core, right next to the execution unitsJust outside the core, placed as part of the coreOften divided into per-core slices connected by a ring or mesh network
Line size64 bytes is common64 bytes is common64 bytes is common

Figures are typical orders of magnitude and vary by process generation and product.

When to use which

L1 cache
The closest cache, used on every instruction fetch and every load and store. It pays off when frequently used data is packed into a small range to improve temporal locality.
L2 cache
Absorbs working data that does not fit in L1. Techniques such as loop blocking work well when they keep the working data to around hundreds of KB to a few MB.
L3 cache
Helps when multiple cores share data or when handling large data that does not fit in the per-core caches. Because it competes for capacity with other cores, it is affected by programs running at the same time.

FAQ

Why isn't the L1 cache made larger?

A larger capacity lengthens the memory-cell arrays and wiring, which increases hit time. L1 is used every cycle in the pipeline, so even one cycle of delay has a large impact on performance. L1 is therefore kept to tens of KB, and large capacity is provided by L2 and L3. With VIPT, capacity is also limited by the relationship between page size and associativity.

Why is only L1 split into instruction and data caches?

In a pipeline, instructions are fetched every cycle while other instructions read and write data. Splitting the caches lets the two accesses happen simultaneously and lets each cache be organized to suit its use. From L2 on, accesses are less frequent, so it is common to use a unified cache whose capacity can be shared flexibly.

Which is better, inclusive or exclusive?

Each has pros and cons. In an inclusive cache, the contents of the upper levels are always also in the lower level, so snoops from other cores can be answered by the lower level alone, but duplication reduces the effective capacity. An exclusive cache has no duplication and makes good use of the total capacity, but moving lines and managing coherence become more complex. The choice depends on the number of cores and the cache sizes.

Are the capacities and cycle counts in the table the same for every CPU?

No, they are representative ranges assuming PC and server CPUs. They vary with the design generation, clock frequency, number of cores and chip organization. Many embedded microcontrollers have no L2 or L3, while some CPUs enlarge L3 by stacking cache chips. Actual values are checked with performance counters or programs that measure memory latency.

Check your understanding

In a typical computer, which lists the memory hierarchy from fastest to slowest access?

  1. ASSD → main memory → L2 cache → L1 cache → registers
  2. BMain memory → registers → SSD → L1 cache
  3. CRegisters → L1 cache → L2 cache → main memory (DRAM) → SSD
Show answer and explanation

Answer: C. Registers → L1 cache → L2 cache → main memory (DRAM) → SSD

Storage closer to the CPU is faster and smaller; farther away it is slower, larger and cheaper. Roughly: registers under one cycle, L1 about 1 ns (tens of KB), L2 a few ns (a few MB), L3 over ten ns (tens of MB), DRAM about 100 ns (tens of GB), and SSDs tens to 100 µs (several TB). Keeping hot data in the fast levels delivers fast, large and affordable storage overall.

What is the main advantage of splitting the L1 cache into an instruction cache (I-cache) and a data cache (D-cache)?

  1. AInstructions and data no longer collide even at the same address, so capacity seems doubled
  2. BInstruction fetch and data reads or writes can proceed in parallel in the same clock cycle
  3. CSince instructions are not modified, the D-cache can be eliminated
Show answer and explanation

Answer: B. Instruction fetch and data reads or writes can proceed in parallel in the same clock cycle

In a pipeline, an instruction is fetched every cycle while another instruction reads or writes data. Splitting L1 into instruction and data caches lets these two accesses proceed in parallel without contending for a single cache port, and each can be organized to suit its own behavior. Capacity is not doubled; the total is divided between them. Data accesses are always needed, so the D-cache cannot be dropped. L2 and beyond are usually unified.

In a multilevel cache with an L2 below the L1, what is the L2's main role?

  1. ATo eliminate L1 misses altogether by having a faster hit time than the L1
  2. BLarger and slower than L1, it reduces the L1 miss penalty
  3. CTo always hold all of the program's data in place of main memory
Show answer and explanation

Answer: B. Larger and slower than L1, it reduces the L1 miss penalty

L1 is built small for speed, so some misses are inevitable. L2 is larger with a longer hit time but still much faster than main memory, so it catches most L1 misses and cuts the L1 miss penalty well below a trip to memory. It is not faster than L1 and cannot eliminate L1 misses. Its capacity is far smaller than main memory, so it cannot hold all data.

  1. Which describes an inclusive cache, where a lower level (such as L2) contains everything in the upper level (L1)?
  2. What generally happens to hit time as cache capacity increases, and why?
  3. With an L1 hit time of 1 ns and miss rate of 5%, an L2 hit time of 10 ns and local miss rate of 20%, and main memory access time of 100 ns, what is the average memory access time?

Practice Memory Hierarchy questions

Key terms explained

Cache memory
Fast, small memory placed close to the CPU. It holds frequently used data to bridge the speed gap with main memory. Mostly built from SRAM.
Average memory access time
The average time per memory access, computed as hit time + miss rate × miss penalty. A basic metric for comparing cache designs.
Miss penalty
The extra time needed to fetch data from the next level when a cache miss occurs, determined by the lower level's access time and the line transfer time.
Cache coherency
In a multicore CPU, the mechanism that keeps copies of the same address in different cores' caches consistent, for example by invalidating other copies when one core writes.
Temporal locality
The tendency to access the same data or instructions again soon after a first access, as with variables reused inside a loop. One of the reasons caches work.

Related comparisons