Performance Analysis

When timing a program, why are measurements often repeated with the first few runs discarded?

  1. AThe first few runs always produce wrong results
  2. BEarly runs are slow because caches are cold or files are being loaded or code JIT-compiled, and measurements vary
  3. CRepeating measurements makes the CPU build faster circuits
Show answer and explanation

Answer: B. Early runs are slow because caches are cold or files are being loaded or code JIT-compiled, and measurements vary

On the first run, caches, TLBs and branch predictors start empty, files must be read from disk because they are not in the OS cache, and in JIT-compiled languages code is being translated to machine code. So the first few runs are slow. Measurements also vary due to other programs, interrupts and temperature-driven frequency changes, so it is important to measure many times and report the mean or median and the spread.

  • A. Results are correct; only the time differs.
  • B. Correct. This is called warm-up.
  • C. Circuits do not change.
Practice this topic Back to topic

More Performance Analysis questions

  1. What is the problem with timing a very short operation once, and what is the fix?
  2. What is the difference between wall-clock time and CPU time?
  3. Which correctly describes throughput and latency as server performance measures?
  4. When evaluating web service response times, why is the 99th percentile (the boundary of the slowest 1%) often examined in addition to the mean?
  5. A server receives 100 requests per second, each taking 0.2 seconds on average (steady state). How many requests are in progress at once on average (Little's law)?