Performance Analysis

In software development, what is the purpose of automatically running benchmarks and recording results on every change?

  1. ATo make benchmark results identical every time
  2. BTo add features to the program automatically
  3. CTo notice quickly when a change degrades performance (a performance regression) and to pinpoint the responsible change
Show answer and explanation

Answer: C. To notice quickly when a change degrades performance (a performance regression) and to pinpoint the responsible change

Small changes accumulating can gradually erode performance without anyone noticing. Running benchmarks under the same conditions on every change and recording results means a slowdown is spotted at once and the responsible change identified immediately. Noticing months later makes it hard to find the cause among many changes. It is also necessary to account for measurement noise and flag only meaningful differences.

  • A. Results vary with changes and environment, so they are recorded and compared.
  • B. It does not add features.
  • C. Correct. Performance must be protected like functionality.
Practice this topic Back to topic

More Performance Analysis questions

  1. Benchmarks on a laptop vary widely between runs of the same program. Which pairs a CPU-side cause with a countermeasure correctly?
  2. When measuring performance on multicore or NUMA systems, why are threads sometimes pinned to specific cores (setting affinity)?
  3. When analyzing the performance of a GPU program, what is the most appropriate thing to check first?
  4. When choosing servers for a data center, which is NOT a consideration beyond peak performance?
  5. When evaluating a claim that "the new method is 100× faster," what is most important to check?