Performance Analysis
When evaluating a claim that "the new method is 100× faster," what is most important to check?
- AWhether 100 is an even number
- BThe length of the claimant's name
- CWhether the baseline was well optimized and whether the comparison used the same conditions and accuracy
Show answer and explanation
Answer: C. Whether the baseline was well optimized and whether the comparison used the same conditions and accuracy
Speedup depends heavily on what it is compared against. Comparing with an unoptimized old program or one using a single core easily yields large factors. Differences in data size, accuracy or measurement scope (for example whether transfer time is included) also make a comparison unfair. Evaluating a claim properly requires checking that the baseline and measurement conditions are clearly stated.
- A. Properties of the number are irrelevant.
- B. Names are irrelevant.
- C. Correct. The baseline can make the factor look arbitrarily large.