Computer Arithmetic

Learn computer arithmetic with multiple-choice questions: fixed point, IEEE 754 floating point (normalization, subnormals, rounding, special values), rounding error and cancellation, bfloat16 and INT8, multipliers and carry-lookahead adders.

—Your accuracy
0 / 50Questions answered
Practice this topicAll questions in this topic, in random order Weak questionsRetry only the questions you last got wrong

Computer Arithmetic: key points

Computers represent numbers with a finite number of bits, so both range and precision are limited. Whereas fixed point keeps the binary point in a fixed position, IEEE 754 floating point splits a number into sign, exponent and significand to cover values from tiny to huge. It is important to understand normalization, subnormal numbers, rounding modes and special values such as infinity and NaN.

Floating-point calculations incur rounding error, and subtracting nearly equal numbers causes cancellation, which loses significant digits. AI uses short formats such as bfloat16, FP16, FP8 and INT8. Along with number formats, this topic covers the circuits that implement arithmetic in hardware, such as multipliers, dividers and carry-lookahead adders.

All Computer Arithmetic questions (50)

  1. In unsigned fixed point with the low 4 of 8 bits as the fraction, what value does the bit pattern 0011 1000 represent?
  2. Which three parts make up an IEEE 754 floating-point number?
  3. How are the bits of an IEEE 754 single-precision (32-bit) number allocated among sign, exponent and significand?
  4. In IEEE 754 double precision (64-bit), how many bits are the exponent and significand?
  5. In IEEE 754 single precision, if the stored exponent field is 130, what is the actual exponent (power of two)?
  6. What is the effect of not storing the leading 1 of the significand (the hidden bit) in IEEE 754 normalized numbers?
  7. What is the hexadecimal bit pattern of decimal 1.0 in IEEE 754 single precision?
  8. Converting −6.25 to single precision, what is the sign–exponent–significand bit string written in hexadecimal?
  9. Which correctly describes representing decimal 0.1 as a binary floating-point number?
  10. In many programming languages, why is the double-precision comparison 0.1 + 0.2 == 0.3 false?
  11. What is the machine epsilon (the gap between 1 and the next larger representable number) in IEEE 754 single precision?
  12. Which gives the approximate number of reliable decimal digits for single and double precision?
  13. What is the role of IEEE 754 subnormal numbers?
  14. On some processors, computations involving many subnormal numbers can slow down dramatically. Which setting is used when performance takes priority?
  15. Under IEEE 754, what result does the standard specify for dividing a positive number by zero?
  16. When x is NaN, what does evaluating x == x give in most programming languages?
  17. In IEEE 754 single precision, what do bit patterns with an all-ones exponent field (255) represent?
  18. Which correctly describes +0 and −0 in IEEE 754 floating point?
  19. Which rounding method does IEEE 754 specify as the default?
  20. Rounding to integers with round half to even, what do 2.5 and 3.5 become?
  21. What is the phenomenon called in which subtracting two nearly equal floating-point numbers greatly reduces the significant digits?
  22. For very large x, computing √(x + 1) − √x in floating point suffers catastrophic cancellation. Which rewrite computes it accurately?
  23. In single precision, adding 1 to 100 million (10⁸) leaves the result unchanged at 100 million. Which explains this?
  24. What is the purpose of Kahan (compensated) summation when adding many floating-point numbers?
  25. What is the advantage of computing a × b + c with an FMA (fused multiply-add) instruction?
  26. What is the main reason bfloat16 is used for AI training?
  27. When training neural networks in FP16 (half precision: 1 sign, 5 exponent, 10 significand bits), which problem tends to arise and is countered by scaling up the loss (loss scaling)?
  28. What is the largest finite value representable in FP16 (1 sign, 5 exponent bits with bias 15, 10 significand bits)?
  29. How are the two FP8 formats, E4M3 (4 exponent, 3 significand bits) and E5M2 (5 exponent, 2 significand bits), usually used?
  30. In AI inference, what is the main benefit of converting (quantizing) weights from FP32 to INT8?
  31. Data ranging from −6.35 to 6.35 is quantized to symmetric INT8 (−127 to 127) centered at zero, with scale = maximum absolute value ÷ 127. Which integer represents 1.0?
  32. Why is fixed-point rather than floating-point arithmetic often used in small microcontrollers and DSPs?
  33. Which gives the resolution and range of the 16-bit signed fixed-point Q15 format (15 fraction bits)?
  34. When two fixed-point numbers with 8 fraction bits are multiplied using an integer multiplier, how should the result be handled?
  35. Which describes saturating arithmetic used in signal processing?
  36. In a carry-lookahead adder, which gives the definitions of each bit's generate signal g and propagate signal p (a and b are that bit's inputs) and the carry into the next bit, c(i+1)?
  37. Why are carry-save adders used to sum the many partial products in a multiplier?
  38. Which describes a Wallace tree used in fast multipliers?
  39. What is the main effect of using radix-4 Booth's algorithm (Booth encoding) in a multiplier?
  40. When two unsigned 8-bit numbers are multiplied, how many bits are needed to represent the result without overflow?
  41. Which expression multiplies an integer x by 10 using shifts and additions instead of a multiply instruction?
  42. Why do typical integer dividers (restoring or non-restoring, producing one quotient bit at a time) tend to be slower than multipliers?
  43. What is the result of arithmetic right-shifting the 8-bit signed integer −8 (1111 1000) by one bit?
  44. In C, what are the results of −7 / 2 and −7 >> 1 (with an arithmetic shift) for int?
  45. In C, what is the result of subtracting 1 from 0 in a 32-bit unsigned integer (unsigned int)?
  46. How is overflow detected when adding two unsigned integers?
  47. Which correctly describes the spacing between adjacent representable floating-point values?
  48. Up to what value can IEEE 754 single precision represent every integer exactly, starting from 0?
  49. Why are monetary amounts often handled as integers in the smallest unit (one yen or one cent) rather than as binary floating point?
  50. In mixed-precision AI training, why are master weights kept in FP32 while most computation is done in FP16 or bfloat16?

Other topics in Computer systems