FP32 vs FP16 vs BF16 vs INT8: Key Differences Explained
FP32 is the 32-bit single-precision floating-point format and is used as the reference for calculations. FP16 and BF16 are both 16 bits but allocate their bits differently. FP16 spends more bits on the mantissa for precision, so its range is narrow, with a maximum of 65504. BF16 keeps the exponent at 8 bits, the same as FP32, preserving an FP32-like range at the cost of precision. INT8 is an 8-bit integer used for quantization, which approximates real numbers by applying a scale, and it greatly reduces memory and power in AI inference.
| Aspect | FP32 | FP16 | BF16 | INT8 |
|---|---|---|---|---|
| Bit width | 32 bits | 16 bits | 16 bits | 8 bits |
| Sign, exponent and mantissa bits | 1, 8, 23 | 1, 5, 10 | 1, 8, 7 | No exponent or mantissa. Two's complement integer from −128 to 127 |
| Largest representable value | About 3.4 × 10³⁸ | 65504 | About 3.4 × 10³⁸ (almost the same as FP32) | 127 (real values are represented with a scale) |
| Smallest normal number | About 1.2 × 10⁻³⁸ | About 6.1 × 10⁻⁵ (2⁻¹⁴) | About 1.2 × 10⁻³⁸ | None (the step between representable values is set by the scale) |
| Precision | About 7 decimal digits (the gap between 1 and the next number is 2⁻²³ ≈ 1.2 × 10⁻⁷) | About 3 decimal digits (2⁻¹⁰ ≈ 9.8 × 10⁻⁴) | 2 to 3 decimal digits (2⁻⁷ ≈ 7.8 × 10⁻³) | 256 levels |
| Conversion from FP32 | None (the reference) | The range is narrow, so large values can overflow and small values can become 0 | Just take the upper 16 bits and round; the range does not overflow | Quantize by choosing a scale (and zero point). Calibration to examine the value range is needed |
| Memory per value | 4 bytes | 2 bytes | 2 bytes | 1 byte |
| Arithmetic unit efficiency | Large multipliers; fewer multiply-accumulates fit in the same area | Arithmetic units are smaller than for FP32, so many more multiply-accumulates can be placed | About the same as FP16. The shorter mantissa makes the multiplier even smaller | Smallest multipliers and lowest power. Products are often accumulated as 32-bit integers |
| Use in training | Master weights, optimizer states, accumulation of products | Mixed-precision training. Loss scaling keeps small gradients from becoming 0 | Widely used in mixed-precision training. Loss scaling is often unnecessary | Generally not used (quantization-aware training simulates the effects of quantization) |
| Use in inference | Accuracy reference. When accuracy takes priority over performance | Inference on GPUs and similar hardware | Inference on GPUs and AI accelerators | Inference of quantized models, especially on edge NPUs |
Figures are typical orders of magnitude and vary by process generation and product.