IEEE 754 Floating Point: From Bits to Numbers

TL;DR: For a normal binary value, IEEE 754 uses $(-1)^s(1+f)2^{e-bias}$: a sign bit, biased exponent, and trailing-significand field with an implicit leading 1. Reserved exponent patterns encode signed zero, subnormals, infinities, and NaNs. Because most exact real results must be rounded to a finite format, floating-point addition and multiplication are not generally associative.

Fixed-point binary works well when range and scale are known. It becomes inconvenient when one calculation must cover values separated by many orders of magnitude. A signed 32-bit fixed-point value with 16 fractional bits, for example, has a step of $2^{-16}$ and a range of roughly $-32{,}768$ to $32{,}767.99998$.

Floating-point allocates bits to an exponent, letting the radix point move with magnitude. IEEE 754 was first published in 1985; the active revision is IEEE 754-2019. It defines binary and decimal formats, operations, rounding, exceptions, and interchange behavior. An implementation may be hardware, software, or a combination, so conformance does not imply that every CPU contains every format in silicon.

Why Floating Point at All?

Scientific notation handles the same problem on paper: $6.674 \times 10^{-11}$ separates a small significand from a large exponent. IEEE 754 is binary scientific notation with strict rules about how many bits each part gets and how they’re encoded.

Two design pressures shape the format:

1. Specified encodings and operations. There is not one bit pattern per mathematical value: zero has two signs and NaNs have many payload encodings. The standard defines how operations and comparisons treat them.
2. Implementable arithmetic. Exponent comparison and significand alignment must map to finite hardware or software algorithms with specified rounding and exception behavior.

The trade-offs taken to achieve both — biased exponents, implicit leading bits, special-purpose exponent codes — give IEEE 754 its distinctive shape.

The Single-Precision Layout (32-bit)

| 31 | 30 ............ 23 | 22 ........................... 0 |
|  s |       e (8 bits)   |          m (23 bits)             |
| 1  |                    |                                  |

• Sign bit $s$: 1 bit. 0 means non-negative, 1 means negative.
• Exponent $e$: 8 bits, stored as an unsigned integer with a bias of 127.
• Trailing-significand field $m$: 23 bits. For normal numbers it stores the fraction after an implicit leading 1.

For normalized numbers (the common case), the value is:

$v = (-1)^s \cdot \left(1 + \frac{m}{2^{23}}\right) \cdot 2^{e - 127}$

The "$1.$" in the significand is implicit — the format saves a bit by exploiting the fact that every normalized binary number can be written with a leading 1 by choice of exponent.

The Double-Precision Layout (64-bit)

Same structure, wider fields:

| 63 | 62 ........... 52 | 51 ............................. 0 |
|  s |     e (11 bits)   |          m (52 bits)              |

• Sign bit $s$: 1 bit.
• Exponent $e$: 11 bits, bias 1023.
• Trailing-significand field $m$: 52 bits.

Value (normalized):

$v = (-1)^s \cdot \left(1 + \frac{m}{2^{52}}\right) \cdot 2^{e - 1023}$

Range: roughly $\pm 1.8 \times 10^{308}$ with about 15-17 significant decimal digits.

Why Bias the Exponent?

Storing the exponent as $e - bias$ rather than two’s complement has a subtle but important benefit: floating-point comparison reduces to integer comparison on the bit pattern (with a sign-bit flip). If two non-negative floats are compared as 32-bit unsigned integers, the larger float is the larger integer. Two’s complement exponents would break this monotonic relationship at the boundary between positive and negative exponents.

The all-zero exponent field is reserved for zeros and subnormals, while the all-ones field is reserved for infinities and NaNs. For binary32, normal exponent fields 1 through 254 correspond to unbiased exponents $-126$ through $+127$.

The Special Cases

Zero has two encodings ($+0$ and $-0$). They compare equal under IEEE comparison but produce different signed results in some operations (e.g., $1 / +0 = +\infty$, $1 / -0 = -\infty$).

Subnormals (historically also called denormals) fill the gap between zero and the smallest normal number. The implicit leading 1 becomes 0, and binary32 uses an effective exponent of $-126$. This provides gradual underflow: absolute spacing remains constant in the subnormal range while relative precision decreases toward zero.

Infinity results from overflow or division by zero. It propagates through arithmetic in the obvious ways: $\infty + 1 = \infty$, $\infty \times 0 = \text{NaN}$.

NaN signals an invalid operation: $\sqrt{-1}$, $0 / 0$, $\infty - \infty$. NaN is contagious — any arithmetic involving NaN yields NaN. Critically, $\text{NaN} \neq \text{NaN}$ in IEEE 754 comparison, which is why test suites use isnan() rather than ==.

Worked Example: Decoding 0x40490FDB

A famous 32-bit constant. In binary:

0x40490FDB = 0100 0000 0100 1001 0000 1111 1101 1011

Split into fields:

• Sign: 0 — positive
• Exponent: 1000 0000 = $128_{10}$ — actual exponent $128 - 127 = 1$
• Trailing significand: 100 1001 0000 1111 1101 1011

The significand is $1.\text{(trailing bits)}_2$, i.e.

$1.10010010000111111011011_2$

Compute the stored fraction:

$m_{frac} = \frac{0b10010010000111111011011}{2^{23}} = \frac{4{,}788{,}187}{8{,}388{,}608} \approx 0.5707963$

So the significand is $1.5707963$. Multiply by $2^1$:

$v \approx 3.1415927$

That’s $\pi$ to single-precision accuracy. The constant 0x40490FDB is the canonical IEEE 754 single-precision encoding of $\pi$ and appears in numerical libraries, GPU shaders, and exam questions worldwide.

Worked Example: Encoding 5.75

Convert $5.75$ to single-precision IEEE 754.

Step 1: convert to binary. $5 = 101_2$. $0.75 = 0.5 + 0.25 = 0.11_2$. Combined: $5.75 = 101.11_2$.

Step 2: normalize. Shift the binary point left until exactly one digit remains to its left:

$101.11_2 = 1.0111_2 \times 2^2$

So the actual exponent is $2$.

Step 3: encode each field.

• Sign: positive, so $s = 0$.
• Exponent: actual $2$ + bias $127$ = $129$ = 1000 0001.
• Trailing significand: drop the leading 1 and store the next 23 bits: 0111 followed by 19 zeros, or 01110000 00000000 0000000.

Step 4: assemble.

0 | 10000001 | 01110000000000000000000

In hex: 0x40B80000. You can verify by reversing the decoding procedure on this pattern and recovering 5.75 exactly.

Note that 5.75 has an exact binary representation — its fractional part is a sum of negative powers of two ($2^{-1} + 2^{-2}$). Most decimal fractions don’t. $0.1$ in single precision is 0x3DCCCCCD, which decodes back to $0.1000000014901...$, off by about $10^{-9}$. That tiny error is the root cause of every "$0.1 + 0.2 \neq 0.3$" surprise newcomers run into.

Rounding directions

IEEE 754-2019 names five rounding-direction attributes:

• Round to nearest, ties to even (default). Round to the closer representable value; on a tie, choose the value whose least-significant significand bit is 0.
• Round to nearest, ties away from zero. This attribute is particularly associated with decimal arithmetic and is not exposed by every binary floating-point environment.
• Round toward zero (truncation).
• Round toward $+\infty$ (ceiling).
• Round toward $-\infty$ (floor).

The default mode minimizes statistical bias — straightforward “round half up” introduces a slight upward bias when many values are accumulated, while round-half-to-even averages out.

Which directions an application can select depends on its language, runtime, compiler, and hardware environment. Reproducibility also depends on evaluation order, intermediate precision, fused operations, library functions, and treatment of subnormals—not only the selected rounding direction.

Why Floating-Point Arithmetic Isn’t Associative

Consider three IEEE 754 single-precision values: $a = 10^{20}, b = -10^{20}, c = 1$.

$(a + b) + c$: First, $a + b = 0$ exactly (the cancellation is clean). Then $0 + c = 1$. Final result: $1$.

$a + (b + c)$: First, $b + c$. The exponent of $b$ is around 67; the exponent of $c$ is 0. Aligning the significands moves $c$ far beyond binary32 precision. After rounding, $b+c=b$. Then $a+b=0$.

The two parenthesizations produce different answers, so floating-point addition is not associative. Whether a compiler may reassociate expressions depends on language rules, implementation, optimization flags, and the floating-point environment. Reproducibility-sensitive code should document those choices and test the actual toolchain instead of relying on one compiler flag as a universal guarantee.

The same property applies to subtraction (cancellation cases), multiplication (extremely small products may underflow to zero), and division (division by very small numbers may overflow to infinity).

Hardware: The FPU

A hardware floating-point adder commonly performs stages like these; processors without an FPU can implement the same semantics in software:

1. Decode both operands’ fields.
2. Compare exponents; shift the smaller-exponent significand right by the difference, with a guard, round, and sticky bit retained for rounding.
3. Add or subtract the aligned significands (using a wide carry-lookahead-style adder).
4. Normalize the result by left-shifting until a leading 1 emerges; adjust the exponent accordingly.
5. Round per the active rounding mode.
6. Encode the result into IEEE 754 format.

The datapath needs exponent comparison, significand alignment, add/subtract logic, leading-zero detection, normalization, rounding, special-case handling, and exception reporting. RISC-V’s F extension defines binary32 instructions and D adds binary64; both are optional, illustrating the distinction between a standard format and mandatory hardware.

The significand path reuses binary arithmetic techniques, but an FPU is more than an integer ALU with extra plumbing: rounding, gradual underflow, NaNs, infinities, signed zeros, and exceptions are part of its architectural behavior.

Wider Formats

Beyond single and double precision, IEEE 754 defines:

• Half precision (binary16): 1 sign + 5 exponent + 10 trailing-significand bits. It is used in graphics and reduced-precision numerical workloads.
• Quadruple precision (128-bit): 1 + 15 + 112. Rare; used in numerical analysis and some financial computation.
• bfloat16 (not an IEEE 754 basic interchange format): 1 + 8 + 7. It keeps binary32-like exponent range with much less precision and is widely used in machine learning.

Each variant trades off range against precision for its target workload.

Common Pitfalls

• Using == without an error model. Exact comparison is correct for some invariants and discrete values; approximate results need a domain-specific absolute/relative tolerance or units-in-last-place policy.
• Forgetting that NaN != NaN. Use isnan() to detect NaN.
• Assuming associativity. Reordering sums can change the answer. Compiler flags can enable unsafe reassociation; understand them before using them.
• Mixing decimal and binary intuitions. $0.1$ has no exact finite binary representation. For decimal business rules, consider scaled integers, decimal floating point, or BCD, and still define rounding at each required boundary.
• Assuming subnormal performance. Some processors handle subnormals at full speed; others incur a penalty or support flush-to-zero modes. Measure the target and preserve gradual underflow when correctness requires it.

Connecting to Wider CPU Architecture

Register organization is ISA-specific. Some architectures use separate integer and floating-point register files; others overlap or unify parts of the state. Loads, stores, conversions, moves, and calling conventions define the boundary.

For wider arithmetic, the integer side relies on structures related to the 8-bit adder, scaled and optimized for its target. The floating-point side adds alignment, normalization, rounding, exception, and special-value handling.

What’s Next

Continue with Two’s Complement Explained for fixed-width signed integers, then BCD: Binary-Coded Decimal Fundamentals for digit-oriented decimal encoding.

To experiment at bit level, use an ALU-style datapath for significand arithmetic, then model alignment, normalization, guard/round/sticky bits, and the special cases as separate stages.

The normative reference is the active IEEE 754-2019 Standard for Floating-Point Arithmetic.