Lossless coding answers the question “how few bits do I need to recover the source exactly?” The answer is the entropy H(X). Most of the world, though, does not need exact recovery. A photograph that's a fraction of a percent off pixel-by-pixel is still the same photograph; an audio signal off by a tenth of a decibel is still the same song. Lossy compression — JPEG, MP3, MPEG, every modern codec — trades a small, controlled amount of fidelity for a large reduction in bits.

Rate-distortion theory is the mathematics of that trade-off. You fix a distortion measure d(x, x̂) — typically squared error — and a tolerable average distortion D. The rate-distortion function is the smallest rate R achievable at that distortion:

R(D) = min_{q(x̂|x) : E[d(X,X̂)] ≤ D}  I(X; X̂)

The minimum is over all conditional distributions q(x̂ | x) that meet the distortion budget; the objective is the mutual information between source and reconstruction. Lossless source coding is exactly the special case D = 0, where R(0) recovers H(X). For a Gaussian source N(0, σ²) under squared-error distortion the answer collapses to a single line:

R(D) = ½ log₂(σ² / D),  0 < D ≤ σ²;  R(D) = 0,  D > σ².

That single formula is enough to derive the half-bit rule of thumb every codec engineer carries around: doubling your tolerable distortion saves you exactly half a bit per sample.

• Distortion measure d(x, x̂): a non-negative function on source × reconstruction pairs. Squared error (x − x̂)² is the canonical choice; Hamming distance is the workhorse for discrete sources.
• Rate-distortion function R(D): the infimum of rates achievable at average distortion ≤ D. Always non-increasing and convex; tends to H(X) as D → 0 (discrete sources) or to ∞ (continuous sources); hits 0 at some finite D_max.
• Test channel q(x̂ | x): the conditional distribution that achieves the minimum. For a Gaussian source the optimal channel is itself Gaussian and easiest to write backwards: x = x̂ + N(0, D), with x̂ ~ N(0, σ²−D). The minimum is reached at the boundary where E[d(X, X̂)] = D exactly.
• Lossless coding as a special case: setting D = 0 recovers Shannon's source-coding theorem, R(0) = H(X). Every entropy bound you have already met is a rate-distortion bound at zero distortion.
• Operational meaning: Shannon's rate-distortion theorem says that for any δ > 0 and any rate R > R(D), there exists a code (operating on long blocks of source symbols) that achieves average distortion ≤ D + δ. Below R(D), no code can. R(D) is both the best possible rate and an achievable one.

• Practical codecs: JPEG, MP3, AAC, MPEG, H.264, and friends are all approximations to R(D) for the relevant source. They diverge from the theoretical optimum in two ways — their distortion measure is perceptual, not squared error, and they use a bounded number of operations per symbol — but the shape of the trade-off is the same.