Bayesian Inference

Posterior equals likelihood times prior, normalized. Watch beliefs update as evidence arrives.

Bayesian Inference

Start with a belief, look at the data, and update. Bayes' rule says exactly how: the posterior is the likelihood times the prior, renormalized.

Beta–Binomial Updater

Set a prior and add coin flips. Each head adds one to α and each tail one to β; watch the posterior tighten as the data washes the prior out.
observations
0H / 0T
posterior mean
0.5000
posterior mode (MAP)
—
posterior std
0.2887

Each flip moves the posterior parameters by exactly one: a head bumps α, a tail bumps β. With no data the posterior equals the prior. As evidence accumulates the curve narrows and centres on the empirical proportion — the prior's influence dwindles. Try a confident prior like Beta(8, 8) and watch how many flips it takes for the posterior to forget it.

Go deeper:Prior, likelihood, posterior

Bayesian inference treats unknown parameters as random variables and uses probability to encode what you believe about them. Start with a prior P(θ) — your beliefs before seeing data. Observe data D, and compute the likelihood P(D | θ) — how plausible the data is for each θ. Bayes' rule combines them into the posterior:

P(θ | D) = P(D | θ) · P(θ) / P(D)

The denominator P(D) — the evidence — just normalizes things; the shape of the posterior is set by the numerator. For a handful of special prior–likelihood pairs called conjugate families, the posterior lives in the same family as the prior, and the update is a one-line formula. The two demos below show the two classic cases: Beta–Binomial for proportions, and Normal–Normal for an unknown mean. The third demo shows that a familiar workhorse — ridge regression — is just a MAP estimate in disguise.

  • Prior P(θ): a probability distribution over the parameter space, encoding what you believe before seeing data. A flat prior says you know nothing; a sharp prior encodes a confident guess.
  • Likelihood P(D | θ): the probability the model assigns to the observed data, viewed as a function of θ with D fixed. Not a probability over θ — that's what makes the posterior different.
  • Posterior P(θ | D): the prior reweighted by the likelihood and renormalized. It is the answer to “given what I've seen, what should I now believe?”
  • Conjugate prior: a prior that, paired with a particular likelihood, yields a posterior in the same family. Beta is conjugate to Binomial; Normal (over μ) is conjugate to Normal (over data with known variance). Conjugacy keeps the math closed-form.
  • As data accumulates, the posterior concentrates and the prior's influence dwindles — Bayesian and frequentist estimates agree in the limit.

Gaussian Posterior over the Mean

Click the strip to add observations and watch the posterior over μ narrow. Its precision is the prior precision plus n times the data precision.

Click anywhere on the strip below to add an observation. The posterior over μ narrows toward the sample mean as more data arrives.

tip: a tight prior σ₀² resists the data; a loose prior lets the data dominate
n
0
sample mean x̄
—
posterior mean
0.000
posterior σ
2.000

With known data variance σ², the posterior over μ is Gaussian: the precisions add, and the posterior mean is a precision-weighted average of the prior mean and the sample mean. With one observation, the posterior already shifts noticeably; with many, it pins down μ almost exactly — and you can see the prior's pull fade.

MAP vs MLE — Ridge Regression as a Gaussian Prior

Slide λ to strengthen a Gaussian prior on the slope and watch the MAP line bend away from the MLE line. That MAP fit is ridge regression.

Click on the canvas to add a data point; click an existing point to remove it. Slide λ to strengthen the Gaussian prior on the slope — the MAP line bends toward zero slope (ridge regression).

MLE (no prior)
ŷ = 0.359 + 1.383 · x
SSE = 0.538
MAP (Gaussian prior, λ = 1.000)
ŷ = 0.368 + 1.348 · x
SSE = 0.585  ·  loss + λ‖β‖² = 2.402

MLE picks the parameters that maximize the data likelihood — for Gaussian noise that's ordinary least squares. MAP adds a prior: a zero-mean Gaussian prior on the slope contributes a λ‖β‖² penalty, exactly recovering ridge regression. As λ → 0 the MAP line collapses to MLE; as λ → ∞ it flattens to ȳ. The point: MAP = MLE + regularizer, and the regularizer is a prior in disguise.

Go deeper:The math objects
  • MAP estimate: argmaxθ P(θ | D) — the posterior mode. With a flat prior, MAP equals MLE. With a Gaussian prior on linear regression coefficients, MAP equals ridge regression. Many regularizers are priors in disguise: L² penalty ↔ Gaussian prior, L¹ penalty ↔ Laplace prior.
  • MLE estimate: argmaxθ P(D | θ) — the parameter that makes the data most likely. It ignores the prior, so it overfits when data is scarce.

Key takeaways

  • Bayes' rule: posterior ∝ likelihood × prior.
  • Conjugate priors keep the update in closed form.
  • For Beta–Binomial, the update is literally α ← α + heads, β ← β + tails.
  • Ridge regression is MAP estimation with a Gaussian prior.

Spot a mistake or have an idea?

Search