Coin flips, sample means, and the gambler's ruin
Flip a coin ten times and anything can happen. Flip it ten thousand times and the average barely budges. That is the law of large numbers, and it is why the casino wins.
Hit "Run" and track the running proportion of heads. Early on it swings wildly; after thousands of flips it hugs 0.5.
Watch the proportion of heads converge to 0.5 as the number of flips grows. The shaded band shows the expected 1/√n convergence rate.
The Law of Large Numbers (LLN) is one of the most fundamental theorems in probability. It says that as you take more and more independent samples from a distribution, the sample average converges to the true expected value. This is why casinos always win in the long run, why polls become more accurate with larger samples, and why insurance companies can predict costs.
The weak LLN says convergence happens in probability; the strong LLN says it happens almost surely (with probability 1). Both versions require finite mean, but say nothing about the rate of convergence -- that's where the Central Limit Theorem comes in.
The simplest illustration: flip a fair coin and track the running proportion of heads. After 10 flips, you might see 70% heads or 30% heads -- that's normal randomness. But after 10,000 flips, the proportion is almost certainly within 1% of 0.5. Hit "Run" and watch it happen in real time.
Key insight: The proportion converges, but the absolute deviation |heads - n/2| actually grows like √n. The LLN says the ratio converges, not that fluctuations disappear -- they just become negligible relative to the total.
Each colored path is a running sample mean. Watch every path settle on the dashed true mean, then switch to the exponential and see how much rougher the start is.
Each colored line is an independent sequence of running sample means. All paths start volatile but converge to the true mean (dashed line) -- this is the Law of Large Numbers.
The LLN works for any distribution with finite mean, not just coin flips. Each colored path represents an independent sequence of running sample means from the same distribution. Watch all paths converge to the true mean (dashed line), no matter how they start.
Key insight: Try the exponential distribution -- early sample means are volatile because the distribution is skewed and has high variance. But convergence is guaranteed nonetheless. The speed of convergence depends on the variance: lower variance means faster convergence.
Set the win probability p and run the gamblers. Compare p = 0.49 with p = 0.51: a 2% edge decides who goes broke.
10 gamblers each start with $50 and bet $1 per round. They win each bet with probability p and stop when they hit $100 (win) or $0 (ruin). At p=0.5, ruin probability = 1 - start/target.
A gambler starts with $N and bets $1 each round, winning with probability p. They stop when they reach a target or go broke. When p = 0.5 (fair game), the gambler is eventually ruined with probability (target - start)/target. Even a tiny edge (p = 0.49) dramatically increases the ruin probability -- the law of large numbers ensures the house edge accumulates relentlessly.
Key insight: Try p = 0.49 vs p = 0.51. With p = 0.49, almost every gambler goes broke. With p = 0.51, most win! This tiny 2% difference becomes decisive over thousands of bets -- the very essence of the LLN applied to gambling.
Spot a mistake or have an idea?