Euler, Runge-Kutta, and adaptive step methods
Most differential equations cannot be solved exactly, so computers take small steps instead. How you take each step makes all the difference.
Step along the cyan Euler curve and compare it with the dashed exact solution; red bars show the error. Try a smaller step size h and watch the error shrink.
Euler's method approximates the solution by following the tangent line at each point. Smaller step sizes produce more accurate results. Red bars show the cumulative error.
Most differential equations cannot be solved exactly. Numerical methods approximate solutions by discretizing the continuous problem: instead of finding y(t) for all t, we compute y at a finite set of points t_0, t_1, t_2, ... separated by a step size h. At each step, we use the ODE itself to estimate where the solution goes next.
The simplest approach is Euler's method: follow the tangent line for a small step. More sophisticated methods like Runge-Kutta evaluate the slope at multiple points within each step to achieve dramatically better accuracy. Understanding these methods is essential for scientific computing, engineering simulation, and computational physics.
Euler's method is the foundation of all numerical ODE solvers. Given dy/dx = f(x,y) with y(x_0) = y_0, each step computes:
y_{n+1} = y_n + h * f(x_n, y_n)
The cyan curve shows the Euler approximation, while the dashed white curve shows the exact solution. Red bars indicate the error at each step. Try different step sizes to see how accuracy improves as h decreases.
Key insight: Euler's method has a local truncation error of O(h^2) per step, but errors accumulate over many steps, giving a global error of O(h). Halving the step size roughly halves the total error -- but doubles the number of steps required.
Change the step size and compare the errors of Euler, Heun, and RK4 on the same problem. RK4 stays on the exact curve long after Euler has drifted away.
| Method | y(3) | Error | Order |
|---|---|---|---|
| Exact | 0.00012341 | --- | --- |
| Euler | 0.00003835 | 8.506e-5 | O(h) |
| Heun (Improved Euler) | 0.00874678 | 8.623e-3 | O(h^2) |
| RK4 | 0.00027979 | 1.564e-4 | O(h^4) |
Higher-order methods achieve much better accuracy for the same step size. RK4 (4th order) is dramatically more accurate than Euler (1st order), especially with larger steps.
Higher-order methods evaluate the slope at multiple points within each step to achieve far better accuracy. The improved Euler (Heun's) method averages the slopes at the start and end of each step (2nd order). The celebrated Runge-Kutta 4 (RK4) method takes a weighted average of four slope evaluations (4th order).
Key insight: The order of accuracy determines how fast the error shrinks with step size. For an order-p method, halving h reduces the error by a factor of 2^p. RK4 (order 4) gains 16x accuracy per halving, making it the workhorse of scientific computing.
Compare the three panels: forward Euler with a large step blows up, forward Euler with a tiny step works, and backward Euler stays stable even with the large step.
Stiff equations have components that decay at vastly different rates. Explicit methods like forward Euler require tiny steps (h < 2/lambda) to avoid instability, while implicit methods like backward Euler remain stable at any step size.
A stiff equation contains dynamics at vastly different time scales. The ODE below has a fast transient that decays at rate 1000, while the solution tracks sin(t) which varies on a scale of order 1:
dy/dt = -1000(y - sin(t)) + cos(t)
Explicit methods like forward Euler must use extremely tiny steps to remain stable (h < 2/1000 = 0.002). Implicit methods like backward Euler solve an equation at each step and remain stable with much larger steps.
Key insight: Stiffness is not about the complexity of the solution -- it is about the stability constraint on step size. Stiff problems force explicit methods into impractically small steps. Implicit methods sacrifice some per-step simplicity for unconditional stability, making them essential for stiff systems in chemistry, circuit simulation, and computational biology.
Spot a mistake or have an idea?