← Math simulations  ·  EE263, Stanford

MMSE estimation: why the best guess is not the measurement

A scalar unknown with a known prior is measured once, with noise. Multiplying the prior by the likelihood gives the posterior, and its mean is the estimate that minimizes mean-squared error. That estimate is the measurement shrunk toward the prior mean by a factor that is the ratio of precisions. Reporting y itself is a worse guess, and the scatter below shows why.

y = x + w,   x ∼ N(0, τ2),   w ∼ N(0, σ2)   ⟶   x̂ = k y,   k = τ2/(τ2 + σ2)

The model

shrinkage factor k0.59
estimate x̂ = k y1.48
posterior std0.77
error of x̂ = k y  (rms)0.77
error of x̂ = y  (rms)1.00
ridge weight λ = σ2/τ20.69

Presets

Show

prior p(x): what you knew beforehand
likelihood p(y | x): what the sensor says
posterior p(x | y): the two combined
the MMSE estimate x̂

Below: 4000 independent trials. Each dot is one (measurement, truth) pair. Given a measurement you read off a guess from a line: the flatter orange line is closer to the dots than the dashed 45° line is.