Maximum Likelihood Estimation
Write the probability of the observed data as a function of the unknown parameter and choose the parameter that makes the data most likely. Logs turn the product into a sum you can differentiate.
What a learner can do afterwards
- Write a likelihood for a sample from a stated model
- Maximise the log-likelihood and verify it is a maximum
- Derive the estimator for a binomial proportion or an exponential rate
1 · Read
Write the chance of the data you actually saw as a function of the unknown parameter. That function is the likelihood. For two trials landing success, failure, the likelihood is p times (1 minus p). The estimate is the parameter value that makes this function largest, and like any estimator you can judge it by bias and variance.
Products are hard to differentiate, so take the log first: it turns the product into a sum and keeps the same maximiser. Differentiate the log-likelihood, set the slope to zero, and solve. Then confirm a maximum with negative curvature, since zero slope alone only marks a stationary point.
With 7 successes in 20 trials, the log-likelihood is 7 log p plus 13 log (1 minus p), and solving gives p equal to 7 over 20, which is 0.35. For exponential lifetimes with sample mean 5 years, the maximiser is 1 over the mean, which is 0.2 per year. In each case the data's own pattern names the estimate.
The likelihood scores each parameter by how probable it makes the data, and the log turns maximising it into routine calculus.
The rewritten readings were written for this dataset.
More to read
Optimising a likelihood
Maximum likelihood estimation seeks parameters that maximise an objective, often the log-likelihood. When that function is differentiable, find candidate maxima by solving its derivative equations. Check boundaries and, where applicable, second derivatives before confirming a maximum. Optimisation techniques transfer across models, although different functions may require different methods.
Differentiating the log-likelihood
Bernoulli likelihoods contain powers, while exponential-model likelihoods involve exponentials. Logarithms turn products of observation contributions into sums, making differentiation easier. Setting the derivative, called the score, to zero gives an equation for an interior estimate. Derivative rules for logarithmic and exponential functions support these MLE calculations.
Maximum likelihood estimators
An interactive likelihood explorer lets you drag the parameter and watch the curve rise to its peak at the estimate. Moving the slider builds the central intuition faster than algebra: the MLE is simply the value that makes the observed data most probable. Trying different samples shows the peak wandering around the truth, previewing sampling variation. Play first for instinct, then derive with calculus for proof.
Explore more
2 · Watch
Take it off screen
Where it sits
Optional connections
This opens up
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
12 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.