Maximum Likelihood Estimation
Write the probability of the observed data as a function of the unknown parameter and choose the parameter that makes the data most likely. Logs turn the product into a sum you can differentiate.
What a learner can do afterwards
- Write a likelihood for a sample from a stated model
- Maximise the log-likelihood and verify it is a maximum
- Derive the estimator for a binomial proportion or an exponential rate
1 · Read
Write the chance of the data you actually saw as a function of the unknown parameter. That function is the likelihood. For three trials landing success, success, failure, the likelihood is p squared times (1 minus p). The estimate is the parameter value that makes this function largest.
Products are hard to differentiate, so take the log first: it turns the product into a sum and keeps the same maximiser. Differentiate the log-likelihood, set the slope to zero, and solve. Then confirm a maximum with negative curvature, since zero slope alone only marks a stationary point.
With 12 successes in 30 trials, the log-likelihood is 12 log p plus 18 log (1 minus p), and solving gives p equal to 12 over 30, which is 0.4. For exponential lifetimes with sample mean 4 years, the maximiser is 1 over the mean, which is 0.25 per year. In each case the data's own pattern names the estimate.
The likelihood scores each parameter by how probable it makes the data, and the log turns maximising it into routine calculus.
2 · Watch
Take it off screen
Where it sits
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.