---
title: "Likelihood and the Maximum Likelihood Estimate"
description: "Likelihood scores parameter values by the probability or density they assign to observed data. A maximum likelihood estimate chooses a value with the highest score. Ordinary least squares is a maximum"
canonical: https://lightmysky.com/learn/computing/likelihood-and-the-maximum-likelihood-estimate-mt_crcBnSqkTE
source: https://lightmysky.com/learn/computing/likelihood-and-the-maximum-likelihood-estimate-mt_crcBnSqkTE.md
retrieved: 2026-11-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# Likelihood and the Maximum Likelihood Estimate

Likelihood scores parameter values by the probability or density they assign to observed data. A maximum likelihood estimate chooses a value with the highest score. Ordinary least squares is a maximum likelihood fit under independent, mean-zero normal errors with a common fixed variance.

Subject: Computing · Area: Machine Learning · Ages 18 to 20
Page: https://lightmysky.com/learn/computing/likelihood-and-the-maximum-likelihood-estimate-mt_crcBnSqkTE

## Ready when they can

- Write the likelihood of a run of coin flips as a function of the bias
- Maximise a simple likelihood and check the answer against intuition
- Explain why the likelihood is a function of the parameter, not of the data

## Lesson: Picking the bias the data likes best

Let p be the chance of heads on each flip. Assume independent flips with the same p. Once a sequence is observed, keep it fixed and vary p. Its probability, viewed as a function of p, is the likelihood L(p). A maximum likelihood estimate chooses a p with the highest L(p). Fixing p and varying possible outcomes is also valid, but answers a prediction question instead.

For one observed sequence with h heads and t tails, L(p) = p^h × (1-p)^t. Each head contributes p and each tail contributes 1-p. Independence lets us multiply. If only the head count is recorded, multiply by the number of arrangements with that count. This factor is the same for every p, so it does not change which p maximizes likelihood.

**Example.** For n independent flips with a common bias and n > 0, the maximizing bias is h/n. With both outcomes present, likelihood rises up to h/n and falls after it. All heads give p = 1; all tails give p = 0. For six heads in ten flips, the estimate is 6/10 = 0.6. As a check, the expected head count is n × p, giving 10 × 0.6 = 6.

*(drawing: For six heads and four tails, 0.6 scores highest among these candidates. The h/n rule locates the maximum over all allowed biases.)*

For a line model, a residual is an observed value minus its prediction. Suppose residual errors are independent and normal, meaning they follow a symmetric bell-shaped distribution. Give them mean zero and a common fixed variance, which measures their spread. Under this model, maximizing likelihood over the line's coefficients minimizes the sum of squared residuals. For continuous data, likelihood uses density rather than the probability of an exact value.

A maximum likelihood estimate is the best-scoring parameter within the chosen model. It does not prove that parameter is the true value. A likelihood is not a probability distribution over parameter values. Another sample can give another estimate, and changing the model can change the fit.

**Recap.** Keep observed data fixed and compare parameters. For independent flips with a common bias, the estimate is heads divided by flips. A best fit does not prove the true bias.

*Lesson, question and flashcard revisions written for this dataset.*

## Practice

8 questions on this page, each with its working shown.

## Optional connections

- [Optimisation with Calculus](https://lightmysky.com/learn/mathematics/optimisation-with-calculus-mt_9I0ql39Lzw)
- [The Binomial Distribution](https://lightmysky.com/learn/mathematics/the-binomial-distribution-mt_X2kVGK67Md)

## Opens up

- [Bayes' Rule and Updating a Belief](https://lightmysky.com/learn/computing/bayes-rule-and-updating-a-belief-mt_A4p5--f7Io)
- [Logistic Regression and the Decision Boundary](https://lightmysky.com/learn/computing/logistic-regression-and-the-decision-boundary-mt_FaVIaT7xl2)
- [Models of Sequence Evolution and Maximum Likelihood Trees](https://lightmysky.com/learn/science/models-of-sequence-evolution-and-maximum-likelihood-trees-mt_Ujtg0xul9N)