---
title: "Gradient Descent: Following the Slope Downhill"
description: "When no formula gives the best parameters, start somewhere and repeatedly step against the slope of the error. The step size decides whether the search creeps, oscillates or diverges, and this one pro"
canonical: https://lightmysky.com/learn/computing/gradient-descent-following-the-slope-downhill-mt_EjfIT1WgMX
source: https://lightmysky.com/learn/computing/gradient-descent-following-the-slope-downhill-mt_EjfIT1WgMX.md
retrieved: 2026-09-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# Gradient Descent: Following the Slope Downhill

When no formula gives the best parameters, start somewhere and repeatedly step against the slope of the error. The step size decides whether the search creeps, oscillates or diverges, and this one procedure trains almost everything that follows.

Subject: Computing · Area: Machine Learning · Ages 19 to 20
Page: https://lightmysky.com/learn/computing/gradient-descent-following-the-slope-downhill-mt_EjfIT1WgMX

## Ready when they can

- Run gradient descent by hand on a one-parameter error function for three steps
- Show what too large and too small a step size each do to the search
- Say why the method finds a local minimum and what that costs

## Lesson: Walking downhill to the local bottom

Some errors have no formula for the best parameters. You start somewhere and repeat one move: step against the slope of the error. Each step lowers the error a little.

The step size rules the search. A tiny size creeps and takes forever. A huge size overshoots, swings back and forth, or diverges. A balanced size walks down steadily.

**Example.** Take error w squared, start at w equal to 4, step size 0.1. The slope is 2w, so the first step is 4 minus 0.8, which lands at 3.2. The next steps give 2.56, then about 2.05.

**Tip.** The method stops at a local minimum, the bottom of the valley it walked into. It cannot promise the global best, and that is the price of the method.

**Recap.** Step against the slope with a sane size, and accept the local bottom you reach.

## Practice

8 questions on this page, each with its working shown.

## Needs first

- [Optimisation with Calculus](https://lightmysky.com/learn/mathematics/optimisation-with-calculus-mt_9I0ql39Lzw)
- [Directional Derivatives and the Gradient](https://lightmysky.com/learn/mathematics/directional-derivatives-and-the-gradient-mt_LW_KckY5Ad)
- [Linear Regression by Least Squares](https://lightmysky.com/learn/computing/linear-regression-by-least-squares-mt_QRRkpIrwWs)
- [The Derivative as a Rate of Change](https://lightmysky.com/learn/mathematics/the-derivative-as-a-rate-of-change-mt_RqyLE3jrAW)

## Opens up

- [Backpropagation: Assigning Blame for an Error](https://lightmysky.com/learn/computing/backpropagation-assigning-blame-for-an-error-mt_5bwaSWfF96)
- [Logistic Regression and the Decision Boundary](https://lightmysky.com/learn/computing/logistic-regression-and-the-decision-boundary-mt_FaVIaT7xl2)