---
title: "Decision Trees and Ensembles"
description: "A tree splits the data one question at a time, choosing each split to make the resulting groups purer. One tree is easy to read and unstable; averaging many trees grown on resampled data trades that r"
canonical: https://lightmysky.com/learn/computing/decision-trees-and-ensembles-mt_xo9EEd4R1i
source: https://lightmysky.com/learn/computing/decision-trees-and-ensembles-mt_xo9EEd4R1i.md
retrieved: 2026-09-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# Decision Trees and Ensembles

A tree splits the data one question at a time, choosing each split to make the resulting groups purer. One tree is easy to read and unstable; averaging many trees grown on resampled data trades that readability for accuracy.

Subject: Computing · Area: Machine Learning · Ages 20 to 22
Page: https://lightmysky.com/learn/computing/decision-trees-and-ensembles-mt_xo9EEd4R1i

## Ready when they can

- Choose the first split of a small dataset using an impurity measure
- Show that a small change in the data changes a single tree's structure
- Explain why averaging many trees reduces variance

## Lesson: One question at a time, then a forest vote

A decision tree sorts data by asking yes or no questions, one at a time. Imagine sorting fruit. First you ask, is it round. Then you ask, is it red. Each answer sends the fruit down a different branch until it lands in a final group, like apple or banana. To classify something new, walk it down the same questions to its group.

**Example.** The guide for good questions is entropy, a number that measures how mixed up a group is. A group with only apples has low entropy because it is pure. A group with half apples and half bananas has high entropy because it is mixed. The algorithm tries every possible question, measures the two groups each would create, and picks the one that lowers entropy the most. Then it repeats on each new group until the groups are pure enough or the questions run out.

One tree is easy to read, but it can be twitchy. Change a few rows of training data and the whole tree can grow a different shape and give different answers. That instability is the price of letting a single tree chase every detail of its training rows.

**Tip.** A random forest fixes the twitch with bootstrapping. Each tree trains on its own resample of the data, drawn at random with replacement, so every tree sees a slightly different dataset. For a prediction, every tree votes and the forest goes with the majority. Averaging many slightly different trees smooths out odd choices, trading readability for accuracy and steadiness.

**Recap.** Split by the question that purifies most, expect single trees to wobble, and steady them with a voting forest.

## Practice

8 questions on this page, each with its working shown.

## Needs first

- [Trees and Binary Search Trees](https://lightmysky.com/learn/computing/trees-and-binary-search-trees-mt_7PGHiAY54b)
- [Beyond Accuracy: Precision, Recall and the Cost of an Error](https://lightmysky.com/learn/computing/beyond-accuracy-precision-recall-and-the-cost-of-an-error-mt_BHGgZM7tak)
- [Overfitting and the Bias-Variance Trade-off](https://lightmysky.com/learn/computing/overfitting-and-the-bias-variance-trade-off-mt_xznyNxeNNw)

## Opens up

- [Clustering: Structure Without Labels](https://lightmysky.com/learn/computing/clustering-structure-without-labels-mt_BAMGOlWgRY)