One question at a time, then a forest vote

Computing · Machine Learning · ages 20-22

Name ______________________ Date ____________

Which group has higher entropy?
- Only apples in a fully pure group
- Only bananas, fully pure
- Half apples and half bananas
How does a decision tree sort data?
- By asking one yes or no question at a time down branches
- By guessing the group straight away without asking anything
- By sorting numbers from large to small only
Changing a few training rows can make a single tree grow a different shape with different answers.
Circle one: True False
In a forest, how is each tree fed its training data?
- Give each tree all rows without allowing repeats
- Draw a resample at random with replacement
- Use one row per tree and discard the remaining training rows
Eight fruits: 5 apples and 3 bananas. A gives 4 apples alone, and 1 apple with 3 bananas. B gives 3 apples with 1 banana, and 2 apples with 2 bananas. Which split has lower size-weighted entropy? Use H=0 for pure, 0.811 for a 1:3 mix, and 1 for half and half.
- A: its weighted entropy is lower
- B: its weighted entropy is lower
- Neither: a mixed child rules out a split
For two-class classification with an odd number of trees, how does a random forest combine the trees' predictions?
- It always trusts the tallest tree in the forest
- It averages the question wordings
- Every tree votes and the majority wins
Which statement describes the usual trade-off between a random forest and one readable tree?
- Usually better accuracy and steadiness, less readability
- Guaranteed accuracy gains with no loss of readability or need for tree voting
- Clearer explanations for every prediction, but no change in steadiness
Rows x=10,20,30,40 have labels C,C,D,D. Change two labels to C,D,C,D. Grow a pure tree using midpoint thresholds and lowest weighted entropy; choose the smaller threshold on ties. Which structure replaces the old root x<25 with leaves C and D?
- Keep x<25: yes C, no D; both children remain pure
- Ask x<15, then on no x<25, then on no x<35; the leaves are C,D,C,D
- Ask x<15: yes C, no D; no further questions are needed

Answer key

For grown-ups. Fold this page away before handing over the rest.

One question at a time, then a forest vote W1-mt_xo9EEd4R1i-s1

Half apples and half bananas · Entropy measures how mixed up a group is, so the half and half group scores highest.
By asking one yes or no question at a time down branches · Each answer picks a branch until the item lands in a final group.
True · Single trees are twitchy: small data shifts can regrow the whole tree.
Draw a resample at random with replacement · Bootstrap sampling draws rows randomly with replacement. Separate draws usually produce different samples, but identical samples are possible.
A: its weighted entropy is lower · Each child contains 4 of the 8 fruits. A scores 0.5 times 0 + 0.5 times 0.811 = 0.4055. B scores 0.5 times 0.811 + 0.5 times 1 = 0.9055. A has the lower score.
Every tree votes and the majority wins · Each tree casts one vote. With two classes and an odd number of trees, one class receives more than half the votes and becomes the prediction. That majority prediction can still be incorrect.
Usually better accuracy and steadiness, less readability · Combining trees often improves accuracy and steadiness when their errors differ, but improvement is not guaranteed. Tracing many trees is harder than tracing one.
Ask x<15, then on no x<25, then on no x<35; the leaves are C,D,C,D · After the edits, x<25 leaves both children mixed. Root thresholds 15 and 35 tie, so use 15. Continuing to pure leaves requires questions at 25 and 35. The tree changes from one question to three.