Fewer numbers, most of the spread

Computing · Machine Learning · ages 20-22

Name ______________________ Date ____________

You project a two-feature dataset onto its first component. What is kept for each point?
- Both original values unchanged
- Its position along the direction of biggest spread
- Its distance from the origin only
If every feature has variance exactly 1, PCA on the covariance matrix and on the correlation matrix give the same components.
Circle one: True False
A scree plot shows 62, 25, 8, 3 and 2 percent. How many components should you keep, and why?
- All five, since more numbers always help
- One, since only the first ever matters
- Two, since they hold most variation
For which pair of variables would it make sense to compute a correlation coefficient?
- Height and weight
- Shoe size and favorite color
- Province of residence and income bracket
- Hair color and eye color
You reduce ten correlated features to two components. What does each example become?
- Two numbers from the two components
- Ten numbers with the variation spread thin
- Two labels naming the original features
Two features have covariance 4, and their standard deviations are 2 and 4. What is their correlation? Round to two decimal places if needed.
Answer: ______________
You standardize every feature of a dataset so each one has mean 0 and standard deviation 1, then run PCA. A friend runs PCA on the raw covariance matrix of the same data without standardizing. Which statement is true?
- Both runs always give the exact same components
- They can differ: scale tilts raw covariance
- PCA on the correlation matrix ignores how the features vary together
- Standardizing the data deletes the correlations between features
Two features have covariance 4, with standard deviations 2 and 4. What is their correlation?
Answer: ______________
A component is 0.7 times height plus 0.7 times weight minus 0.1 times age. Why is it hard to name?
- The rows were shuffled at random
- It blends several features instead of matching one
- Height and weight can never mix
A friend keeps all five components since more numbers always help. Components hold 62, 25, 8, 3 and 2 percent. Best reply?
- Agree, extra tiny components always pay off
- Keep one, since only the first ever matters
- Keep two: they hold 87 percent

Answer key

For grown-ups. Fold this page away before handing over the rest.

Fewer numbers, most of the spread W1-mt_l-9XHg_ChZ-s1

Its position along the direction of biggest spread · Projection keeps one number per point: where it lands along that direction.
True · Correlation divides by deviations of 1, which changes nothing, so both matrices match.
Two, since they hold most variation · The first two sum to 87 percent, while 8, 3 and 2 add little.
Height and weight · Correlation measures how strongly two numeric variables move together along a straight line. It only makes sense when both variables are numbers.
Two numbers from the two components · Two kept directions give two numbers per example. They hold most of the spread only when their explained shares are large; correlation alone does not set the shares.
0.5 · The correlation is the covariance divided by the product of the standard deviations: 4 divided by 2 times 4, which is 0.5.
They can differ: scale tilts raw covariance · PCA chases directions of biggest spread. On raw covariance, a feature with a large scale swamps the others, so the components can tilt toward it. Standardizing gives every feature an equal say, which can change the answer.
0.5 · Divide covariance by both deviations: 4 divided by 8 is 0.5.
It blends several features instead of matching one · Blends of several original features resist clean labels in their language.
Keep two: they hold 87 percent · Stopping where gains turn tiny keeps most variation with far fewer numbers.