Small updates on big weights

Computing · Machine Learning · ages 22-24

Name ______________________ Date ____________

What moves in a full fine-tune?
- Every parameter, costing a whole copy to store
- Only the input layer, leaving every other layer frozen
- Nothing; the base weights train themselves
A rank r update to an n by n matrix stores 2 times n times r numbers, cheaper than n squared whenever r sits below half of n.
Circle one: True False
What is catastrophic forgetting?
- Forgetting to save the trained adapter after the run finishes
- New training washing out skills the model once held
- The base weights freezing by accident
A question needs answering once, next week, and the budget is thin. Training a low-rank adapter for it is the right call.
Circle one: True False
A task repeats daily and prompting keeps missing. The budget covers a small adapter but not a full run. What fits?
- Keep prompting, since training never repays a daily task
- Pay for a full fine-tune anyway
- Train a low-rank adapter and watch old-task checks
A 200 by 200 layer gets a rank 10 adapter, stored as B with shape 200 by 10 and A with shape 10 by 200. How many numbers does the adapter store in total?
Answer: ______________
A matrix with rows [1, 3] and [x, 9] has rank 1, so the second row is a multiple of the first. What is x?
Answer: ______________
A matrix with rows [2, 4] and [3, x] has rank 1, so the second row is a multiple of the first. What is x?
Answer: ______________
Old-task scores start slipping midway through adapting. What is the right response?
- Train longer, since old scores always dip before recovering
- Stop or lighten training: forgetting has started
- Delete the old-task checks as outdated
A team adapts a translation model to legal text and keeps an old-task check running on general translation. That score was 0.90 at the start and reads 0.70 midway through training. What should they do?
- Keep training at the same pace, since only the new legal score matters from now on
- Stop or lighten the training, because the old skill is being washed out
- Train for longer so the new task fully overrides the old one
- Raise the rank of the adapter so it can hold both skills

Answer key

For grown-ups. Fold this page away before handing over the rest.

Small updates on big weights W1-mt_NaA2iGtMPc-s1

Every parameter, costing a whole copy to store · Full means full: all weights move and all must be kept.
True · Two thin factors hold n r plus r n against the full square.
New training washing out skills the model once held · Moving everything can overwrite what earlier training built.
False · A one-off need with a thin budget is the case for prompting, since no training runs at all. Adapting only repays itself when the task repeats.
Train a low-rank adapter and watch old-task checks · Repeating failure repays training, and the thin update fits the budget.
4000 · B holds 200 times 10, which is 2000 numbers, and A holds 10 times 200, another 2000, so the adapter stores 4000 numbers against 40000 for the full update.
3 · The second column shows the multiple: 9 is 3 times 3. Applying that multiple to the first column gives x = 3 times 1 = 3.
6 · Three is 1.5 times 2, so x is 1.5 times 4.
Stop or lighten training: forgetting has started · Slipping old scores are the alarm; the checks exist to ring it.
Stop or lighten the training, because the old skill is being washed out · A slipping old-task score is catastrophic forgetting showing up on the check that was meant to catch it. The lesson's response is to stop or train less and freeze more.