What advantage do neural networks hold over plain token counting?

• They memorise every exact sequence in the training text
• They learn patterns that cover unseen sentences
• They skip training data entirely

Where does the supervision signal for next-token guessing come from?

• The text itself, which shows what comes next
• Human labels attached to every training sentence
• A teacher correcting each guess live

Low held-out loss guarantees that the model's answers are true.

Where does the free-supervision advantage stop?

• At user intent, which text patterns do not encode
• At rare tokens, which need extra human counting to learn
• At long books, which exceed memory

A model trained only to guess next tokens handles new grammar well. Why?

• Grammar rules were labelled by hand before training began
• The test sentences sat inside the training set
• Good compression of text keeps grammatical patterns

Two models post the same low held-out loss, but one answers users far better. What best explains the split?

• The worse model must have trained on far less text than the better one
• Loss measures guessing skill, while user fit needs something extra
• Loss values below a threshold stop meaning anything

A critic says next-token training cannot yield any task ability at all. Which fact answers them?

• One objective over enough text forces grammar, facts, and rough task sense along the way
• Models memorise every training sentence word for word, which is where task ability comes from
• Human labellers secretly grade each position

A team claims their model follows instructions because pretraining loss is low. What is the flaw?

• Loss is computed too slowly to matter
• Pretraining text never contains any instruction-style passages
• Low loss rewards likely tokens, not obedience to intent