Deep Learning: Convolution, Sequence and Scale
Deep networks work because their architecture matches the data: convolution shares weights across positions in an image, and sequence models carry information along a sentence. What changed in practice was scale, in data and in the hardware that could process it.
What a learner can do afterwards
- Say what weight sharing in a convolution assumes about the input
- Explain why a sequence model needs some memory of what came earlier
- Name what a deep model needs that a small one does not, beyond layers
1 · Read
Deep networks win by matching the data. A convolution slides one small filter across the image and reuses the same weights at every spot. That reuse is called weight sharing, and it assumes a pattern means the same thing anywhere.
One edge filter sweeps a photo and paints a feature map of everywhere edges appear. Early layers find edges and corners, deeper layers combine them into shapes and objects. The network learns faster because the filter is reused, not relearned per corner.
A sequence model reads a sentence in order and carries memory of earlier words along. Without that memory, each word would stand alone and order would mean nothing. What changed in practice was scale: far more data plus hardware with many cores to process it.
A deep model needs more than layers: lots of examples, compute to train on, and an architecture that fits the data. Give images convolution and give sentences memory.
Share weights across places, carry memory along sequences, and feed both with data and compute at scale.
2 · Watch
Take it off screen
Where it sits
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.