From Perceptron to Multilayer Network
A single artificial neuron is a linear score with a threshold, and it cannot separate patterns that are not linearly separable. Stacking layers with a non-linear function between them removes that limit, because the middle layers build features of their own.
What a learner can do afterwards
- Show that a single unit cannot represent exclusive-or
- Build a two-layer network by hand that does represent it
- Explain why a stack of layers without a non-linearity collapses to one layer
1 · Read
One artificial neuron takes signals, combines them into a score, and fires past an activation threshold. That draws one straight dividing line, so it can only split patterns a straight line can split.
Exclusive-or answers yes in two opposite corners and no in the other two. No single straight line can fence the yes corners together, so one neuron cannot learn it. Give each yes corner its own hidden unit, then let the output fire when either fires.
Stack two layers with a nonlinear step between them and the middle layer builds features of its own. The first layer spots edges, the next combines them into shapes. Without that nonlinear step, the whole stack collapses into one layer.
When someone claims one neuron learned exclusive-or, ask to see the line. Then ask what the hidden layer learned, since that is where the new features live.
One neuron draws one line, exclusive-or needs more, and hidden layers with a nonlinear step build the features that solve it.
2 · Watch
Take it off screen
Where it sits
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.