This is a neural network with the lid off. It is the smallest network that can still learn something — sixteen inputs, two hidden neurons, two outputs — and it is kept that small on purpose: so small that every single multiply, every running sum, every gradient is a number you can read on screen. Where most visualizations show you the shape of learning, this one shows you the arithmetic.
Press Step and the timeline walks one operation at a time. A four-by-four card unfolds into sixteen input nodes; glowing packets carry each input along a wire and, above each, the exact multiply — input times weight equals product. Each hidden neuron tallies its sixteen products like a receipt, draws the line, and writes the total Z; ReLU either passes it or clamps it to zero and the neuron dims. The two outputs become probabilities, and the error is laid bare as a subtraction: prediction minus truth. Then the lighting reverses, red packets carry that error backwards, the chain rule splits it across the incoming wires, and each weight updates in the open: old weight plus gradient equals new weight, and the wire physically thickens or thins to match.
Run Training instead and the same engine sprints over the whole deck, and you watch the two hidden neurons specialize — their receptive fields sharpening into the features that separate a 1 from a 0. The math here is the genuine article, gradient-checked against finite differences. When you are ready to see the same arithmetic at full scale — thousands of weights reading real digits — step next door to the Neuron Lab; this room is the close-up you graduate from.
Specifications
- Topology
- 16 → 2 → 2, ReLU hidden, softmax output
- Training
- Real cross-entropy backprop, numerically ∇-checked
- Why two neurons
- Small enough that every wire and multiply stays visible
- Simulated
- Forward trace, gradient flow, live weight updates
More from Machine learning
