Previous: Digit Lab Next: 4×4 Drivetrain

The White-Box — A Neural Net With Nothing to Hide

Machine learning · c. 1986

About

The White-Box — A Neural Net With Nothing to Hide

Preparing exhibit…

Operate

Speed

TRAINING SET

1 = has a vertical stroke · 0 = does not

How it works

The training set contains synthetic 4×4 images. Class 1 has a vertical stroke; class 0 does not. The network is trained to distinguish these two classes.
Each image supplies 16 pixel-intensity values to the input layer. Sphere brightness represents the corresponding numerical value.
Each line represents a learned weight: blue is positive, orange is negative, and thickness represents magnitude. The weights begin from a deterministic random initialization.
During the forward pass, each hidden unit forms a weighted sum of all 16 inputs and a bias. ReLU then computes max(0, z). The inspector lists every term in these two hidden-unit sums.
The two output logits pass through softmax to produce class probabilities. Cross-entropy loss is −log p(y), where y is the labeled class.
Backpropagation applies the chain rule to calculate the derivative of the loss with respect to each weight and bias. A gradient is a local sensitivity, not a causal share of the error.
Gradient descent updates each parameter in the direction that reduces the current example's loss: θ ← θ − η∇θ. Line thickness changes as the weights are updated.
Run Training repeats this loop over the full data set. Watch the loss curve fall and the weight heat maps organize as a vertical-stroke detector emerges from the arithmetic.

Inspector

Run state

mode—card—class—calculation step—weight updates—epochs—last epoch loss—

positive weight
negative weight

thickness = strength

.museum-body--immersive { overflow: auto !important; } .museum-main--fill, .museum-runtime { height: auto !important; min-height: 100vh; overflow: visible !important; } .museum-runtime > .exhibit-root, .museum-runtime > :not(#mw-about):not(noscript):not(script) { display: none !important; } .museum-runtime #mw-about { display: block !important; position: static; width: min(100%, 800px); margin: 24px auto; padding: 24px; box-sizing: border-box; } .museum-runtime [data-about-close], .museum-runtime #mw-about > h2 { display: none !important; }

c. 1986 · Machine learning

The White-Box — A Neural Net With Nothing to Hide

A 16–2–2 classifier that exposes its weighted sums, ReLU activations, probabilities, loss, and gradients.

This deliberately small network classifies synthetic 4×4 patterns using sixteen inputs, two ReLU hidden units, and two softmax outputs. Its scale allows the exhibit to display each weighted-sum term and parameter update; it is not a claim about the smallest network capable of learning.

Step mode traces a forward pass from pixel values through weighted sums, biases, ReLU, output logits, and softmax probabilities. Cross-entropy supplies the loss. The backward trace applies the chain rule to compute local parameter sensitivities; gradient values should not be read as causal shares of an error.

Run Training repeats the calculation across the synthetic data set and updates parameters using gradient descent. Weight heat maps can reveal a vertical-stroke detector as the classifier adapts. Finite-difference checks provide a numerical implementation diagnostic, while outcomes remain specific to this small model and data set.

Specifications

Topology: 16 → 2 → 2, ReLU hidden, softmax output
Training: Cross-entropy, backpropagation, gradient descent, numerical check
Why two neurons: Small enough that every wire and multiply stays visible
Displayed: Forward trace, gradients, and parameter updates

All exhibits

Operate

TRAINING SET

In the current issue