Backpropagation, one neuron at a time.
Training looks mysterious until you shrink the network down to one neuron. The forward pass makes a prediction. The loss says how wrong it was. Backpropagation is the bookkeeping that tells each weight how much of that wrongness belongs to it.
Below, the whole chain is visible: weighted sum, sigmoid, squared error, gradients, and the update rule. Press one step at a time and watch the prediction move toward the target.
Four small moves. One training step.
Backpropagation is easiest when it stops being one big formula. Move through the four steps below: predict, measure the error, send the error backward, then update the weights.
1. Forward pass
First compute z = (0.85 x 0.80) + (-0.55 x 0.35) + 0.10 = 0.588. Sigmoid turns that into 0.643.
The important trick is reuse. During the forward pass, the neuron saves intermediate values like z and sigmoid(z). During the backward pass, each local derivative reuses those values and passes one number of blame to the thing before it.
A deep network is the same idea repeated thousands of times. Each layer receives an upstream gradient, multiplies it by its local derivative, updates its own weights, and sends the result backward. Reverse-mode autodiff is backpropagation made systematic.
What this toy leaves out is batching, matrix multiplication, regularisation, and optimizers with memory. But the core shape is already here: compute the loss, follow the chain rule backward, and take a small step against the gradient.