One gradient descent step on a curved loss functionThe loss is (w minus 2) squared. The minimum is at w equals 2. The starting weight is 0.5. A learning rate of 0.1 moves the weight to 0.8 and lowers the loss.LosswMinimum: w = 2Start: 0.5
Starting weight Updated weight

L(w) = (w − 2)2. At w = 0.5, the gradient is −3.

New w = 0.5 − 0.10 × (−3) = 0.80

Loss: 2.250 → 1.440

The step moves toward the minimum and lowers the loss.