Perceptron Update
The original learning rule that nudges weights when a prediction is wrong.
Formula
Variables
Example
Correct prediction: no update
How it works
This rule updates a perceptron’s weight vector by moving it in the direction of the input vector when the model under-predicts, and in the opposite direction when it over-predicts. The factor (y - ŷ) is the error signal: if the prediction is correct, it is zero and nothing changes; if it is wrong, the update magnitude depends on how wrong the prediction was. This works because each adjustment shifts the decision boundary slightly toward classifying the current example correctly on the next pass.
When to use it
Use this in simple linear classification tasks, especially when training a basic perceptron or teaching online learning with label feedback one example at a time. It is most appropriate when you want a fast, interpretable update rule for linearly separable or approximately separable data.
Worked example, step by step
Suppose the current weights are w = [0.2, -0.5], the learning rate is η = 0.1, the input is x = [3, 1], the true label is y = 1, and the perceptron predicts ŷ = 0. Step 1: Compute the error term: y - ŷ = 1 - 0 = 1 Step 2: Multiply by the learning rate: η(y - ŷ) = 0.1 × 1 = 0.1 Step 3: Scale the input vector: 0.1 × x = 0.1 × [3, 1] = [0.3, 0.1] Step 4: Add to the current weights: w := [0.2, -0.5] + [0.3, 0.1] = [0.5, -0.4] Final answer: the updated weight vector is w = [0.5, -0.4].
Common mistakes to avoid
- Forgetting that no update happens when y = ŷ
- Using the wrong sign and subtracting the update when the prediction is incorrect
- Confusing the learning rate with the label or applying it to the weights instead of the error term
Did You Know?
Rosenblatt built the Perceptron in 1958 as physical hardware — the ancestor of neural networks.
Share this formula
More in Machine Learning & AI
View allLinear Regression Model
BasicPredicts a continuous value as a weighted sum of input features plus a bias.
Gradient Descent Update
BasicIteratively moves parameters in the direction that most reduces the loss.
ReLU Activation
BasicRectified Linear Unit: outputs the input if positive, else zero.
Leaky ReLU
BasicA ReLU variant that lets a small gradient flow for negative inputs.
Tanh Activation
BasicSquashes input to the range (-1, 1); zero-centred activation.
Softmax
BasicTurns a vector of scores into a probability distribution that sums to 1.