Hinge Loss (SVM)

Hinge loss measures how well a classifier separates two classes using a signed score rather than a probability. If the true label y and the model score \u005chat{y} have the same sign and their product is at least 1, the point is not just classified correctly but is also beyond the margin, so the loss becomes zero. If the margin is too small or the point is misclassified, the loss grows linearly, which encourages SVMs to push the decision boundary away from the data for better separation.

Use hinge loss when training linear classifiers such as support vector machines or other margin-based models on binary labels coded as -1 and +1. It is especially useful when you want a large safety margin between classes rather than merely correct predictions.

• Using 0/1 labels instead of -1/+1 without converting the formula
• Treating \u005chat{y} like a probability instead of a raw decision score
• Forgetting that the loss is zero only when y\u005chat{y} \u2265 1, not merely when y\u005chat{y} > 0