Categorical Cross-Entropy

Categorical cross-entropy measures how well a predicted probability distribution matches the true class label when exactly one class is correct. With one-hot labels, every term except the correct class is zero, so the loss reduces to the negative log of the model’s assigned probability to the true class; this strongly penalizes confident wrong answers and rewards confident correct ones. It comes directly from the negative log-likelihood of a multinomial model, which is why it is the standard objective for probabilistic multi-class classifiers.

Use this for multi-class classification problems where each example belongs to one and only one class, such as digit recognition, topic classification, or image labeling. It is especially appropriate when your model outputs softmax probabilities and you want training to reflect calibrated confidence, not just hard class choices.

• Using probabilities that do not sum to 1, instead of softmax outputs
• Taking the log of the predicted class label rather than the predicted probability for the true class
• Forgetting that the loss is averaged or summed over a batch depending on the implementation