Machine Learning & AI

Basic

Categorical Cross-Entropy

Multi-class loss summing the log-probability of the correct class.

Formula

L = -\sum_{i} y_i \log \hat{y}_i

Variables

y_iOne-hot true label

\hat{y}_iPredicted probability

Example

true=class2, p=0.7: L=-log0.7=0.357

Did You Know?

Paired with softmax, this loss has a beautifully simple gradient: predicted minus true.

loss classification

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Softmax

Turns a vector of scores into a probability distribution that sums to 1.