Machine Learning & AI

Basic

Softmax

Turns a vector of scores into a probability distribution that sums to 1.

Formula

\sigma(z)_i = \dfrac{e^{z_i}}{\sum_{j} e^{z_j}}

Variables

z_iScore for class i

jAll classes

Example

z=[1,2,3]: softmax=[0.09,0.24,0.67]

Did You Know?

Softmax is the standard output layer for multi-class classifiers, from digit recognition to LLMs.

classification activation probability

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Binary Cross-Entropy

Loss for binary classification comparing predicted probability to the true label.