Machine Learning & AI

Basic

Binary Cross-Entropy

Loss for binary classification comparing predicted probability to the true label.

Formula

L = -\big[y\log\hat{y} + (1-y)\log(1-\hat{y})\big]

Variables

yTrue label (0/1)

\hat{y}Predicted probability

Example

y=1, y_hat=0.9: L=-log0.9=0.105

Did You Know?

Cross-entropy comes straight from information theory — it measures surprise between distributions.

loss classification

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Softmax

Turns a vector of scores into a probability distribution that sums to 1.