Binary Cross-Entropy
Loss for binary classification comparing predicted probability to the true label.
Formula
Variables
Example
y=1, y_hat=0.9: L=-log0.9=0.105
How it works
Binary cross-entropy measures how well a model’s predicted probability matches the actual binary outcome. It penalizes confident wrong predictions very heavily: if the true label is 1, the loss gets small only when the predicted probability is close to 1, and if the true label is 0, the loss gets small only when the predicted probability is close to 0. This comes from the negative log-likelihood for a Bernoulli outcome, so it is the natural loss when modeling probabilities for two-class problems.
When to use it
Use this loss for binary classification tasks where the model outputs a probability, such as spam detection, disease screening, or click/no-click prediction. It is especially appropriate when you want the model to be calibrated, not just correct on the final 0/1 label.
Worked example, step by step
Given y = 0 and \hat{y} = 0.2 Step 1: Substitute into the formula: L = -[y log(\hat{y}) + (1-y) log(1-\hat{y})] Step 2: Plug in the values: L = -[0·log(0.2) + 1·log(1-0.2)] Step 3: Simplify: L = -[log(0.8)] Step 4: Calculate: L = -(-0.2231) Step 5: Final answer: L ≈ 0.223
Common mistakes to avoid
- Using 0/1 labels inside the log instead of the predicted probability
- Forgetting the minus sign, which would make better predictions look worse
- Using raw model outputs instead of probabilities between 0 and 1
Did You Know?
Cross-entropy comes straight from information theory — it measures surprise between distributions.
Share this formula
More in Machine Learning & AI
View allLinear Regression Model
BasicPredicts a continuous value as a weighted sum of input features plus a bias.
Gradient Descent Update
BasicIteratively moves parameters in the direction that most reduces the loss.
ReLU Activation
BasicRectified Linear Unit: outputs the input if positive, else zero.
Leaky ReLU
BasicA ReLU variant that lets a small gradient flow for negative inputs.
Tanh Activation
BasicSquashes input to the range (-1, 1); zero-centred activation.
Softmax
BasicTurns a vector of scores into a probability distribution that sums to 1.