Logistic Regression
Applies the sigmoid to a linear model to output a class probability.
Formula
Variables
Example
linear score 0: probability = 0.5
How it works
Logistic regression turns a weighted sum of input features into a value between 0 and 1 by passing it through the sigmoid function. That output is interpreted as the probability of the positive class, so large positive scores give probabilities near 1 and large negative scores give probabilities near 0. It works well because the linear part can learn how features combine, while the sigmoid converts that score into a smooth, probability-like prediction.
When to use it
Use logistic regression when you need to predict one of two classes and want a probability, such as spam vs. not spam, fraud vs. not fraud, or disease vs. no disease. It is especially useful when the relationship between features and the log-odds of the outcome is roughly linear.
Worked example, step by step
Suppose w = [2, -1], x = [1.5, 0.5], and b = -0.5 Step 1: Compute the linear score: w^T x + b = (2)(1.5) + (-1)(0.5) + (-0.5) Step 2: Calculate: 3 - 0.5 - 0.5 = 2.0 Step 3: Apply the sigmoid: y_hat = 1 / (1 + e^(-2.0)) Step 4: Evaluate: y_hat ≈ 1 / (1 + 0.1353) ≈ 0.881 Final answer: the model predicts a probability of about 0.881, or 88.1%, for the positive class.
Common mistakes to avoid
- Treating the output as a hard class label instead of a probability
- Forgetting to include the bias term b in the linear score
- Using logistic regression for a multi-class problem without a suitable extension such as one-vs-rest or softmax
Did You Know?
Despite the name, logistic regression is a classification algorithm, not a regression one.
Share this formula
More in Machine Learning & AI
View allLinear Regression Model
BasicPredicts a continuous value as a weighted sum of input features plus a bias.
Gradient Descent Update
BasicIteratively moves parameters in the direction that most reduces the loss.
ReLU Activation
BasicRectified Linear Unit: outputs the input if positive, else zero.
Leaky ReLU
BasicA ReLU variant that lets a small gradient flow for negative inputs.
Tanh Activation
BasicSquashes input to the range (-1, 1); zero-centred activation.
Softmax
BasicTurns a vector of scores into a probability distribution that sums to 1.