Linear Regression Model
Predicts a continuous value as a weighted sum of input features plus a bias.
Formula
Variables
Example
Weights [2,3], x=[1,4], b=1: y = 2+12+1 = 15
How it works
Linear regression models a target as a straight-line combination of input features, where each feature contributes according to its weight. The dot product \(\mathbf{w}^\top \mathbf{x}\) adds up all these weighted contributions, and the bias \(b\) shifts the prediction up or down to fit data that does not pass through zero. It works well when the relationship between inputs and the output is approximately linear or can be approximated that way.
When to use it
Use linear regression when you need a numeric prediction, such as estimating house prices, sales, temperature, or exam scores from measured features. It is a good first model when you want something simple, fast, and interpretable.
Worked example, step by step
Weights: \(\mathbf{w} = [4, -1, 2]\) Features: \(\mathbf{x} = [3, 5, 2]\) Bias: \(b = 6\) Step 1: Multiply each weight by its matching feature \(4\cdot 3 = 12\) \(-1\cdot 5 = -5\) \(2\cdot 2 = 4\) Step 2: Add the weighted values \(12 + (-5) + 4 = 11\) Step 3: Add the bias \(11 + 6 = 17\) Final prediction: \(\hat{y} = 17\)
Common mistakes to avoid
- Forgetting to add the bias term \(b\)
- Mixing up the order of features and weights
- Using the formula for classification instead of a continuous prediction
Did You Know?
Linear regression dates to Gauss and Legendre around 1800 — the oldest algorithm still core to modern ML.
Share this formula
More in Machine Learning & AI
View allGradient Descent Update
BasicIteratively moves parameters in the direction that most reduces the loss.
ReLU Activation
BasicRectified Linear Unit: outputs the input if positive, else zero.
Leaky ReLU
BasicA ReLU variant that lets a small gradient flow for negative inputs.
Tanh Activation
BasicSquashes input to the range (-1, 1); zero-centred activation.
Softmax
BasicTurns a vector of scores into a probability distribution that sums to 1.
Binary Cross-Entropy
BasicLoss for binary classification comparing predicted probability to the true label.