Machine Learning & AI

Intermediate

L1 Regularization (Lasso)

Penalizes absolute weights, driving many to exactly zero for feature selection.

Formula

J = \text{MSE} + \lambda\sum_j |w_j|

Variables

\lambdaStrength

w_jWeights

Example

Lasso can zero out irrelevant features

Did You Know?

L1 produces sparse models — effectively doing automatic feature selection.

regularization sparsity feature selection

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Softmax

Turns a vector of scores into a probability distribution that sums to 1.