Machine Learning & AI

Advanced

Batch Normalization

Normalizes layer inputs across a mini-batch to speed and stabilize training.

Formula

\hat{x} = \dfrac{x-\mu_B}{\sqrt{\sigma_B^2+\epsilon}}

Variables

\mu_BBatch mean

\sigma_B^2Batch variance

\epsilonStability constant

Example

Rescales activations to ~zero mean, unit variance

Did You Know?

Batch norm (2015) let networks train far faster and go much deeper.

deep learning normalization training

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Softmax

Turns a vector of scores into a probability distribution that sums to 1.