Machine Learning & AI

Advanced

PCA (Covariance)

Finds directions of maximum variance via eigenvectors of the covariance matrix.

Formula

\mathbf{C} = \tfrac{1}{n}\mathbf{X}^\top\mathbf{X},\;\; \mathbf{C}\mathbf{v}=\lambda\mathbf{v}

Variables

\mathbf{C}Covariance matrix

\mathbf{v}Eigenvector

\lambdaEigenvalue

Example

Top eigenvectors are the principal components

Did You Know?

PCA compresses data by keeping only the directions that carry the most variance.

dimensionality reduction pca unsupervised

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Softmax

Turns a vector of scores into a probability distribution that sums to 1.