Machine Learning & AI

Intermediate

Gini Impurity

Probability of misclassifying a random sample; the default split metric in CART trees.

Formula

G = 1 - \sum_i p_i^2

Variables

p_iProportion of class i

Example

50/50 split: G = 0.5

Did You Know?

Gini impurity is faster to compute than entropy and usually gives near-identical trees.

decision tree impurity

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Softmax

Turns a vector of scores into a probability distribution that sums to 1.