Gini Impurity
Probability of misclassifying a random sample; the default split metric in CART trees.
Formula
Variables
Example
50/50 split: G = 0.5
How it works
Gini impurity measures how mixed the classes are in a node: it is the probability that two items drawn at random from that node would belong to different classes, written in a compact form as 1 minus the sum of squared class proportions. Squaring the proportions makes pure nodes score 0 and penalizes mixtures more when the classes are spread out, so the value rises as the node becomes less certain. In decision trees, lower impurity means a split has created groups that are easier to classify correctly.
When to use it
Use Gini impurity when evaluating splits in classification trees, especially CART-style decision trees. It is also useful any time you want a quick measure of class mixture in a bucket, node, or segment of categorical data.
Worked example, step by step
Suppose a node contains 3 classes with proportions p1 = 0.6, p2 = 0.3, p3 = 0.1. Step 1: Square each proportion: 0.6^2 = 0.36, 0.3^2 = 0.09, 0.1^2 = 0.01 Step 2: Add the squared values: 0.36 + 0.09 + 0.01 = 0.46 Step 3: Subtract from 1: G = 1 - 0.46 = 0.54 Final answer: Gini impurity = 0.54
Common mistakes to avoid
- Forgetting that the class proportions must add up to 1 before applying the formula
- Using the class counts directly instead of converting them to proportions
- Thinking a lower Gini value means the majority class is smaller, when it actually means the node is more pure
Did You Know?
Gini impurity is faster to compute than entropy and usually gives near-identical trees.
Share this formula
More in Machine Learning & AI
View allLinear Regression Model
BasicPredicts a continuous value as a weighted sum of input features plus a bias.
Gradient Descent Update
BasicIteratively moves parameters in the direction that most reduces the loss.
ReLU Activation
BasicRectified Linear Unit: outputs the input if positive, else zero.
Leaky ReLU
BasicA ReLU variant that lets a small gradient flow for negative inputs.
Tanh Activation
BasicSquashes input to the range (-1, 1); zero-centred activation.
Softmax
BasicTurns a vector of scores into a probability distribution that sums to 1.