Gini Impurity
Probability of misclassifying a random element; used in decision trees.
Formula
Variables
Example
Pure node (p=1): Gini = 0
How it works
Gini impurity measures how mixed the classes are in a node of a decision tree. It is built from the class proportions, and the subtraction from 1 leaves a value near 0 when one class dominates and larger when the classes are more evenly split, reflecting a higher chance of picking the wrong class at random.
When to use it
Use Gini impurity when training or interpreting classification trees, especially in algorithms like CART. It is also useful anytime you want a quick numerical measure of class mix within a subset of labeled data.
Worked example, step by step
Suppose a node contains 3 classes with proportions p1 = 0.5, p2 = 0.3, and p3 = 0.2. Gini = 1 - (p1^2 + p2^2 + p3^2) Gini = 1 - (0.5^2 + 0.3^2 + 0.2^2) Gini = 1 - (0.25 + 0.09 + 0.04) Gini = 1 - 0.38 Gini = 0.62 So the node has a Gini impurity of 0.62, which indicates a fairly mixed class distribution.
Common mistakes to avoid
- Forgetting to square each class proportion before summing
- Using counts directly instead of converting them to proportions
- Assuming a higher Gini value means a better split, when it actually means more impurity
Did You Know?
CART uses Gini impurity while ID3 uses entropy.
Share this formula
More in Data Science & ML
View allMean Squared Error (MSE)
BasicAverage of squared differences between predicted and actual values.
Root Mean Squared Error (RMSE)
BasicSquare root of MSE, in the same units as the target variable.
Mean Absolute Error (MAE)
BasicAverage of absolute differences between predictions and actual values.
Coefficient of Determination (R²)
IntermediateProportion of variance in the target explained by the model.
Classification Accuracy
BasicFraction of predictions that are correct.
Precision
IntermediateOf all positive predictions, the fraction that were correct.