Data Science & ML

Machine learning metrics, loss functions, and statistical learning formulae.

This Data Science & ML formula category brings together the key metrics, loss functions, and statistical learning equations used to build, evaluate, and improve machine learning models. It’s designed to help students, exam-takers, and working data professionals quickly understand and apply essential formulas like MSE, precision, recall, F1 score, and activation functions, which are crucial for measuring performance and making better predictions in real-world projects.

Mean Squared Error (MSE)

MSE = \frac{1}{n}\sum_{i=1}^{n}(y_i - \hat{y}_i)^2

Average of squared differences between predicted and actual values.

Root Mean Squared Error (RMSE)

RMSE = \sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_i - \hat{y}_i)^2}

Square root of MSE, in the same units as the target variable.

Mean Absolute Error (MAE)

MAE = \frac{1}{n}\sum_{i=1}^{n}|y_i - \hat{y}_i|

Average of absolute differences between predictions and actual values.

Coefficient of Determination (R²)

R^2 = 1 - \frac{\sum(y_i-\hat{y}_i)^2}{\sum(y_i-\bar{y})^2}

Proportion of variance in the target explained by the model.

Classification Accuracy

Accuracy = \frac{TP + TN}{TP + TN + FP + FN}

Fraction of predictions that are correct.

Precision

Precision = \frac{TP}{TP + FP}

Of all positive predictions, the fraction that were correct.

Recall (Sensitivity)

Recall = \frac{TP}{TP + FN}

Of all actual positives, the fraction correctly identified.

F1 Score

F_1 = 2\cdot\frac{Precision \cdot Recall}{Precision + Recall}

Harmonic mean of precision and recall.

Specificity

Specificity = \frac{TN}{TN + FP}

Of all actual negatives, the fraction correctly identified.

Sigmoid Function

\sigma(x) = \frac{1}{1 + e^{-x}}

Maps any real number to the range (0, 1); used in logistic regression.

Softmax Function

\sigma(z_i) = \frac{e^{z_i}}{\sum_{j=1}^{K} e^{z_j}}

Converts a vector of scores into a probability distribution.

ReLU Activation

f(x) = \max(0, x)

Rectified Linear Unit: outputs the input if positive, else zero.

Binary Cross-Entropy Loss

L = -\frac{1}{n}\sum[y\log(\hat{y}) + (1-y)\log(1-\hat{y})]

Loss function for binary classification.

Gradient Descent Update

\theta = \theta - \alpha \nabla J(\theta)

Iterative optimization step to minimize a cost function.

Cosine Similarity

\cos\theta = \frac{A \cdot B}{\|A\|\|B\|}

Measures similarity between two vectors by the angle between them.

Euclidean Distance

d = \sqrt{\sum_{i=1}^{n}(a_i - b_i)^2}

Straight-line distance between two points in n-dimensional space.

Manhattan Distance

d = \sum_{i=1}^{n}|a_i - b_i|

Distance measured along axes at right angles (taxicab geometry).

Shannon Entropy

H = -\sum_{i=1}^{n} p_i \log_2 p_i

Measures the uncertainty or information content of a distribution.

Gini Impurity

Gini = 1 - \sum_{i=1}^{n} p_i^2

Probability of misclassifying a random element; used in decision trees.

Z-Score Normalization

z = \frac{x - \mu}{\sigma}

Standardizes features to zero mean and unit variance.

Min-Max Normalization

x' = \frac{x - x_{min}}{x_{max} - x_{min}}

Rescales features to the range [0, 1].

Bias-Variance Decomposition

Error = Bias^2 + Variance + \sigma^2

Total expected error decomposes into bias, variance, and irreducible noise.

Categorical Cross-Entropy

L = -\sum_{i=1}^{K} y_i \log(\hat{y}_i)

Loss for multi-class classification with one-hot labels.

TF-IDF Weight

w_{t,d} = tf_{t,d} \times \log\frac{N}{df_t}

Term importance: frequency in a document scaled by rarity across documents.