Naive Bayes Posterior
Classifies by assuming features are conditionally independent given the class.
Formula
Variables
Example
Multiply prior by each feature likelihood
How it works
Naive Bayes estimates how plausible each class is after observing features by combining a class prior with the likelihood of each feature under that class. The key simplification is the "naive" assumption that, once the class is known, the features behave independently, so the joint likelihood can be written as a product of feature-wise likelihoods. This comes from Bayes’ rule, and although the independence assumption is often imperfect, it can still produce strong classifiers because the relative scores between classes are often well-separated.
When to use it
Use this when you need a fast probabilistic classifier for high-dimensional data, especially when features are counts, categories, or binary indicators. It is common in spam detection, text classification, sentiment analysis, and simple medical or fault-detection screening tasks.
Worked example, step by step
Suppose we want to classify an email as Spam (S) or Not Spam (N) using two binary features: contains the word "free" and contains the word "meeting". Given: P(S) = 0.30, P(N) = 0.70 P(free=1 | S) = 0.80, P(meeting=1 | S) = 0.10 P(free=1 | N) = 0.05, P(meeting=1 | N) = 0.60 Observed email: free=1, meeting=1 Compute unnormalized posterior scores: Score(S) = P(S)P(free=1|S)P(meeting=1|S) = 0.30 × 0.80 × 0.10 = 0.024 Score(N) = P(N)P(free=1|N)P(meeting=1|N) = 0.70 × 0.05 × 0.60 = 0.021 Normalize to get actual posteriors: P(S | x) = 0.024 / (0.024 + 0.021) = 0.024 / 0.045 = 0.5333 P(N | x) = 0.021 / (0.024 + 0.021) = 0.021 / 0.045 = 0.4667 Final answer: the email is classified as Spam because it has the higher posterior probability.
Common mistakes to avoid
- Treating the product as the final probability instead of normalizing across classes
- Forgetting that the independence assumption is conditional on the class, not unconditional
- Using zero feature likelihoods without smoothing, which can force an entire class score to zero
Did You Know?
Its "naive" independence assumption is usually false, yet it works remarkably well for spam filtering.
Share this formula
More in Machine Learning & AI
View allLinear Regression Model
BasicPredicts a continuous value as a weighted sum of input features plus a bias.
Gradient Descent Update
BasicIteratively moves parameters in the direction that most reduces the loss.
ReLU Activation
BasicRectified Linear Unit: outputs the input if positive, else zero.
Leaky ReLU
BasicA ReLU variant that lets a small gradient flow for negative inputs.
Tanh Activation
BasicSquashes input to the range (-1, 1); zero-centred activation.
Softmax
BasicTurns a vector of scores into a probability distribution that sums to 1.