TF-IDF
Weights a word by its frequency in a document against its rarity across the corpus.
Formula
Variables
Example
Common words get low weight, rare terms high
How it works
TF-IDF is a way to score how informative a word is in a particular document. The term-frequency part rewards words that appear often in that document, while the inverse-document-frequency part downweights words that appear in many documents and therefore carry less distinguishing power. This combination works because a word is useful for identifying a document only when it is both prominent in that document and relatively uncommon across the corpus.
When to use it
Use TF-IDF when you need to rank keywords, compare documents, or build simple text features for search, classification, or clustering. It is especially useful in domains like information retrieval, topic analysis, and document similarity where exact word meaning is less important than term distinctiveness.
Worked example, step by step
Suppose a corpus has N = 2,000 documents. A document contains the term "quasar" 8 times, so tf = 8. The term "quasar" appears in df = 20 documents across the corpus. Compute the inverse document frequency: log(N/df) = log(2000/20) = log(100). Using natural log, log(100) ≈ 4.605. Now multiply by term frequency: tfidf = 8 × 4.605 ≈ 36.84. Final answer: TF-IDF for "quasar" in this document is about 36.84.
Common mistakes to avoid
- Using df as the number of times a word appears in one document instead of the number of documents containing it
- Forgetting that the log base matters for the scale, even though different bases preserve ranking
- Assuming a high TF-IDF word is always globally important rather than just distinctive for that corpus
Did You Know?
TF-IDF was the backbone of search engines and text mining for decades before embeddings.
Share this formula
More in Machine Learning & AI
View allLinear Regression Model
BasicPredicts a continuous value as a weighted sum of input features plus a bias.
Gradient Descent Update
BasicIteratively moves parameters in the direction that most reduces the loss.
ReLU Activation
BasicRectified Linear Unit: outputs the input if positive, else zero.
Leaky ReLU
BasicA ReLU variant that lets a small gradient flow for negative inputs.
Tanh Activation
BasicSquashes input to the range (-1, 1); zero-centred activation.
Softmax
BasicTurns a vector of scores into a probability distribution that sums to 1.