Data Science & ML
Intermediate

TF-IDF Weight

Term importance: frequency in a document scaled by rarity across documents.

Formula

wt,d=tft,d×log⁡Ndftw_{t,d} = tf_{t,d} \times \log\frac{N}{df_t}

Variables

tfTerm frequency
NTotal documents
df_tDocs containing term

Example

Rare but frequent-in-doc terms score highest

Did You Know?

TF-IDF was the backbone of search engines before neural embeddings.

nlptext