What makes a document win?
Repeat a word. Add irrelevant text. Change how rare a word is. Follow the arithmetic from TF-IDF to BM25.
Predict, then reveal
The documents
Three scoring rules
Choose your winner before you reveal the rankings.
Open the score
| Query term | tf | df / N | TF-IDF IDF | BM25 IDF | BM25 TF factor | Raw contribution | Cosine contribution | BM25 contribution |
|---|
Connect the demo to the Bayesian derivation
1. Rank by posterior odds
P(R = 1 | d, q) / P(R = 0 | d, q)
= prior odds × likelihood ratio.
For a fixed query and a document-independent prior, the prior factor does not affect the order.
2. Add term evidence
Conditional independence makes the likelihood ratio a product. Taking logs gives a sum. In the binary model, subtract the absent-term reference to get
Σ xₜ log[pₜ(1 − uₜ) / (uₜ(1 − pₜ))].
Here xₜ indicates presence, pₜ = P(xₜ = 1 | R = 1, q), and uₜ = P(xₜ = 1 | R = 0, q). To restrict the sum to query terms, assume pₜ = uₜ for terms outside q.
3. Approximate the evidence
With pₜ = ½ and the collection proxy uₜ = (dfₜ + ½)/(N + 1), the binary weight becomes log[(N − dfₜ + ½)/(dfₜ + ½)]. BM25 adds a saturating count approximation motivated by an eliteness model, plus an empirical length correction.
4. Inspect the choices
The demo uses positive IDF, log[1 + (N − dfₜ + ½)/(dfₜ + ½)]. It changes relative term weights. Neither this choice nor the BM25 rational TF factor follows from Bayes alone.
Set f′ = f/B with B = 1 − b + b dl/avgdl. Then (k₁ + 1)f′/(f′ + k₁) = (k₁ + 1)f/(f + k₁B).
Exact formulas and tokenisation
All logarithms are natural. Query terms are deduplicated, so each query term has binary TF. A document’s length is its number of tokens after stopword removal. DF and IDF are recalculated after each edit. Only the average length can be held fixed for a controlled what-if; uncheck that box to rebuild the full collection statistics.
Raw TF-IDF sum = Σₜ∈q fₜ,𝑑 log(N/dfₜ).
This is an unnormalised overlap score. It uses a binary query vector and TF-IDF document weights.
TF-IDF cosine = ⟨v(q), v(d)⟩ / (‖v(q)‖ ‖v(d)‖), with vₜ(d) = fₜ,𝑑 log(N/dfₜ) and vₜ(q) = 1[t ∈ q] log(N/dfₜ).
Cosine uses IDF on both sides. Its document norm includes the full vocabulary. If either vector has zero norm, the demo assigns score 0. Out-of-vocabulary query terms are omitted from the vector.
BM25 = Σₜ∈q log[1 + (N − dfₜ + ½)/(dfₜ + ½)] × (k₁ + 1)fₜ,𝑑 / [fₜ,𝑑 + k₁(1 − b + b dl/avgdl)].
An absent term contributes 0, including at k₁ = 0. These are ranking scores, not relevance probabilities.
Lowercase; retain ASCII letters and digits; no stemming. Remove these words: a an and are as by from in is it of or so that the to which with. “List” and “lists” remain distinct.
Reading: IIR, queries as vectors, IIR, BM25, and Robertson and Zaragoza (2009), The Probabilistic Relevance Framework.