Why raw dense and BM25 scores should not be added directly

A dense-search score and a BM25 score are produced by different ranking models with different scales and distributions. Even when both systems return higher values for better matches, a value from one system does not necessarily have the same meaning as a value from the other.

Adding those values together can make ranking sensitive to score ranges rather than relevance. A query containing a rare identifier, for example, may receive a strong BM25 signal, while a broad natural-language query may benefit more from dense retrieval. Rank fusion avoids requiring a universal score scale before combining those signals.

  • Dense retrieval ranks semantic proximity in embedding space.
  • BM25 ranks term-based relevance using query and document terms.
  • Score magnitude can vary by query, corpus, and retrieval implementation.
  • Rank position is a simpler common representation across result lists.

Use reciprocal rank fusion as the combination layer

Run the same user query through the dense and sparse retrieval paths, then retain a bounded candidate list from each. For every document appearing in either list, calculate an RRF score by summing 1 divided by k plus the document’s rank for each list where it appears.

The constant k reduces the impact of small rank changes at the top of a list. A commonly used starting point is 60, but it is a configuration choice rather than a universal optimum. The important property is that documents ranked highly by either retrieval method receive useful credit, while documents supported by both rise naturally.

  • Dense list: retrieve the top N document IDs from S3 Vectors.
  • Sparse list: retrieve the top N document IDs from Quickwit BM25.
  • For each document d, use: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Sort documents by the combined RRF score and fetch the final records.

Keep the retrieval contract consistent and inspect disagreements

Fusion works best when both paths search the same logical document set and return stable document identifiers. Apply tenant, authorization, language, document-status, or time-window constraints consistently before fusion. A document that is eligible in only one path can otherwise create confusing results and weaken access-control assumptions.

Log the dense rank, sparse rank, fused rank, and query class for sampled searches. The most useful review cases are disagreements: results that rank highly in one list but not the other. These cases reveal whether a query needs better text normalization, improved chunking, metadata filters, or a different candidate depth.

  • Use one canonical document or chunk ID across dense and sparse indexes.
  • Apply equivalent filters to both retrieval paths before combining ranks.
  • Start with a modest, fixed candidate depth for each path and evaluate relevance.
  • Review exact-term queries and semantic paraphrases separately during testing.