Why raw dense and BM25 scores are difficult to combine

Dense retrieval and BM25 answer different matching questions. Dense search can surface documents with related meaning even when their wording differs from the query. BM25 emphasizes lexical evidence, which is especially useful when users include exact product names, identifiers, uncommon terms, or quoted phrases.

The scores returned by these methods should not automatically be treated as comparable numbers. Their ranges, distributions, and meanings depend on the underlying retrieval method, the query, and the collection. Adding them together without calibration can make one method dominate for reasons unrelated to relevance.

RRF avoids that comparison. It uses only each document's position in a ranked list, making it a straightforward baseline when dense and sparse retrieval are produced separately.

  • Dense retrieval is useful for semantic similarity.
  • BM25 is useful for exact and term-driven matches.
  • Rank positions are easier to combine than unrelated score scales.

Fuse two candidate lists with RRF

For a query, request a ranked candidate list from regional S3 Vectors and another from Quickwit BM25. Normalize document identifiers so that the same logical document can be recognized across both result sets. Then assign every returned document an RRF contribution from each list in which it appears.

The common RRF formula is: score(d) = Σ 1 / (k + rank_i(d)). Here, rank_i(d) is a document's one-based position in result list i, and k is a positive constant that reduces the advantage of a first-place result over other highly ranked results. Documents appearing in both lists receive contributions from both.

For example, if a document is ranked 2nd by dense retrieval and 5th by BM25, its fused score is 1 / (k + 2) + 1 / (k + 5). Sort documents by the fused score, then return the top results or pass them to a later reranking stage.

  • Retrieve a bounded candidate set from each retrieval method.
  • Use a shared document ID as the fusion key.
  • Treat ranks as one-based and apply the same k value to both lists.
  • Deduplicate before presenting the final ranked results.

Make fusion observable before making it complex

Start by logging which retriever contributed each final result, its rank in each source list, and its final fused rank. These fields make it possible to inspect whether sparse retrieval is rescuing exact-match queries, whether dense retrieval is broadening semantic recall, and whether one source rarely contributes to the final page.

Evaluate with representative queries rather than a single aggregate impression. Include exact-name searches, short ambiguous queries, natural-language questions, acronym-heavy queries, and queries that mix a product term with descriptive language. Review relevance at the result level and note which retrieval path found each useful document.

RRF is a practical baseline, not a declaration that every query needs equal dense and sparse influence. Once observed query behavior supports a change, teams can test candidate depths, the fusion constant, or query-aware routing while keeping the simple fused ranking as a reference point.

  • Log source ranks and final fused ranks for every returned document.
  • Build evaluation queries that represent real search behavior.
  • Change one fusion parameter at a time.
  • Keep a simple baseline for comparison during iteration.