Why raw dense and BM25 scores should not be added directly

A dense-search score and a BM25 score are produced by different models and different scoring systems. Their numeric ranges, distributions, and meanings can change with the index, query, corpus, embedding model, and BM25 configuration. A value that looks larger in one system is not automatically stronger evidence of relevance than a value from the other.

Direct score addition can therefore create fragile ranking behavior. A tuning choice that appears reasonable for one collection may overemphasize lexical matches in another, or suppress them entirely. This is particularly noticeable for mixed query traffic: natural-language questions often benefit from semantic retrieval, while product codes, error messages, names, and quoted phrases often depend on sparse matching.

  • Dense retrieval is useful for semantic similarity and paraphrases.
  • BM25 is useful for exact vocabulary, rare tokens, and identifiers.
  • Raw scores from the two systems are not inherently comparable.
  • Rank-based fusion avoids needing a shared score scale.

Fuse two ranked lists with Reciprocal Rank Fusion

RRF starts with two ranked candidate lists for the same query: one from dense search and one from BM25. For each document, add a contribution from every list in which it appears. The contribution is 1 divided by a constant plus the document’s rank in that list. The document’s final fused score is the sum of those contributions.

Using ranks makes the method straightforward to operate. A document that ranks well in both lists rises naturally, while a document that is highly ranked in only one list can still be retained. The constant, commonly written as k, reduces the gap between adjacent top positions and prevents the first few ranks from dominating too aggressively.

  • Request a bounded candidate set from S3 Vectors for dense retrieval.
  • Request a bounded candidate set from Quickwit BM25 for sparse retrieval.
  • Deduplicate candidates by a stable document identifier.
  • Compute: RRF(document) = Σ 1 / (k + rank).

Make fusion observable before making it complex

Start with the same candidate depth for both retrieval paths and a fixed k value. Log the source ranks for every fused result, not just its final position. Those logs reveal whether useful results come from agreement between dense and sparse retrieval or whether one source consistently provides the decisive candidate.

Evaluation should reflect the queries users actually send. Include semantic questions, exact-title searches, identifiers, typo-prone terms, and multi-concept requests. When reviewing failures, distinguish candidate-generation problems from ranking problems: if a relevant document appears in neither initial list, fusion cannot recover it. If it appears in one list but too low in the final ordering, adjust candidate depth or fusion policy before changing the underlying indexes.

  • Record dense rank, BM25 rank, and fused rank per result.
  • Track whether judged-relevant documents enter either candidate list.
  • Use stable document IDs so the same item can be merged reliably.
  • Treat candidate depth and k as evaluation parameters, not universal constants.