Why raw dense and BM25 scores should not be added

A dense-search score and a BM25 score are produced by different retrieval models and have different scales and distributions. Even when both are useful indicators within their own result lists, a score of 0.8 from one system has no inherent relationship to a score of 8.0 from the other.

Adding raw scores creates an accidental weighting scheme. A change in embedding model, index configuration, corpus composition, or BM25 analyzer can shift score ranges without changing the underlying relevance of documents. That can cause one retrieval mode to dominate simply because its numbers are numerically larger.

Rank-based fusion avoids that comparison. It asks a simpler question: did a document appear near the top of one or both independently useful result lists?

  • Use dense retrieval for semantic similarity and wording variation.
  • Use BM25 for exact terms, names, codes, and uncommon vocabulary.
  • Treat each system's raw score as local to that system unless it has been explicitly calibrated.

Fuse two candidate lists with reciprocal rank fusion

Run the same user query through the dense path backed by S3 Vectors and the sparse path backed by Quickwit BM25. Request a candidate window from each path, then identify documents using a stable shared ID. RRF assigns each document a contribution based on its position in each list.

For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs over the lists in which d appears. The constant k reduces the advantage of a document that is only marginally higher in one list, while documents that rank well in both lists accumulate a stronger combined score.

A common starting value for k is 60, but it is a tuning parameter rather than a rule. Keep the dense and sparse candidate depths intentionally larger than the number of results shown to the user, because fusion can promote a document that is not at the top of either individual list.

  • Deduplicate by document or chunk ID before presenting results.
  • Record rank positions separately for dense and BM25 results.
  • Sort by fused score, then use a deterministic tie-breaker.
  • Apply filters consistently to both retrieval paths before fusion.

Evaluate the fusion policy with query slices

Do not judge hybrid retrieval only on a blended average. Build a small evaluation set that includes semantic paraphrases, exact-title queries, product or error-code queries, short queries, and longer natural-language questions. These slices reveal when one retrieval mode is carrying useful traffic that the other misses.

Inspect the overlap between the two candidate lists as well as final relevance. Low overlap is not automatically bad: it may indicate complementary retrieval. The important question is whether documents introduced by either path improve the quality of the fused top results.

Start with equal treatment of the two lists, then adjust only in response to observed failure cases. If your application has a clear requirement for exact identifiers, for example, a policy that preserves strong BM25 matches may be appropriate. Keep those decisions explicit and testable rather than relying on unexamined score arithmetic.

  • Log the source ranks that contributed to each fused result.
  • Review misses where relevant documents appear in only one retrieval path.
  • Test candidate depth and the RRF k value against a fixed query set.
  • Separate retrieval evaluation from later reranking or generation evaluation.