Why raw dense and BM25 scores should not be added casually

A dense-search score and a BM25 score are produced by different ranking models. Their ranges, distributions, and meanings can vary with the embedding model, index configuration, query wording, corpus composition, and sparse-search settings. A rule such as “multiply BM25 by 0.3, then add it to vector similarity” can appear to work for one query set while behaving poorly for another.

The issue is not that either signal is unhelpful. The issue is calibration: a score of 0.8 from one retriever is not inherently equivalent to a score of 0.8 from the other. Before using a weighted score sum, a team should have evaluation data showing that its normalization and weights remain suitable across the query classes that matter.

  • Dense retrieval can help with paraphrases and related concepts.
  • BM25 can preserve lexical precision for names, codes, and exact terminology.
  • Score scales should be treated as retriever-specific unless they have been deliberately calibrated.
  • Rank positions are often easier to combine safely than raw scores.

Fuse two ranked lists with reciprocal rank fusion

RRF assigns each document a contribution based on its position in each ranked result list. For a document d, the fused score is the sum of 1 divided by k plus the rank of d for every list in which it appears: RRF(d) = Σ 1 / (k + rank_i(d)). Documents that rank well in both dense and sparse results rise naturally, while a document that appears in only one list can still be retained.

The constant k reduces the advantage of a document moving only a few positions near the top of a list. A commonly used starting point is 60, but it is a parameter to validate rather than a universal truth. Keep ranks one-based, deduplicate documents by a stable document or chunk identifier, sort by the fused score, and return the top results.

  • Run dense retrieval against regional S3 Vectors.
  • Run sparse retrieval with Quickwit BM25 for the same user query.
  • Fetch a candidate depth from each retriever, such as the top N results.
  • Deduplicate candidates and sum their rank-based RRF contributions.

Make fusion observable and evaluate by query type

Log more than the final fused ranking. For each returned result, retain its dense rank, BM25 rank, fused score, and which retrievers contributed it. This makes it possible to investigate whether a result won because both systems agreed, because BM25 recognized an exact token, or because dense retrieval found a semantic match absent from the lexical list.

Evaluate on representative queries rather than relying on a few memorable examples. Include exact-title searches, product or policy identifiers, short ambiguous queries, natural-language questions, and domain-specific vocabulary. Compare dense-only, BM25-only, and RRF rankings using relevance judgments appropriate to the application, then adjust candidate depth or k only when the evaluation supports the change.

  • Store retriever ranks alongside the final result for debugging.
  • Inspect queries where dense and sparse retrieval disagree sharply.
  • Use stable chunk identifiers so duplicate content is not shown twice.
  • Treat fusion parameters as versioned retrieval configuration.