Why raw-score merging is usually fragile

A dense retrieval score and a BM25 score should not be assumed to be directly comparable. Their ranges can differ, their distributions can shift as an index changes, and a larger number in one system does not necessarily represent the same relevance confidence as a larger number in the other.

A common failure mode is to add or average uncalibrated scores. This can quietly favor whichever retriever happens to emit numerically larger values. Score normalization can help in some cases, but it introduces additional assumptions about score distributions and can become sensitive to query type.

  • Use dense retrieval for semantic similarity and paraphrased intent.
  • Use BM25 for literal wording, identifiers, acronyms, and rare terms.
  • Treat each retriever’s ranking as meaningful even when its score scale is not.
  • Evaluate merged results on representative queries, not only broad topical searches.

Fuse two candidate lists with RRF

Run the same user query through the dense and sparse retrieval paths, then collect a candidate list from each. For every document, RRF assigns a contribution based on its rank in each list: 1 divided by a constant plus the document’s rank. Sum the contributions for documents that appear in one or both lists, then sort by the combined value.

Using a constant such as 60 is a widely used starting point because it gives substantial credit to high ranks while preventing rank-one placement from overwhelming every other signal. The right value is a tuning choice: keep it explicit in configuration and validate it against your own relevance judgments.

  • For each result, calculate: RRF score = sum of 1 / (k + rank).
  • Use rank positions beginning at 1, not 0.
  • Deduplicate by a stable document or chunk identifier before final sorting.
  • Retrieve enough candidates from both paths that useful overlap and complementary matches can surface.

Make fusion operationally useful

RRF is most effective when both retrieval paths operate over compatible units. If dense retrieval indexes document chunks while BM25 indexes whole documents, the fused list can mix levels of granularity. Choose a shared retrieval unit, or define a clear roll-up rule from chunks to documents before presenting results.

Instrument the process separately from the final answer or application action. Record which retriever contributed each candidate, its original rank, the fused rank, and whether users selected or used the result. This makes it possible to discover queries where one path consistently supplies the useful evidence.

  • Keep metadata filters consistent across dense and sparse retrieval when possible.
  • Log dense rank, BM25 rank, fused rank, and document identifiers for debugging.
  • Create a test set containing exact-term, paraphrase, mixed-intent, and no-match queries.
  • Use relevance feedback to tune candidate depth and the RRF constant before adding more complex ranking logic.