Why raw-score blending is fragile

A dense retriever ranks documents by semantic proximity in an embedding space. BM25 ranks documents using term occurrence and corpus-level statistics. Even when both systems return numeric scores, those values arise from different models and different distributions.

Adding or averaging uncalibrated scores can make one retriever dominate for accidental reasons, such as a wider score range. The result may change unexpectedly as documents are added, text is re-chunked, embeddings are replaced, or BM25 parameters are adjusted.

  • Dense search is often useful for paraphrases and related concepts.
  • BM25 is often useful for exact terminology, identifiers, and uncommon names.
  • Raw score magnitude alone is not evidence that two ranking systems are equally confident.

Fuse ranks with RRF

RRF assigns each document a contribution based on its position in each result list, then sums those contributions. For a document d, the common form is: RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the one-based rank from retriever i and k is a positive constant.

The constant k reduces the difference between adjacent high ranks and limits the influence of any single list. In practice, retrieve a bounded candidate list from both dense search and BM25, deduplicate by document or chunk identifier, calculate the fused score, and sort descending.

  • Use one-based ranks: the first result has rank 1.
  • A document returned by both retrievers receives two contributions.
  • A document missing from one list simply receives no contribution from that list.
  • Keep the source ranks in debug output so ranking decisions remain inspectable.

Make fusion a measurable retrieval layer

Start with a small evaluation set containing realistic queries and documents judged relevant for each query. Include cases that stress both retrieval modes: product codes, quoted phrases, abbreviations, natural-language questions, and terminology that may not appear verbatim in the answer.

Review failures by source rather than only by final rank. If BM25 finds a result that dense retrieval misses, inspect the exact terms. If dense retrieval recovers a paraphrase that BM25 misses, inspect the chunk boundaries and language used in the query. RRF makes this diagnosis clearer because it preserves each retriever's independent ranking signal.

  • Evaluate recall at a candidate cutoff before optimizing final ordering.
  • Test several candidate-list depths; a shallow list can prevent useful overlap from reaching fusion.
  • Treat k and source-specific weighting as evaluation parameters, not universal constants.
  • Log dense rank, BM25 rank, and fused rank for sampled production queries.