Why raw-score blending is fragile

Dense retrieval and BM25 calculate relevance differently. A dense-search score reflects the relationship between query and document embeddings, while BM25 is based on term occurrence and document statistics. Even when both systems return numeric scores, equal-looking values do not imply equal relevance.

A weighted formula such as dense_score + bm25_score requires score normalization and ongoing tuning. That tuning can become unstable as corpora, embedding models, analyzers, or query patterns change. Rank-based fusion offers a simpler starting point because it only needs the ordering returned by each retriever.

  • Use dense search to capture semantic similarity.
  • Use BM25 to preserve exact-match and rare-term behavior.
  • Avoid treating scores from separate ranking systems as interchangeable.

Fuse two ranked lists with RRF

Run the same user query through the dense and sparse retrieval paths, requesting a sufficiently deep candidate list from each. For every document, add a contribution based on its position in each list: RRF(d) = Σ 1 / (k + rank_i(d)). The sum is taken across the result lists in which document d appears.

The constant k reduces the impact of small rank differences near the top of a list. A commonly used starting value is 60, but it is a tuning parameter rather than a universal rule. Documents appearing in both lists receive two contributions, while a strong result from only one retriever can still enter the final ranking.

  • Use one stable document ID across the dense and BM25 indexes.
  • Treat rank as one-based: the first result has rank 1.
  • Deduplicate documents before returning the fused top-k.
  • Keep the original retrieval scores for debugging, even if fusion does not use them.

Make fusion observable and testable

Start with a small evaluation set drawn from real search behavior. Include semantic queries such as paraphrases, lexical queries containing product codes or names, and mixed queries that contain both a concept and an exact constraint. Compare dense-only, BM25-only, and RRF result sets against relevance judgments or carefully reviewed examples.

Operationally, log which retrieval path contributed each returned document and its rank in that path. This makes hybrid results explainable: a document may win because both retrievers found it, because BM25 recognized an exact identifier, or because dense search found a meaningful paraphrase. Those observations are more actionable than a single opaque combined score.

  • Fetch more candidates than the number of results shown to users.
  • Apply shared filters consistently before fusion when possible.
  • Inspect queries where one retriever contributes no useful candidates.
  • Tune candidate depth and k using held-out queries, not anecdotes alone.