Why raw score blending is fragile

Dense retrieval scores and BM25 scores express different signals. A dense search score reflects proximity in an embedding space, while BM25 emphasizes query-term occurrence, frequency, and document-length normalization. Even when both systems return a numeric score, the values do not necessarily share a stable range or interpretation.

Adding those scores with fixed weights can therefore create surprising behavior. A small change to an embedding model, analyzer, corpus, or query wording may shift one score distribution enough to alter the balance. Rank-based fusion avoids requiring a shared score scale.

  • Use dense retrieval for semantic similarity, paraphrases, and concept-level matches.
  • Use BM25 for identifiers, uncommon names, quoted phrases, and exact vocabulary.
  • Avoid assuming that a score of 0.8 from one retriever means the same thing as 0.8 from another.

Fuse two candidate lists with RRF

Run the same user query through dense search and BM25, requesting a bounded candidate set from each. For every document returned by either list, add a contribution based on its rank: 1 divided by k plus the rank. The document's fused score is the sum of its contributions across lists.

The constant k reduces the advantage of being ranked first and makes the method less sensitive to small position changes near the top of a list. A commonly used starting point is 60, but it is a tuning choice rather than a universal rule. Preserve document identity consistently across both result sets so duplicate records merge into one candidate.

  • Dense rank 1 contributes 1 / (k + 1).
  • BM25 rank 4 contributes 1 / (k + 4).
  • A document appearing in both lists receives both contributions.
  • Sort the merged documents by total RRF score, then return the top results.

Make the fusion observable and testable

Start with a small query set drawn from real product, support, or content-search traffic. Include queries with exact tokens, abbreviations, natural-language questions, and ambiguous wording. Review whether the fused top results contain the relevant documents and whether either retrieval path is routinely contributing nothing useful.

Log the source ranks that produced each fused result. This makes failures diagnosable: a missing result may indicate sparse recall, dense recall, document-index mismatch, or an issue after fusion. If a downstream reranker is used, retain the pre-rerank fused list as well, so retrieval quality and reranking quality can be evaluated separately.

  • Choose a candidate depth that leaves enough documents for overlap and complementary recall.
  • Track whether final results came from dense search, BM25, or both.
  • Version embedding models, sparse indexing settings, and fusion parameters with evaluation results.
  • Treat k and candidate depth as measured configuration choices, not fixed defaults.