Why raw score blending is fragile

Dense-search similarity values and BM25 scores are not naturally comparable. Their ranges, distributions, and meaning can change with the embedding model, index settings, query length, corpus composition, and sparse-search configuration. Adding the scores directly can make one retrieval method dominate for accidental mathematical reasons rather than because it produced better results.

Rank-based fusion avoids treating either score as a universal relevance measurement. Instead, it asks a narrower question: how highly did each retrieval method place a document for this query? This makes fusion easier to reason about when combining results generated by different retrieval systems.

  • Dense retrieval helps with semantic matches and paraphrases.
  • BM25 helps with exact vocabulary, identifiers, and rare terms.
  • A score of 0.7 from one method does not inherently equal a score of 0.7 from another.
  • Ranks are comparable even when underlying score scales are not.

Fuse candidate lists with RRF

Retrieve a bounded candidate list from dense search and another from BM25, then deduplicate them by a stable document or chunk identifier. For each candidate, add an RRF contribution from every list in which it appears: 1 divided by k plus the candidate’s rank. Sort candidates by the resulting sum.

The constant k reduces the advantage of a single first-place result and makes the method less sensitive to small rank changes near the top of a list. It is a tunable application-level policy, not a universal constant. Start with one fixed value, evaluate on representative queries, and change it only when the evidence supports doing so.

  • For each result list, use one-based ranks: 1, 2, 3, and so on.
  • Compute: RRF score = sum of 1 / (k + rank) across participating lists.
  • Use the same canonical identifier to deduplicate dense and BM25 hits.
  • Keep the source ranks in logs so fused outcomes can be inspected later.

Evaluate the disagreements, not just the aggregate

A hybrid strategy is most valuable on queries where the retrieval methods disagree. Build a small evaluation set that includes natural-language questions, exact identifiers, abbreviations, multilingual or domain-specific wording where relevant, and queries containing terms that may not be well represented by embeddings.

Review whether RRF surfaces documents that either individual method missed near the top. Also inspect regressions: a fused list can promote a document that appears moderately high in both lists over a document that is excellent in only one. The appropriate trade-off depends on the search task and should be validated with relevance judgments rather than assumed.

  • Track dense rank, BM25 rank, fused rank, and the selected document ID.
  • Separate exact-match queries from conceptual or exploratory queries.
  • Inspect queries with no overlap between the two candidate lists.
  • Version the embedding model, corpus snapshot, and fusion policy alongside evaluation results.