Why raw-score blending is risky

A dense-search score and a BM25 score are produced by different retrieval models and may have very different ranges, distributions, and meanings. Adding them directly can make one retrieval path dominate merely because its numeric scale is larger, not because its result is more relevant.

Score normalization can be useful when it is measured and maintained carefully, but it adds assumptions about score distributions. RRF avoids that dependency: a document earns credit based on where it appears in each result list.

  • Dense retrieval is useful for semantic similarity and paraphrases.
  • BM25 is useful for exact vocabulary, rare tokens, and keyword-heavy queries.
  • Ranks are easier to combine safely than unrelated raw scores.

Apply reciprocal rank fusion to two result lists

Run the same user query through dense retrieval and BM25 retrieval, requesting a bounded candidate list from each. For every document in either list, calculate an RRF score by adding 1 divided by k plus its rank from each list where it appears.

The formula is RRF(d) = Σ 1 / (k + rank_i(d)). A commonly used starting value for k is 60, although it should be treated as a tunable constant rather than a universal answer. Rank numbering should be consistent; using rank 1 for the first result is conventional.

  • Deduplicate by a stable document or chunk identifier before returning results.
  • Assign no contribution for a retrieval list where a document is absent.
  • Sort documents by descending fused score, then apply a deterministic tie-breaker.
  • Keep the source ranks in logs to make fusion decisions inspectable.

Tune the candidate window, not just the final top-k

If an application ultimately needs five chunks, retrieving only five candidates from each path can hide documents that become valuable after fusion. Fetch a larger candidate window from dense and sparse retrieval, fuse it, and then select the final context set.

Evaluate this pipeline with queries that represent real traffic. Include both semantic questions and exact-match queries, then inspect whether the fused list preserves the documents each retrieval method finds uniquely. The goal is not for both methods to agree; it is for the merged ranking to retain useful complementary evidence.

  • Start with an explicit per-source candidate limit and record it with evaluation results.
  • Check for duplicate chunks, near-duplicate content, and documents from the same source crowding out diversity.
  • Measure relevance at the final application cutoff, not only across the larger fused candidate set.
  • Review failures by query type before changing k or candidate limits.