Why merge ranks instead of raw scores?

Dense and sparse systems usually produce scores with different meanings and ranges. A vector similarity score is not automatically comparable to a BM25 score, even when both are useful indicators of relevance. Adding them directly can make the result depend more on score scale than on retrieval quality.

RRF avoids that calibration problem by using a document’s position in each ranked list rather than its raw score. A document that appears near the top of either list receives a meaningful contribution; one that appears in both lists receives contributions from both.

  • Dense retrieval helps when the query and document use different but related language.
  • BM25 helps preserve exact matches for tokens, acronyms, codes, and quoted phrases.
  • Rank-based fusion does not require dense and sparse scores to share a common scale.
  • The approach can be implemented after two independent retrieval calls.

Apply reciprocal rank fusion to two candidate lists

For each query, request a bounded candidate list from dense search and another from BM25. Normalize result identities to a shared document or chunk ID, then assign each candidate an RRF score. For a document d, calculate score(d) as the sum of 1 divided by k plus its rank in every list where it appears.

The constant k reduces the difference between adjacent high ranks and prevents the first few positions from dominating too sharply. A commonly used starting value is 60, but it should be treated as a tuning parameter rather than a universal optimum. Sort candidates by their combined RRF score and return the top results, optionally passing that fused set to a later reranking stage.

  • Retrieve the same candidate depth from both paths as an initial baseline, such as top 20 or top 50.
  • Use one-based ranks: the first result has rank 1.
  • Deduplicate by a stable chunk ID before producing the fused ranking.
  • Preserve source ranks and original scores in logs for debugging and evaluation.

Make fusion observable and evaluate it by query type

A fused ranking is easier to operate when each result retains evidence about how it arrived. Store whether the item came from dense retrieval, BM25, or both, along with its rank in each list. This makes it possible to explain surprising results and identify whether a query class is relying too heavily on one retrieval path.

Evaluate with a small, representative set of labeled queries before changing production defaults. Include semantic paraphrases, exact identifier lookups, multi-term questions, and queries with rare terminology. Compare dense-only, sparse-only, and fused rankings using the same corpus, filters, and candidate depths.

  • Inspect failures where a relevant exact match appears only in BM25 results.
  • Inspect failures where terminology differs but the intended concept is the same.
  • Measure ranking quality at the depth users actually inspect, such as top 5 or top 10.
  • Tune candidate depth and k separately; changing both at once obscures the cause of an improvement or regression.