Why fuse ranks instead of raw scores?

A dense retrieval score and a BM25 score are not naturally interchangeable. Their ranges, distributions, and meanings depend on the embedding model, index configuration, query terms, and corpus. Adding the two values directly can make one retriever dominate simply because its numeric scale is larger.

RRF avoids score calibration by using position in each result list. A document that appears near the top of either list receives credit; a document that ranks well in both receives more credit. This produces a simple merged candidate set while preserving the strengths of semantic and lexical retrieval.

  • Dense retrieval helps when relevant content uses different wording than the query.
  • BM25 helps when exact terms, codes, names, or phrases matter.
  • Rank-based fusion avoids assuming dense and sparse scores mean the same thing.

Apply reciprocal rank fusion to two result lists

For each query, retrieve a bounded list from dense search and another from BM25 search. Assign rank 1 to the first result in each list. For every document d, compute RRF(d) as the sum of 1 divided by k plus the rank of d for every list in which it appears. The constant k reduces the influence of small rank differences near the top.

A common starting point is k = 60, but it is a tuning parameter rather than a universal rule. Use the same document identifier in both retrieval paths so duplicate results can be combined. After calculating fused scores, sort descending and return the top candidates, or pass them to a later reranking stage if your application uses one.

  • Retrieve, for example, the top N dense and top N BM25 candidates.
  • Use: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Treat a missing document from a list as contributing zero for that list.
  • Deduplicate by a stable document or chunk identifier before returning results.

Make the fusion pipeline observable and testable

Start with representative queries rather than relying on a single search example. Include natural-language questions, exact product or internal terms, acronyms, identifiers, and queries with ambiguous wording. For each query, inspect which retriever contributed the final top results and whether fusion surfaces useful documents that either source alone missed.

Log the dense rank, BM25 rank, and fused rank for returned documents. These fields make it easier to diagnose regressions after changing chunking, embeddings, analyzers, or retrieval depth. They also reveal whether one retriever rarely contributes, which may indicate a data, query, or indexing issue rather than a need to change the fusion formula.

  • Keep dense and sparse candidate depths configurable.
  • Evaluate with labeled queries when available, plus structured manual review.
  • Record per-source ranks alongside the fused result.
  • Test exact-match and semantic queries separately before choosing defaults.