Why rank fusion is safer than score fusion

Dense retrieval and BM25 produce scores with different meanings. A vector similarity score reflects the relationship between an embedded query and embedded documents, while a BM25 score reflects term statistics and document length effects. Adding those values directly requires careful normalization and ongoing calibration.

RRF avoids that comparison. It only asks where a document ranked in each candidate list. This makes it a useful baseline when introducing hybrid retrieval, especially when corpora, embedding models, or BM25 configuration may change over time.

  • Dense retrieval can surface conceptually related language.
  • BM25 can preserve exact identifiers, product names, and rare terms.
  • Rank-based fusion does not depend on matching score ranges.
  • Independent candidate generation keeps each retrieval path inspectable.

Build two candidate lists before combining them

For each user query, create a dense representation and request a ranked candidate list from the vector index. In parallel, submit the original text query to the BM25 index. Apply the same required filters to both paths, such as tenant, document type, language, or access scope.

Choose a candidate depth that gives the fusion step room to recover documents favored by either retriever. The fused list is then deduplicated by a stable document or chunk identifier. If multiple chunks from one source document are allowed, decide explicitly whether to keep them independently or collapse them before presentation.

  • Use the same corpus version and filter semantics for both indexes.
  • Retain each result's rank, identifier, and retrieval source.
  • Deduplicate candidates before producing the final ranking.
  • Log dense-only, sparse-only, and overlapping candidates for evaluation.

Apply RRF and evaluate query classes separately

A common RRF formulation assigns each document a contribution of 1 divided by k plus its rank for every list in which it appears. Sum those contributions and sort descending. The constant k reduces the influence of small rank differences near the top; treat it as a configuration choice to validate against representative queries rather than a universal default.

Evaluation should distinguish query types. Exact-code and identifier searches may rely heavily on BM25, while paraphrased questions may benefit from dense retrieval. Review failed searches with the individual candidate lists visible: this reveals whether the issue is embedding coverage, lexical matching, filtering, chunking, or fusion behavior.

  • Score each document as the sum of 1/(k + rank) across result lists.
  • Use a stable tie-break rule, such as the best individual rank.
  • Measure relevance for exact-match, semantic, and mixed queries separately.
  • Keep fusion configuration versioned alongside retrieval logs.