Why raw dense and BM25 scores should not be added directly

A dense-search score and a BM25 score are produced by different ranking systems. Their ranges, distributions, and meanings are not inherently comparable. A score that looks large in one system is not necessarily stronger evidence of relevance than a smaller score in the other.

Adding raw scores can therefore make ranking sensitive to implementation details rather than relevance. A change in embedding model, similarity calculation, BM25 configuration, or corpus composition may alter score scales and unexpectedly shift the combined ranking.

Rank-based fusion avoids this comparison. It uses each retriever’s ordering rather than treating their numerical scores as a shared unit.

  • Dense retrieval helps with synonymy, paraphrases, and conceptual similarity.
  • BM25 helps with literal terms, identifiers, codes, and rare vocabulary.
  • Rank positions are easier to combine across independently scored result lists.

Apply reciprocal rank fusion to two candidate lists

Run the same user query through the dense path and the BM25 path. Request a reasonably sized candidate list from each, then identify documents with a stable shared identifier. For every document that appears in either list, calculate an RRF score by adding 1 divided by k plus its rank for each list where it appears.

In notation, RRF(d) = Σ 1 / (k + rank_i(d)). The constant k reduces the influence of small differences near the very top of a list. It is a tuning parameter, not a universal relevance threshold; evaluate a few values using representative queries and relevance judgments.

For example, a document ranked 2nd by dense retrieval and 10th by BM25 receives contributions from both rankings. A document that appears only in one list can still rank well, but agreement between the two retrievers is rewarded.

  • Use one-based ranks: the first result has rank 1.
  • Deduplicate by a canonical document or chunk ID before presenting results.
  • Treat a missing document from a list as contributing zero from that list.
  • Keep the original dense and BM25 ranks for debugging and evaluation.

Make fusion operationally reliable

Fusion only works cleanly when both retrieval systems refer to the same retrieval unit. If dense search indexes chunks while BM25 indexes whole documents, define a mapping before fusion. One practical approach is to retrieve and fuse chunk IDs, then group the final results by parent document while retaining the best-scoring chunk as supporting context.

Use the same filters and access constraints in both paths whenever possible. A regional or tenant filter applied to only one retriever can create confusing rankings and, more importantly, can make the candidate sets inconsistent.

Evaluate the fused list separately from each individual retriever. Build a small query set that includes exact-match queries, semantic questions, mixed queries, and zero-result cases. Review not only top-1 relevance but also whether useful results appear in the first page users actually inspect.

  • Store a canonical ID and metadata needed for filtering in both indexes.
  • Log query text, applied filters, source ranks, fused score, and selected result IDs.
  • Test candidate-list depth because fusion cannot recover documents absent from both lists.
  • Use relevance feedback to tune k, candidate depth, and any downstream reranking policy.