Why raw dense and BM25 scores should not be added

A dense-search score and a BM25 score are produced by different retrieval models and have different distributions. A high score in one system does not necessarily mean the same thing as a high score in the other. Adding them directly can make one retriever dominate simply because its numeric range is larger.

Normalizing scores can help in controlled settings, but it introduces operational work. Score distributions can change as documents, embeddings, analyzers, and query patterns change. A fusion method based on rank avoids treating these independent scores as if they were measurements on a shared scale.

  • Dense retrieval emphasizes semantic similarity between query and document vectors.
  • BM25 emphasizes overlap between query terms and indexed terms.
  • Score magnitudes and score ranges are retriever-specific.
  • Rank position is easier to compare across independently generated result lists.

Fuse two result lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in each result list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) for every list in which d appears. The constant k reduces the gap between adjacent top ranks and prevents a single first-place result from overwhelming all other evidence.

In a typical implementation, issue the dense and sparse searches in parallel, keep a bounded candidate set from each, and combine matching document IDs in an application-side map. Documents retrieved by both methods accumulate two contributions. Documents found by only one method can still rank well when they appear near the top of that list.

  • Use one stable, canonical document ID across dense and BM25 indexes.
  • Retrieve the same candidate depth from both paths as a starting point.
  • Choose and record a fixed k value so ranking behavior is reproducible.
  • Sort documents by fused score, then apply a deterministic tie-breaker such as document ID.

Make fusion debuggable before making it complex

Log the dense rank, sparse rank, and final fused rank for returned documents. This makes it possible to explain why an exact-match document rose in the final list or why a semantically related result remained visible despite limited keyword overlap. It also exposes failures such as mismatched document IDs, stale indexing paths, or filters applied to only one retriever.

Evaluate fusion using representative queries rather than relying on a single query type. Include natural-language questions, product names, error messages, acronyms, and identifiers. RRF is a strong baseline because it is straightforward to inspect; only add weighted fusion or learned reranking after the baseline and its failure modes are understood.

  • Store per-result provenance: dense, sparse, or both.
  • Apply equivalent tenant, access, and document-status filters to both retrieval paths.
  • Review queries where the two lists have little overlap.
  • Treat RRF as a baseline for relevance evaluation, not a substitute for evaluation.