Why raw dense and BM25 scores should not be added

A dense retrieval score and a BM25 score are produced by different models and have different meanings. Even if both rank documents well within their own result lists, a dense score of 0.7 is not inherently equivalent to a BM25 score of 7.0. Adding them directly introduces an arbitrary weighting problem.

That problem becomes more noticeable across query types. A query containing an exact error code may benefit strongly from sparse retrieval, while a natural-language question with paraphrased wording may benefit from dense retrieval. A fixed score normalization rule can behave differently as query length, term rarity, and embedding similarity distributions change.

  • Dense retrieval ranks by vector similarity.
  • BM25 ranks using term occurrences and weighting statistics.
  • Score ranges and distributions may vary by query.
  • Rank position is often safer to compare than raw score magnitude.

Merge two candidate lists with reciprocal rank fusion

RRF assigns each document a contribution based on its rank in each list, then sums those contributions. For a document d, the fused score is: RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs over the ranked lists where d appears, and k is a positive constant that reduces the influence of very high individual ranks.

In a Talqora Vector retrieval flow, obtain a top-N candidate list from dense search backed by regional S3 Vectors and a top-N list from sparse BM25 search backed by Quickwit. Deduplicate document identifiers, calculate the RRF score for each candidate, then sort the merged set by that score. A document appearing near the top of both lists will naturally rise above documents supported by only one retrieval method.

  • Choose the same document identity across both indexes, such as a chunk ID.
  • Retrieve enough candidates from each branch to allow useful overlap and recovery.
  • Start with a fixed k, then evaluate it on representative queries.
  • Preserve source ranks in logs so fused outcomes can be inspected.

Make fusion observable before making it complex

The first implementation should make it easy to answer why a result was returned. Store the dense rank, BM25 rank, fused score, and document identifier for the final candidates. This exposes cases where one branch contributes valuable unique results, where both branches repeatedly agree, and where duplicate or inconsistent identifiers prevent proper merging.

Evaluate with a small query set that reflects real retrieval work: exact identifiers, product terminology, short ambiguous queries, and natural-language questions. Review not only the top result but also whether the desired document appears in the candidate set. If a result is absent from both lists, fusion cannot recover it; improve candidate generation, chunking, metadata filters, or indexing coverage first.

  • Compare dense-only, BM25-only, and fused ranked lists side by side.
  • Track whether relevant documents are retrieved before judging final ordering.
  • Use query categories to spot where either retrieval branch is weak.
  • Add more elaborate weighting only after a simple rank-based baseline is understood.