Why raw dense and BM25 scores should not be added directly

Dense retrieval ranks documents by proximity between embeddings. BM25 ranks them from term-level evidence such as matching query terms, term frequency, and document statistics. Both produce ordered result sets, but their numerical scores have different meanings and ranges.

Adding these scores directly can make one retriever dominate merely because its score distribution is larger. Normalization can help in controlled settings, but it introduces choices about sampling, query behavior, and index-specific score distributions. Rank-based fusion begins with the more stable fact available from both systems: result order.

  • Dense retrieval is useful for semantic similarity and paraphrases.
  • BM25 is useful when exact terms, identifiers, and rare vocabulary matter.
  • A raw score of 0.8 in one retriever is not inherently comparable to a score of 8 in another.
  • Fusion should preserve evidence from either retrieval path.

Fuse two candidate lists with reciprocal rank fusion

Run the same user query through both retrieval paths, requesting enough candidates from each list to create a useful merged pool. For every document identifier returned by either search, assign an RRF contribution based on its rank in each list.

The common formula is RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the one-based rank of document d in result list i, and k is a positive constant. Sum contributions across lists, sort by the resulting value, and return the top documents.

  • Use stable document IDs to deduplicate documents appearing in both result lists.
  • Treat an absent document as contributing zero from that retriever.
  • Choose k deliberately; a larger value reduces the difference between nearby ranks.
  • Keep the dense and BM25 ranks available in logs for later relevance analysis.

Implement fusion at the retrieval boundary

A clean implementation places fusion after Talqora Vector has obtained a dense ranked list from regional S3 Vectors and a sparse ranked list from Quickwit BM25. This keeps each retriever responsible for its own query execution and makes the fusion logic small, inspectable, and independent of either score scale.

Start with equal weight for both lists. If evaluation later shows that one source is more reliable for a defined query class, weighted RRF can express that preference by multiplying each list's reciprocal-rank contribution. Make such changes from judged queries or application feedback, rather than from assumptions about score magnitude.

  • Retrieve a fixed candidate depth from both sources before fusion.
  • Apply metadata filters consistently to both retrieval paths when your application requires them.
  • Log query text, source ranks, fused rank, and selected document IDs.
  • Evaluate failures separately for exact-match queries, semantic queries, and mixed queries.