Why raw dense and BM25 scores should not be added

A dense retrieval score reflects a relationship in embedding space, while a BM25 score is driven by term frequency, document frequency, and document length. Even when both are useful relevance signals, their numeric ranges and distributions have different meanings.

Adding those values directly can make one retrieval path dominate simply because it emits larger numbers. That behavior may change as embeddings, analyzers, document collections, or query patterns change. A fusion method based on rank avoids assuming that a score of 0.8 from dense search has the same interpretation as a BM25 score of 0.8.

  • Dense retrieval helps with semantic similarity and paraphrased queries.
  • BM25 helps with exact vocabulary, product names, error codes, and identifiers.
  • Score scales may vary across retrieval methods and over time.

Fuse ranked lists with RRF

RRF assigns each document a contribution based on its position in each ranked result list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) for every list where the document appears. The constant k dampens the effect of very high positions and is chosen as an application parameter.

In a Talqora retrieval flow, retrieve a candidate list from regional S3 Vectors for dense search and another from Quickwit BM25 for sparse search. Match results by a stable document or chunk identifier, calculate the RRF score in the application layer, sort descending, and return the top fused candidates to the next stage.

  • Use the same stable ID in dense and sparse indexes.
  • Choose a candidate depth for each path before fusion, such as the top N results.
  • Treat missing documents in one list as contributing zero from that list.
  • Keep the original ranks and source labels for debugging.

Evaluate fusion with representative query slices

RRF removes the need to calibrate incompatible scores, but it does not remove the need to evaluate relevance. Build a small set of real query types, including natural-language questions, exact-name lookups, acronym-heavy requests, and queries containing codes or quoted terms.

Compare dense-only, sparse-only, and fused rankings using human judgments or task-specific success criteria. Review failures by query type. If exact-match queries are being displaced, inspect sparse candidate depth; if paraphrases are absent, inspect embedding coverage, chunking, and dense candidate depth before changing the fusion rule.

  • Record the fused rank and each source rank for reviewed results.
  • Check whether relevant documents appear in either candidate list before fusion.
  • Evaluate changes on recurring production-like queries, not only curated examples.
  • Version retrieval settings so ranking changes can be reproduced.