Why raw dense and BM25 scores should not be added

A dense search system and a BM25 index produce scores for different reasons. A vector similarity score reflects the relationship between an embedded query and embedded documents. BM25 reflects term matches, term rarity, document length, and other lexical statistics. Even when both are useful signals, their numeric ranges and distributions do not share a natural unit.

Adding those values directly creates a hidden calibration problem. A small implementation change—such as a different embedding model, a revised chunking strategy, or an index update—can shift one score distribution and quietly change the balance of the combined ranking. The result may look stable in code while behaving differently in production.

  • Dense retrieval helps when query and document use different wording.
  • BM25 helps when exact tokens, names, error codes, or product terms matter.
  • Raw score magnitudes are not inherently comparable across retrieval methods.
  • A fusion method should remain understandable when either retrieval system changes.

Fuse ranked lists with Reciprocal Rank Fusion

Reciprocal rank fusion assigns a contribution to each document based on its rank in each result list. For a document d, compute RRF(d) as the sum of 1 divided by k plus its rank in every list where it appears. The constant k reduces the difference between adjacent top ranks and prevents one first-place result from overwhelming all other evidence.

In a Talqora retrieval flow, request a candidate list from dense search backed by regional S3 Vectors and another from sparse search backed by Quickwit BM25. Deduplicate by a stable document or chunk identifier, calculate the RRF score for every candidate, then sort descending. A document that ranks well in both lists naturally rises, while a strong result from only one method can still be retained.

  • Use one stable ID for the same chunk across dense and sparse indexes.
  • Retrieve more candidates than the final number you intend to return.
  • Represent missing results as no contribution, not as an invented score.
  • Start with a fixed k and evaluate it against representative queries before tuning.

Make fusion observable and safe to iterate

Hybrid ranking is easiest to improve when each stage is inspectable. Log the query, the dense rank, the BM25 rank, the fused rank, and the document ID for returned candidates. For privacy-sensitive systems, apply the same logging and retention rules used elsewhere in the application, and avoid recording source text unnecessarily.

Evaluate with a compact query set that reflects real retrieval work: exact identifier lookups, natural-language questions, terminology-heavy requests, and ambiguous queries. Review not only whether the desired item appears, but where it appears. When a result moves after a change, the per-retriever ranks explain whether the movement came from dense retrieval, BM25, or fusion.

  • Track candidate overlap between dense and sparse result lists.
  • Record rank contributions so fused ordering can be explained.
  • Test filters and access controls before fusion, not only after ranking.
  • Change chunking, retrieval depth, and fusion settings independently when diagnosing regressions.