Why raw dense and BM25 scores should not be added blindly

A dense retriever and a BM25 retriever produce scores for different reasons. A vector similarity score reflects a relationship in embedding space, while BM25 reflects term frequency, document frequency, and document-length normalization. Even if both scores appear numeric and ordered, their ranges and distributions are not inherently comparable.

Adding the two values together creates an unstated assumption: that one unit of dense score means the same thing as one unit of BM25 score. That assumption can change as documents are added, analyzers are adjusted, embeddings change, or queries vary in length and specificity. A fusion method based on rank avoids requiring that cross-system score equivalence.

  • Use dense retrieval to broaden recall for paraphrases and conceptual matches.
  • Use BM25 to preserve matches for names, codes, error messages, and exact phrases.
  • Treat each retriever's score as meaningful within its own result list first.
  • Merge candidate lists after retrieval rather than forcing a shared score scale.

Fuse the ranked lists with Reciprocal Rank Fusion

Reciprocal Rank Fusion, commonly abbreviated as RRF, gives each document a contribution based on its position in every ranked list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) for every list where d appears. The constant k reduces the difference between nearby top ranks and keeps a single first-place result from overwhelming all other evidence.

In a Talqora retrieval flow, request a candidate set from regional S3 Vectors and a candidate set from Quickwit BM25. Identify documents by a stable shared identifier, deduplicate them, calculate their RRF values, and sort descending. A document returned by both systems will accumulate evidence; a document ranked highly by only one system can still remain competitive.

  • Choose the same candidate depth for both retrievers as a straightforward starting point.
  • Use a stable document or chunk ID to join dense and sparse results.
  • Assign ranks starting at 1, not 0.
  • Start with a conventional positive k value and tune it using judged queries rather than score intuition.

Evaluate fusion using query groups, not a single average

Hybrid retrieval should be evaluated against the query patterns that matter in production. Build a small, reviewed set containing exact-term queries, acronym or identifier queries, natural-language questions, short ambiguous queries, and paraphrases. For each query, record whether relevant chunks appear in the candidate set and where they rank after fusion.

RRF is intentionally simple, but it still has design choices. Candidate depth, the k constant, field selection for BM25, chunking strategy, and embedding choice all affect the final list. Change one variable at a time, keep the judged set stable, and inspect failures directly. The goal is not to make every query look more semantic; it is to return the best evidence for the query type.

  • Measure recall at a candidate cutoff before focusing on final ranking.
  • Inspect cases where dense-only and BM25-only results disagree.
  • Keep exact identifiers and rare product terms in the evaluation set.
  • Re-run evaluation after changing chunk boundaries, sparse fields, or embedding models.