Why raw dense and BM25 scores should not be added

Dense retrieval scores and BM25 scores are produced by different ranking functions. Even when both are returned as numbers, their ranges, distributions, and meanings are not inherently aligned. A score of 0.7 from one system is not necessarily stronger or weaker than a score of 12 from another.

Directly adding uncalibrated scores can make one retrieval path dominate because of its numerical scale rather than because it found better candidates. This is especially risky when query wording varies: a precise identifier may benefit from lexical retrieval, while a paraphrased question may benefit from dense retrieval.

  • Dense search can surface semantically related passages with different wording.
  • BM25 can reward exact terms, names, codes, and rare phrases.
  • Score scales can change with index settings, query composition, and implementation details.
  • Ranking positions are usually safer to combine than raw scores.

Fuse candidate lists with reciprocal rank fusion

RRF assigns each document a contribution based on its position in each ranked list. For a document d, calculate RRF(d) = sum of 1 divided by (k + rank_i(d)) across the retrieval lists where d appears. The rank starts at 1, and k is a positive constant that reduces the impact of small rank differences near the top of a list.

In a Talqora integration, an application can issue a dense query against regional S3 Vectors and a sparse query against Quickwit BM25, then deduplicate document identifiers and compute the fused ranking locally. A document appearing near the top of both lists receives contributions from both; a document unique to one list can still remain competitive.

  • Retrieve a candidate set from dense search and another from BM25.
  • Use a stable document or chunk identifier to deduplicate results.
  • Apply the same k value to both lists as an initial baseline.
  • Sort documents by fused score, then return the desired top-k results.

Make fusion observable and tune it with judged queries

RRF avoids score calibration, but it still has choices worth testing. Candidate depth determines what can enter the fused result set, while the k constant determines how sharply rank position affects contribution. Evaluate these settings using representative queries and relevance judgments rather than relying on a single anecdotal search.

Logging per-query retrieval provenance is equally important. Record whether each final result came from dense search, BM25, or both, along with its rank in each source list. These fields make it easier to diagnose failures such as missing exact matches, overly broad semantic matches, or duplicate chunks crowding the result set.

  • Include exact-term, semantic, ambiguous, and long-tail queries in evaluation.
  • Inspect top results from each source before and after fusion.
  • Keep candidate depth and fusion settings versioned with the application.
  • Consider reranking the fused shortlist only after verifying first-stage retrieval quality.