Why raw dense and BM25 scores should not be added

A dense-search score and a BM25 score are produced by different ranking functions. Their ranges, distributions, and sensitivity to query length can differ substantially. A value that appears large in one system is not automatically equivalent to the same-looking value in the other.

Adding those scores directly creates a hidden calibration problem. It may work for a narrow query set, then behave differently for acronym-heavy queries, long natural-language questions, or searches containing exact product names. Rank-based fusion avoids requiring a shared score scale.

  • BM25 rewards term overlap and term rarity.
  • Dense retrieval ranks by embedding similarity.
  • Raw scores should be treated as system-specific unless they have been explicitly calibrated.
  • Ranks are easier to combine because each list already expresses an ordering.

Fuse two candidate lists with RRF

For each query, retrieve a candidate list from dense search and another from BM25. In a Talqora architecture, the dense list can come from regional S3 Vectors and the sparse list can come from Quickwit BM25. Keep the document or chunk identifier with every result so matching items can be merged.

RRF assigns each item a contribution based on its rank in each list: score(d) = Σ 1 / (k + rank_i(d)). The sum is taken across result lists where document d appears. The constant k reduces the impact of very small rank differences near the top of a list; a commonly used starting value is 60, but it should be treated as a tunable application parameter.

  • Retrieve the same candidate depth from both systems to start, such as the top N results.
  • Use one stable ID per retrievable chunk or document when deduplicating.
  • Assign ranks beginning at 1, not 0.
  • Sort merged items by descending RRF score, then apply a deterministic tie-break rule.

Make fusion observable and test it by query shape

RRF is simple, but it still needs evaluation. Log the source ranks that contributed to each fused result, not only the final order. This makes it possible to see whether a result was supported by both retrieval methods, rescued by exact-term matching, or surfaced only through semantic similarity.

Build a small evaluation set that reflects how people actually search. Include exact identifiers, short keyword queries, paraphrased questions, mixed natural-language and code-like queries, and queries with ambiguous terms. Compare dense-only, BM25-only, and fused rankings using the same relevance judgments.

  • Record dense rank, BM25 rank, and final fused rank for returned items.
  • Check whether duplicate chunks from the same source crowd out useful coverage.
  • Evaluate candidate depth and k separately; changing both at once makes results harder to interpret.
  • Apply metadata or access-control constraints consistently before fusion when those constraints are required.