Why raw score blending is fragile

A dense-search score and a BM25 score do not necessarily share a common scale or meaning. Their ranges can change with index configuration, query wording, document length, embedding choice, or the search engine’s scoring details.

Adding those values together can therefore make one retrieval method dominate for reasons unrelated to relevance. A query may receive strong lexical evidence from BM25, for example, while the dense score has a numerically larger range. Score calibration is possible, but it requires careful data collection and ongoing validation.

  • Dense retrieval is useful when relevant documents use different wording from the query.
  • BM25 is useful when exact terms, identifiers, error codes, or product names matter.
  • A stable merge strategy should avoid assuming that two engine scores are interchangeable.

Fuse ranks with reciprocal rank fusion

Reciprocal rank fusion, or RRF, combines result lists by position rather than by their original scores. For each document, add a contribution from every list in which it appears: 1 divided by k plus its rank. The constant k reduces the difference between very high ranks and lower ranks, helping the merge remain less sensitive to a single list.

In a Talqora integration, an application can issue a dense query against regional S3 Vectors and a sparse BM25 query against Quickwit, then perform RRF after receiving the two result sets. Use a stable document identifier shared by both indexes so results can be deduplicated before sorting.

  • RRF formula: score(document) = Σ 1 / (k + rank).
  • Choose the same candidate depth from both systems as a starting point, such as the top N results.
  • Treat rank 1 as the first result in each list.
  • Return the documents with the highest fused scores after deduplication.

Make fusion observable and query-aware

RRF is simple, but it should not be invisible. Log whether each final result came from dense retrieval, BM25 retrieval, or both. This makes it easier to inspect failures: an exact identifier that never reaches the sparse candidate list is a different problem from a semantic match that is ranked too low after fusion.

You can also adjust retrieval policy by query shape without changing the fusion mechanism. Queries containing quoted phrases, version strings, ticket numbers, or unusual punctuation may warrant a larger BM25 candidate pool. Natural-language questions may benefit from ensuring the dense search also contributes enough candidates.

  • Store the source ranks and fused rank for debugging and relevance review.
  • Evaluate representative query groups, not only an overall relevance average.
  • Keep index-time document IDs and metadata consistent across dense and sparse indexes.
  • Revisit candidate depth and k when corpus size, query traffic, or content patterns change.