Why raw dense and sparse scores should not be added

Dense retrieval ranks documents by vector similarity, while BM25 ranks them from term-frequency and corpus statistics. A value from one method does not inherently mean the same thing as a value from the other. Adding scores directly can cause one retrieval path to dominate simply because its numeric range is wider.

Score normalization can help, but it introduces choices about distributions, query classes, outliers, and refresh behavior. Those choices are especially fragile when the corpus changes or when queries vary between short keyword searches and longer natural-language questions.

  • Dense search can recover semantically related wording.
  • BM25 can strongly reward exact terms, identifiers, and rare vocabulary.
  • Raw score ranges may shift as indexes and corpora evolve.
  • A fusion method should preserve useful ranking evidence without assuming shared score units.

Fuse ranks with reciprocal rank fusion

RRF merges independently ranked lists by assigning each document a contribution based on its position in each list. For a document d, calculate RRF(d) = sum of 1 / (k + rank_i(d)) across the retrieval lists where d appears. The constant k reduces the influence of small rank differences near the top of a list.

In a Talqora retrieval flow, request a candidate set from dense search and another from Quickwit BM25, then identify documents that occur in either result set. Sum their reciprocal-rank contributions, sort by the combined value, and return the highest-ranked merged candidates to the next stage of the application.

  • Use one-based ranks: the first result has rank 1.
  • Deduplicate by a stable document or chunk identifier before sorting.
  • Choose the same candidate depth for both paths as a clear initial baseline.
  • Treat k as a ranking-policy parameter and evaluate it with representative queries.

Make fusion observable before making it complex

Log the dense rank, BM25 rank, and final fused rank for every returned candidate. These fields make it possible to see whether a result was supported by both retrieval methods or rescued by only one. They also help diagnose failures such as exact-match documents being pushed down or semantically relevant documents never entering the candidate pool.

Start with a small offline query set that reflects real retrieval work: product names, error strings, abbreviations, natural-language questions, and mixed queries containing both concepts and identifiers. Review the fused top results alongside the individual lists before introducing learned reranking or score-based weighting.

  • Include queries with exact identifiers and rare terms.
  • Include paraphrases that lack the source document's exact wording.
  • Track whether relevant documents appear in either candidate list before fusion.
  • Keep the retrieval lists and fusion configuration available for debugging.