Why raw dense and sparse scores should not be added

A dense-search score and a BM25 score do not share a universal scale. Their ranges, distributions, and meanings depend on the underlying retrieval method, index configuration, query terms, and corpus. A rule such as "0.6 times dense plus 0.4 times BM25" may appear to work on a small test set but can behave differently when query vocabulary or document length changes.

This matters especially for mixed query traffic. A natural-language question may benefit from semantic similarity, while a query containing an error code, product name, file path, or quoted phrase may depend on lexical matching. A fusion layer should preserve evidence from both retrievers without assuming one score unit equals another.

  • Dense retrieval ranks by proximity in an embedding space.
  • BM25 ranks using term-frequency and corpus-statistical signals.
  • Raw-score addition requires calibration that can drift as data and queries change.
  • Rank-based fusion avoids comparing incompatible score scales.

Fuse the two ranked lists with RRF

Reciprocal Rank Fusion assigns each document a contribution based on its position in each result list. For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the document's one-based rank in retriever i and k is a positive constant. Sum the contribution from dense retrieval and the contribution from BM25, then sort by the final total.

Choose a retrieval depth before fusion, such as taking the top N results from each retriever. Deduplicate by a stable document or chunk identifier, not by title or text alone. A document returned by both systems receives two contributions; a document returned by only one can still rank well when it appears near the top of that list.

  • Run dense search against Talqora's regional S3 Vectors-backed dense retrieval path.
  • Run sparse search through the Quickwit BM25 retrieval path.
  • Collect the top N candidates from each path.
  • Deduplicate, compute RRF scores, sort descending, and pass the fused set onward.

Tune with query slices, then observe misses

Start with equal treatment of the two rankers and inspect results across representative query slices: conversational questions, exact identifiers, short keyword searches, and domain-specific terminology. RRF is useful here because it gives a stable baseline before introducing more complex weighting or learned reranking.

Treat fusion as a candidate-generation step rather than the final definition of relevance. Log which retrieval path contributed each result, its original rank, and whether users selected or accepted it when that signal is available. These records reveal whether a miss came from sparse recall, dense recall, chunking, metadata filtering, or a downstream ranking decision.

  • Keep the dense and BM25 ranks alongside the fused rank for debugging.
  • Evaluate queries with exact tokens separately from semantic paraphrases.
  • Check whether relevant documents are absent from both candidate lists before changing fusion.
  • Add reranking only after the candidate set has reliable recall.