Why raw-score fusion is usually fragile

A dense-search score and a BM25 score do not necessarily represent the same thing. Their ranges, distributions, and sensitivity to query length can differ. Even a score normalization strategy that appears to work for one collection can become unreliable after documents, embeddings, analyzers, or query patterns change.

Rank-based fusion avoids requiring a shared score scale. Instead of asking whether a dense score of one value is equivalent to a BM25 score of another, it asks a narrower question: which documents repeatedly appear near the top of independently useful result lists?

  • Use dense retrieval for semantic similarity and paraphrase tolerance.
  • Use BM25 for exact wording, codes, names, and rare terms.
  • Avoid assuming scores from separate retrieval methods are numerically comparable.
  • Evaluate fusion using judged queries rather than score intuition.

Fuse dense and sparse candidates with RRF

In Talqora’s retrieval architecture, a query can produce a dense candidate list from regional S3 Vectors and a sparse candidate list from Quickwit BM25. RRF assigns each document a contribution based on its position in each list, then sums those contributions. Documents present in both lists receive support from both retrieval signals.

A common form is RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the one-based rank of document d in result list i, and k is a positive constant. The constant reduces the impact of small rank differences near the top. Start with the same candidate depth for both retrieval paths, then test whether one path needs a deeper list for your corpus.

  • Retrieve a bounded candidate list from dense search and from BM25.
  • Deduplicate documents by a stable document identifier.
  • Assign each document an RRF contribution for every list in which it appears.
  • Sort by summed RRF score and return the leading fused results.

Make fusion observable before making it complex

RRF is easy to implement, but it should still be treated as a retrieval policy that needs inspection. Log the source ranks for each fused result: whether it came from dense search, BM25, or both; its rank in each list; and its final fused position. These fields make it possible to diagnose why a result appeared and whether one retriever is dominating.

Build a small query set that reflects real work: natural-language questions, exact product names, abbreviations, identifiers, and mixed queries. For each query, compare dense-only, BM25-only, and fused results. If fusion consistently harms a particular query class, investigate query routing, metadata filters, document chunking, or candidate depth before introducing more elaborate weighting.

  • Record per-result dense rank, BM25 rank, and fused rank.
  • Review failures by query type, not only by aggregate relevance.
  • Check that filters and document identifiers are applied consistently across both paths.
  • Change one retrieval parameter at a time and retain a baseline run.