Why raw dense and BM25 scores should not be added directly

A dense-search score and a BM25 score are measurements from different retrieval models. Their ranges, distributions, and sensitivity to query length can differ substantially. Adding them with a fixed weight may appear to work on a small test set while behaving unpredictably across document types or query classes.

Score normalization can help in some systems, but it creates another model to operate. It needs representative query traffic, monitoring for distribution changes, and decisions about how to handle outliers. When the immediate goal is reliable candidate fusion, rank-based methods avoid treating incomparable scores as if they had a shared meaning.

  • Keep the dense and sparse result lists separate until fusion.
  • Use a stable document or chunk identifier shared by both indexes.
  • Request more candidates from each retriever than the final number shown to a user.

Fuse ranked lists with reciprocal rank fusion

For each candidate document d, RRF assigns a combined score based on its rank in every result list: RRF(d) = Σ 1 / (k + rank_i(d)). A document returned by both regional S3 Vectors and Quickwit receives contributions from each list. A document that ranks highly in only one list can still remain competitive.

The constant k reduces the difference between nearby positions and prevents rank one from dominating the entire result set. A commonly used starting point is k = 60, but it is a tuning parameter rather than a universal rule. Evaluate it against the queries and relevance judgments that reflect the intended workload.

  • Assign ranks starting at 1, not 0.
  • Deduplicate by canonical document or chunk ID before sorting fused results.
  • Treat a missing result from one retriever as no contribution, not as rank zero.
  • Use a deterministic secondary sort, such as document ID, for equal fused scores.

Choose candidate depth and evaluate the fused result set

RRF can only promote documents that at least one retriever returned. Candidate depth therefore matters: retrieving a final top 10 from each source and then asking fusion for a top 10 leaves little room for complementary results. Over-fetch from both dense and sparse retrieval, fuse the union, and then return the final cutoff.

Evaluate dense-only, BM25-only, and fused retrieval on the same query set. Break results down by query type, including natural-language questions, exact identifiers, short keyword queries, and domain-specific terminology. The useful outcome is not merely a higher aggregate metric; it is knowing which retriever contributes relevant candidates for each search behavior.

  • Start with an equal candidate depth from dense and sparse retrieval.
  • Measure recall-oriented metrics at the fusion depth before optimizing final ranking.
  • Inspect queries where only one retriever found a relevant result.
  • Log retriever rank contributions so fused outcomes can be explained during debugging.