Why raw score addition is fragile

A dense-search score and a BM25 score are not automatically measurements on the same scale. Their ranges, distributions, and sensitivity to query length can differ. Even when both are returned as numbers that increase with relevance, adding them directly assumes that one unit of dense score means the same thing as one unit of BM25 score.

That assumption can make retrieval behavior hard to reason about. A change in indexing, analyzer settings, embedding models, or query composition may shift one score distribution without changing the other. The resulting ranking can then change because of score scale rather than because documents became more relevant.

  • BM25 is driven by lexical term matches and corpus statistics.
  • Dense retrieval ranks by vector similarity in an embedding space.
  • Score magnitudes can vary across queries and retrieval implementations.
  • A fusion method should avoid relying on unvalidated score equivalence.

Fuse ranks with reciprocal rank fusion

RRF starts with two independently ranked candidate lists: one from dense retrieval and one from BM25. For each document, add a contribution based on its rank in each list. A common form is RRF(d) = Σ 1 / (k + rank_i(d)), where k is a positive constant and rank_i is the document’s one-based position in result list i.

Documents appearing near the top of either list receive meaningful credit, while documents returned by both methods accumulate credit from both. Because RRF uses ordinal position rather than the original score, it does not require dense and sparse scores to be normalized into a shared numeric range.

  • Retrieve a bounded candidate list from regional S3 Vectors.
  • Retrieve a bounded candidate list from Quickwit BM25.
  • Key results by a stable document or chunk identifier.
  • Sum rank-based contributions and sort documents by the fused value.

Choose candidate depth and inspect disagreements

Candidate depth is part of the retrieval design. If each source contributes too few results, fusion cannot rescue documents that were never retrieved. If lists are unnecessarily deep, downstream work increases and lower-ranked candidates may add noise. Start with a depth that fits the size of the result set you can evaluate, then inspect retrieval examples before changing several parameters at once.

Review queries where the dense and lexical lists disagree. An identifier-heavy query may benefit from the BM25 list, while a paraphrased question may be surfaced by dense retrieval. These examples help determine whether a problem is candidate generation, document chunking, metadata filtering, or ranking after fusion.

  • Log each document’s dense rank, BM25 rank, and fused rank.
  • Keep source-specific ranks available for debugging and evaluation.
  • Apply metadata and access filters consistently before presenting results.
  • Evaluate with representative queries, including exact-match and paraphrase cases.