Why raw score merging is fragile

A dense-search score and a BM25 score are produced by different retrieval methods. Their ranges, distributions, and sensitivity to query length can differ substantially. Adding them together, or selecting whichever score is numerically larger, creates an implicit assumption that the scores share a common scale.

That assumption can fail quietly. A query containing a product code may benefit strongly from sparse retrieval, while a natural-language question may benefit more from dense retrieval. Rank-based fusion avoids requiring either system’s scores to be directly comparable.

  • BM25 emphasizes term matches and term statistics.
  • Dense retrieval ranks by vector similarity.
  • A score of 0.8 in one method is not inherently equivalent to 0.8 in another.
  • Rank positions are easier to combine than unrelated score scales.

Fuse the two ranked lists with RRF

Reciprocal Rank Fusion, or RRF, assigns each document a contribution based on its position in every result list where it appears. For a document d, calculate RRF(d) as the sum of 1 divided by k plus its rank in each list. The constant k reduces the influence of small rank differences near the top.

For example, retrieve a candidate list from regional S3 Vectors and another from Quickwit BM25. Normalize document identifiers before joining the lists, compute each document’s fused score, then sort descending by that score. A document appearing in both lists is naturally rewarded, while a strong result from only one method can still remain competitive.

  • Use one stable document or chunk identifier across dense and sparse indexing.
  • Treat rank as one-based: the first result has rank 1.
  • Compute: fused_score = Σ 1 / (k + rank_i).
  • Pick k through offline relevance evaluation rather than score-range inspection.

Make fusion operationally reliable

Hybrid retrieval quality depends on candidate hygiene as much as on the fusion formula. Apply the same tenant, access-control, language, and content-state filters to both retrieval paths before fusion. Otherwise, a document excluded from one index but present in the other can create confusing or unsafe results.

Also decide where deduplication occurs. If several chunks belong to the same parent document, fuse chunk-level results first when passages are the response unit. If the interface presents documents, aggregate or select the best chunk per parent after fusion so one document does not occupy several top positions.

  • Log dense rank, BM25 rank, and fused rank for inspected queries.
  • Evaluate queries with exact terms, paraphrases, acronyms, and mixed-language wording.
  • Keep index versions aligned when documents are added, changed, or deleted.
  • Use the same identifier mapping when joining results from both retrieval systems.