Why raw dense and BM25 scores should not be casually added

A dense-search score and a BM25 score are produced by different retrieval models and have different distributions. A larger value in one system does not inherently represent the same relevance confidence as a larger value in the other.

Adding those values with a fixed weight can work only after careful calibration, and that calibration may change with embedding models, corpus composition, query language, or sparse-search settings. Rank-based fusion avoids making that assumption.

  • Dense retrieval is useful when relevant passages use different wording from the query.
  • BM25 is useful when exact terms, identifiers, names, and rare vocabulary matter.
  • Score ranges alone are not a reliable shared relevance scale.

Fuse ranks with the RRF formula

RRF assigns each document a contribution based on its position in every ranked list where it appears. For a document d, compute RRF(d) as the sum of 1 divided by k plus rank(d) across the lists. The constant k reduces the influence of the very top ranks and is typically selected as an application-level tuning parameter.

For example, request a candidate list from dense search backed by regional S3 Vectors and another from Quickwit BM25. Use document IDs to merge duplicates, add each list's rank contribution, then sort documents by their combined RRF score.

  • Use one-based ranks: the first result has rank 1.
  • Only add a contribution when a document appears in a result list.
  • Deduplicate by a stable document or chunk identifier before returning results.
  • Keep the source ranks for debugging and relevance review.

Make fusion observable before making it complicated

Start with a fixed candidate depth for both retrieval paths and inspect which source contributed to the final results. Queries containing product codes or quoted phrases may lean on BM25, while descriptive questions may receive more useful candidates from dense search.

Evaluate the fused list using a small, representative query set with relevance judgments or human review. If results are weak, first check chunking, metadata filters, query construction, and candidate depth before introducing more scoring logic.

  • Log the query, final document IDs, source ranks, and fusion rank.
  • Track cases found by dense search only, BM25 only, and both paths.
  • Apply the same access-control and metadata-filtering rules to every candidate path.
  • Treat k and per-path candidate depth as retrievable configuration, not permanent constants.