Why raw-score blending is fragile

A dense search result is typically ordered by a vector similarity or distance calculation. A BM25 result is ordered by a lexical relevance formula based on term occurrence and document statistics. Both scores are useful within their own retrieval systems, but their numeric ranges and distributions can differ substantially.

Adding the two scores together with a fixed weight can work in a narrowly controlled corpus, but it creates tuning pressure. A query with a strong exact term match may produce a different BM25 score distribution than a broad natural-language query, while dense score distributions can also shift with embedding models, chunking, and corpus composition. Rank-based fusion avoids requiring those raw values to mean the same thing.

  • Dense retrieval can surface paraphrases and conceptually related passages.
  • BM25 can prioritize exact identifiers, rare terms, quoted phrases, and spelling-sensitive queries.
  • Raw scores should be treated as engine-specific ranking signals unless they have been deliberately calibrated.
  • A fusion method should preserve useful candidates from both retrieval paths.

Fuse result lists with reciprocal rank fusion

RRF assigns each document a contribution based on its position in each ranked list. For a document at rank r, the contribution is 1 divided by k plus r, where k is a positive constant. The final RRF score is the sum of that contribution across the dense and sparse lists in which the document appears.

For example, if a passage ranks second in dense retrieval and tenth in BM25, it receives contributions from both rankings. A passage found only by BM25 can still appear in the final list, and a passage found by both systems is rewarded for agreement. The constant k softens the difference between neighboring ranks so that fusion is not dominated by the first few positions of one list.

  • Retrieve a candidate set from regional S3 Vectors for the dense path.
  • Retrieve a candidate set from Quickwit BM25 for the sparse path.
  • Deduplicate candidates using a stable document or chunk identifier.
  • Compute: RRF(document) = Σ 1 / (k + rank), then sort descending.

Make fusion observable and easy to revise

Store more than the final fused rank during evaluation. Record whether each candidate came from dense retrieval, BM25, or both; retain its rank in each list; and keep the fused score. This makes it possible to diagnose retrieval behavior when an expected passage is absent or appears too low.

Start with a representative query set rather than optimizing for a single query style. Include entity lookups, product names, error messages, conceptual questions, and longer natural-language requests. Review failures separately: some indicate missing or poorly chunked source content, while others indicate that candidate depth, query processing, or ranking needs adjustment.

  • Use the same chunk identifier in both indexes so fusion can reliably join candidates.
  • Choose candidate depths large enough to let each retriever contribute, then evaluate the combined list.
  • Log per-query dense rank, BM25 rank, fused rank, and retrieval source.
  • Treat the RRF constant and candidate depths as evaluation parameters, not permanent assumptions.