Why raw-score blending is fragile

A dense retriever and a BM25 retriever may each return a score, but those values usually have different meanings. A dense score commonly reflects geometric similarity between embeddings, while BM25 is based on term statistics such as frequency and document-length normalization. Even when both scores increase for better matches, their scales and distributions need not align.

A fixed formula such as 0.5 × dense_score + 0.5 × bm25_score can therefore produce unstable behavior. A small change in embedding model, index configuration, corpus composition, or query vocabulary may shift one score distribution enough to dominate the combined ranking. Calibrating scores can be useful, but it requires representative judgments and ongoing validation.

  • Dense retrieval helps with semantic paraphrases and related concepts.
  • BM25 helps with exact terminology, codes, names, and uncommon tokens.
  • Raw scores from separate retrieval systems should not be assumed comparable.
  • A fusion method should remain understandable when either retriever changes.

Fuse ranked lists with reciprocal rank fusion

Reciprocal rank fusion (RRF) combines result lists using position rather than raw score. Run the dense query against the vector index and the lexical query against the BM25 index, then assign each returned document a contribution based on its rank in each list. Documents that appear near the top of one or both lists accumulate the strongest final score.

For a document d, a common RRF formulation is: RRF(d) = Σ 1 / (k + rank_i(d)). The sum is taken across result lists, rank_i(d) is the one-based position of d in list i, and k is a positive constant that reduces the difference between adjacent ranks. The important property is that the method only needs ordering: it does not require dense and sparse scores to share a scale.

  • Retrieve a bounded candidate list from S3 Vectors for the dense query.
  • Retrieve a bounded candidate list from Quickwit BM25 for the lexical query.
  • Deduplicate results by a stable document or chunk identifier.
  • Sum reciprocal-rank contributions and sort descending for the fused list.

Make fusion operationally reliable

Use the same retrieval unit in both systems whenever possible. If the dense index stores passage-level chunks while the sparse index returns whole documents, RRF will mix entities with different granularity. A stable shared identifier, such as a chunk ID plus document metadata, makes deduplication and later inspection much easier.

Start with equal treatment of the two ranked lists and evaluate using a query set that reflects real traffic: semantic questions, exact-lookups, mixed queries, and queries with abbreviations. Review not only whether a relevant item appears, but where it appears. If tuning is needed, adjust candidate depths, list weights, or the RRF constant deliberately and record the reason for each change.

  • Log dense rank, BM25 rank, and fused rank for returned items.
  • Keep document IDs and chunk boundaries consistent across both indexes.
  • Test failure cases, including rare terms and paraphrased questions.
  • Treat fusion settings as versioned retrieval configuration, not hidden application logic.