Why raw-score mixing is fragile

Dense retrieval scores and BM25 scores are produced by different ranking models. Their ranges, distributions, and meanings are not inherently aligned: a score that appears large in one system is not automatically more important than a smaller-looking score in the other.

A common mistake is to add or average the two scores directly. That approach can work only after careful calibration on representative queries, and the calibration may drift as content, embedding models, analyzers, or retrieval settings change.

  • Dense retrieval emphasizes semantic proximity in embedding space.
  • BM25 emphasizes lexical evidence from query terms and document terms.
  • Score scales can change independently across the two retrieval systems.
  • Rank positions are usually easier to combine than raw scores.

Fuse ranks with reciprocal rank fusion

Run the same user query through a dense retrieval path and a BM25 retrieval path. Keep the ordered document IDs from each result set, then calculate an RRF score for every document that appears in either list.

For each list, add 1 divided by k plus the document rank. The constant k reduces the advantage of being ranked first in a single list and makes the method less sensitive to small rank differences near the top.

  • Formula: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Use one term in the sum for each retrieval list containing document d.
  • Choose a fixed k, often starting with 60, then validate it against your own relevance judgments.
  • Sort documents by their summed RRF score and return the top results.

Implement the pattern at the application layer

An API-first application can issue dense and sparse requests concurrently, collect the top N results from regional S3 Vectors and Quickwit BM25, and perform fusion after both responses arrive. Deduplicate by a stable document or chunk ID before rendering the final ranking.

Start with the same candidate depth for both paths, such as the top 50 results, then inspect query classes rather than relying on a single aggregate measure. Product names, error codes, policy language, and natural-language questions often reveal where one retrieval method contributes results the other misses.

  • Use stable IDs so the same chunk can receive contributions from both lists.
  • Log each document's dense rank, sparse rank, and fused rank for debugging.
  • Evaluate representative queries with human relevance labels when possible.
  • Treat candidate depth and k as tunable retrieval parameters, not universal constants.