Why raw-score mixing is a fragile default

A dense retrieval score and a BM25 score are produced by different ranking models. Their ranges, distributions, and sensitivity to query length can differ substantially. Adding them directly, or applying a fixed weight before understanding those differences, can make one retrieval path dominate for reasons unrelated to relevance.

This is particularly visible in mixed query traffic. A natural-language question may benefit from semantic matching, while a query containing a product code, error string, or quoted phrase may depend on lexical matches. A hybrid system should preserve useful candidates from both paths rather than assume a single score scale can represent both.

  • Dense retrieval ranks documents by vector similarity.
  • BM25 ranks documents using term-based matching and weighting.
  • Score magnitudes are model-specific, not automatically interchangeable.
  • Normalize or calibrate scores only when you have evidence that the method is stable for your corpus.

Apply reciprocal rank fusion to the two result lists

RRF combines rankings rather than raw scores. Retrieve a candidate list from dense search and another from BM25, then assign each document a contribution based on its position in each list. Documents that rank well in either list receive credit; documents supported by both lists accumulate more credit.

For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank_i(d), for every ranking i in which d appears. The constant k reduces the impact of small position differences near the top of a list. Its value is a tuning choice, so start with a documented default and evaluate changes against representative queries.

  • Retrieve top-N candidates from regional S3 Vectors for dense search.
  • Retrieve top-N candidates from Quickwit BM25 for sparse search.
  • Use a stable document identifier to deduplicate candidates across lists.
  • Sort the union of candidates by descending RRF score before returning the final top-K.

Make fusion observable before making it complex

The first implementation should record enough information to explain each result: whether it came from dense retrieval, BM25, or both; its rank in each source list; and its final fused rank. This makes failures diagnosable. For example, an exact identifier missed by dense retrieval can still be traced to sparse retrieval, while a terminology mismatch may be recovered by the dense path.

Evaluate with a small, deliberately varied query set before introducing learned rerankers or aggressive score transformations. Include exact-name searches, abbreviations, long questions, domain terms, and ambiguous requests. Review not only whether a relevant document appears, but whether it reaches the position users will actually inspect.

  • Track source-list ranks and fused rank for every returned document.
  • Measure recall at the candidate depth used before fusion.
  • Inspect queries where only one retrieval path contributes relevant results.
  • Tune candidate depth, final top-K, and k separately rather than changing all three at once.