Why raw dense and sparse scores should not be added directly

A dense-search score and a BM25 score are generated by different retrieval models and have different distributions. Even when both systems rank highly relevant documents near the top, a score of 0.8 in one list does not have an inherent relationship to a score of 8.0 in the other.

Min-max scaling, z-scores, or hand-selected weights can work in controlled cases, but they depend on score behavior remaining stable across queries, collections, and index changes. That makes direct score blending a maintenance concern rather than a simple ranking operation.

  • Dense retrieval is useful for semantic similarity and paraphrased intent.
  • BM25 is useful for exact phrases, rare terms, codes, names, and lexical constraints.
  • Rank position is comparable even when raw scoring functions are not.

Fuse ranked lists with RRF

Run the same user query through dense retrieval in regional S3 Vectors and sparse retrieval in Quickwit BM25. Request a sufficiently deep candidate list from each system, then merge results using a stable document identifier shared by both indexes.

For every document returned by either search, add 1 / (k + rank) for each list in which it appears. Rank starts at 1, and k is a positive constant that reduces the gap between adjacent top positions. The final score is the sum of those contributions.

  • RRF score: score(document) = Σ 1 / (k + rank_i).
  • A document appearing in both lists receives contributions from both rankings.
  • A document high in one list can still remain competitive when the other method misses it.
  • Use a deterministic tie-breaker, such as document ID or a secondary retrieval rank.

Make the fusion layer observable and easy to revise

Keep retrieval and fusion as separate steps in your application or service. Store the dense rank, sparse rank, fused score, and final rank for sampled queries. This makes it possible to explain why an item appeared and to identify cases where one retrieval method dominates unexpectedly.

Start with equal treatment of the two ranked lists. If evaluation later shows that a query class needs different behavior, route that class to a distinct policy or apply a documented weight to one source's RRF contribution. Make changes against relevance judgments or curated query sets, not isolated anecdotes.

  • Log candidate counts and ranks from both retrieval paths.
  • Confirm that document IDs resolve consistently across dense and sparse indexes.
  • Evaluate exact-term, semantic, and mixed-intent queries separately.
  • Revisit candidate depth and the RRF constant when ranking behavior changes.