Why raw-score blending is risky

A dense retriever and a BM25 retriever may both return a numeric score, but the numbers represent different scoring systems. Their ranges, distributions, and sensitivity to query length can differ substantially. A dense score of 0.8 is not inherently equivalent to a BM25 score of 8.

Normalizing scores can help in controlled settings, but it introduces another system to tune and monitor. Changes in embeddings, indexing, corpus composition, or BM25 configuration can alter score distributions and quietly change the balance between retrieval paths.

  • Do not assume dense and sparse scores share a common scale.
  • Avoid selecting weights solely because they work for a small set of example queries.
  • Treat ranking order as more portable than score magnitude when combining independent retrievers.

Fuse result ranks with RRF

RRF assigns each document a contribution based on its position in each ranked list. For a document at rank r, the contribution is 1 divided by k plus r. Sum that contribution across the dense and sparse lists, then sort documents by the total.

The constant k reduces the advantage of appearing at the very top of only one list. A common starting value is 60, but it is a policy choice rather than a universal optimum. The useful property is that a document appearing in both lists is rewarded without requiring the systems' raw scores to match.

  • Request a candidate list from dense search and another from BM25 search.
  • Deduplicate by a stable document or chunk identifier.
  • For each appearance, add 1 / (k + rank), using ranks that start at 1.
  • Sort by the summed RRF score and return the top fused candidates.

Make fusion observable and query-aware

Start with the same candidate depth for both retrieval paths, then inspect which source contributes to the final results. Logging the dense rank, sparse rank, fused rank, and document identifier makes it possible to diagnose whether one path is being crowded out or whether duplicate chunks are consuming the candidate budget.

RRF is also a useful baseline for later experimentation. If evaluation data shows that exact identifiers, error codes, or product names need stronger lexical treatment, adjust retrieval depth or add a query classification rule. Keep the baseline intact so each change can be compared against a simple, understandable ranking policy.

  • Record source-specific ranks alongside the fused score.
  • Evaluate queries containing exact tokens as well as paraphrased intent.
  • Choose candidate depth with downstream context limits in mind.
  • Version fusion settings such as k, candidate depth, and deduplication rules.