Why raw dense and sparse scores should not be averaged

A dense-search score and a BM25 score are produced by different ranking functions. Their ranges, distributions, and sensitivity to query length can differ substantially. A score of 0.8 from one retrieval system does not inherently mean the same thing as a score of 0.8 from another.

A weighted score average can still be appropriate when scores have been carefully calibrated against representative relevance data. Without that work, however, a simple average can accidentally let one retrieval path dominate because of score scale rather than result quality.

  • Dense retrieval is useful for paraphrases and conceptual similarity.
  • BM25 is useful for exact names, error codes, product terms, and rare tokens.
  • Score normalization rules can drift as content, embedding models, or indexes change.
  • Rank positions are often more stable inputs for a first hybrid retrieval implementation.

Fuse independently retrieved lists with reciprocal rank fusion

Reciprocal rank fusion, or RRF, combines ranked lists instead of combining raw scores. Retrieve a candidate list from dense search and another from BM25, then assign each document a contribution based on its position in each list. Documents returned by both paths accumulate contributions and tend to move upward.

For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) for every list containing d. The constant k reduces the difference between adjacent high ranks and prevents the first position in one list from overwhelming all other evidence. A commonly used starting value is 60, but it should be treated as a tuning parameter rather than a universal rule.

  • Request a bounded candidate set from regional S3 Vectors for dense retrieval.
  • Request a bounded candidate set from Quickwit BM25 for sparse retrieval.
  • Deduplicate documents using a stable document or chunk identifier.
  • Sum RRF contributions, sort descending, and return the top fused results.

Make fusion observable before making it sophisticated

Log the source rank and fused rank for each selected result. This makes it possible to see whether a result was supported by dense retrieval, BM25, or both. It also helps identify query classes where one path is consistently absent or where candidate limits are too small.

Start with a small evaluation set containing realistic searches: exact identifiers, natural-language questions, mixed terminology, and queries with abbreviations. Review the top results for each retrieval path and for the fused list. If a class of queries is weak, adjust candidate depth or apply query-aware routing before introducing score calibration.

  • Track overlap between dense and BM25 candidate lists.
  • Record rank contributions alongside the final fused order.
  • Evaluate exact-match and semantic-query slices separately.
  • Change one variable at a time: candidate depth, k, or result cutoff.