Why raw dense and sparse scores should not be added

A dense-search score and a BM25 score are produced by different models and different assumptions. Their numeric ranges, distributions, and meanings are not inherently comparable. A document with a strong BM25 score is not automatically more relevant than a document with a high vector-similarity score, even when both are returned for the same query.

Adding raw scores can make ranking fragile. Small changes to embedding models, index settings, query length, or corpus composition may change score distributions and silently shift which retrieval method dominates. That makes relevance tuning depend on score calibration rather than on the underlying quality of each candidate list.

  • Dense retrieval is useful for semantic similarity and paraphrases.
  • BM25 is useful for lexical matches, uncommon terms, and exact identifiers.
  • Score values from the two systems should be treated as method-specific signals.
  • Candidate rank is often more portable than candidate score.

Fuse ranked lists with reciprocal rank fusion

Reciprocal rank fusion (RRF) combines lists using position rather than raw score. For each document, add a contribution from every list in which it appears: 1 divided by a constant plus the document's rank. The final fused score is the sum of those contributions, and documents are sorted by that sum.

For example, if a document is ranked highly by both S3 Vectors and Quickwit BM25, it receives two contributions and moves upward. A document found only by one method can still appear, which preserves the complementary recall of dense and sparse retrieval. The constant, commonly written as k, reduces the difference between nearby ranks and prevents the first position from overwhelming the rest of the candidate set.

  • Run dense and sparse retrieval independently for the same query.
  • Keep each result's document ID and one-based rank.
  • Compute: RRF(document) = sum of 1 / (k + rank) across lists.
  • Deduplicate by document ID, sort by fused score, then return the top results.

Make fusion operationally reliable

Start with equal weighting and retrieve a reasonably sized candidate set from each method. Equal weighting is a useful baseline because it reveals whether the two retrieval paths contribute distinct results before introducing more tuning variables. If one path is unavailable or returns no candidates, return the ranked results from the other path rather than failing the query.

Evaluate fusion with representative queries, including natural-language questions, product names, error messages, acronyms, and long-tail identifiers. Inspect not only the top result but also whether relevant documents enter the candidate set. If your application has relevance judgments or user feedback, use them to compare candidate depth, weighting choices, and any downstream reranking strategy.

  • Use stable document IDs across dense and sparse indexes.
  • Apply the same access-control and filtering rules before or during both retrieval paths.
  • Log source ranks alongside the fused rank for debugging.
  • Tune list depth, k, and optional source weights only against representative evaluation data.