Why raw dense and sparse scores should not be added

A dense retrieval score and a BM25 score do not inherently mean the same thing. Their values depend on the similarity function, index configuration, document length effects, tokenization, and the query itself. A score of 0.7 from one retrieval path is therefore not automatically comparable with a score of 0.7 from another.

Adding uncalibrated scores can create brittle behavior. A small change to embedding generation, BM25 tuning, or corpus composition may shift one score distribution enough to dominate the combined ranking. This is particularly risky for mixed query sets, where some searches are semantic questions and others contain exact product names, error codes, or part numbers.

  • Use dense retrieval for semantic relatedness and paraphrases.
  • Use BM25 for lexical matches, rare terms, and exact identifiers.
  • Avoid assuming score ranges from separate engines share a common meaning.

Fuse ranks with a stable, simple formula

RRF combines positions in ranked lists rather than the underlying scores. For each document, add a contribution from every list in which it appears: RRF(d) = Σ 1 / (k + rank_i(d)). The constant k reduces the difference between nearby ranks and prevents the first position in one list from overwhelming all other evidence.

For example, retrieve a candidate list from regional S3 Vectors for dense search and a candidate list from Quickwit BM25 for sparse search. Normalize both results to the same canonical document or chunk ID, calculate the fused score, then sort descending. A document that ranks well in both lists tends to rise, while a strong result unique to either retriever can still remain competitive.

  • Choose a shared canonical ID before fusion; do not merge on display text.
  • Keep ranks one-based: first result has rank 1.
  • Apply a consistent k value and record it with the retrieval configuration.
  • Deduplicate repeated IDs within each individual result list before scoring.

Make fusion observable before making it complex

Start with a fixed candidate depth from each retriever and inspect the fused results for representative queries. Include queries with natural-language phrasing, domain terminology, abbreviations, quoted phrases, and identifiers. The goal is not merely to see whether a desired document appears, but to understand which retrieval path supplied it and how fusion changed its position.

Log the dense rank, sparse rank, fused score, and source availability for returned items. These fields make retrieval regressions diagnosable when embeddings, chunking, analyzers, or indexes change. If RRF is insufficient for a particular use case, those observations provide a better basis for later work such as query routing, field-specific retrieval, or a learned reranker.

  • Evaluate dense-only, sparse-only, and fused rankings on the same query set.
  • Inspect queries where one list has no matching candidate.
  • Version chunking, embeddings, BM25 analysis settings, and fusion parameters together.
  • Treat RRF as a transparent baseline rather than a substitute for relevance evaluation.