Why raw dense and sparse scores should not be added

Dense retrieval ranks documents by proximity between embeddings. Sparse retrieval with BM25 ranks documents from term-level evidence, including term frequency and collection statistics. Even when both systems return a numeric score, the values describe different ranking functions.

Adding those values directly introduces an implicit assumption: that a one-unit change in a dense score means the same thing as a one-unit change in a BM25 score. That assumption is usually untested and can change as embeddings, analyzers, document collections, or query mixes change.

  • Dense retrieval can recover semantic paraphrases that share few query terms.
  • BM25 can strongly reward exact identifiers, names, codes, and rare terms.
  • Score ranges alone do not establish comparable relevance meaning.
  • A rank-based method avoids choosing a global score conversion formula.

Fuse ranked lists with RRF

Run the dense and sparse retrieval paths independently, then retain an ordered candidate list from each. For every document that appears in either list, compute an RRF score: RRF(d) = Σ 1 / (k + rank_i(d)). The sum is taken over the lists in which document d appears.

The constant k reduces the advantage of a document appearing at the very first rank. A larger k makes differences among early ranks less sharp; a smaller k gives more weight to the top of each list. Treat k and the depth fetched from each retriever as explicit application configuration, not hidden defaults.

  • Use one-based ranks: the first returned document has rank 1.
  • Deduplicate by a stable document identifier before presenting results.
  • Fetch enough candidates from both lists for overlap and complementary matches to matter.
  • Sort documents by descending fused score, with a deterministic tie-breaker.

Evaluate the fusion policy on the queries you expect

RRF is useful because it removes score-scale calibration from the first version of a hybrid retrieval pipeline, not because it eliminates evaluation. Build a small query set that includes exact-term lookups, natural-language questions, mixed identifier-and-description queries, and terms likely to have synonyms.

For each query, inspect whether the combined ranking preserves strong exact matches while introducing relevant semantic alternatives. Compare dense-only, BM25-only, and fused lists. When a fused result is surprising, determine whether the issue is candidate generation, document content, identifiers, or the fusion policy before changing constants.

  • Record the source rank from dense retrieval and BM25 for each fused result.
  • Keep query sets versioned as documents and relevance judgments evolve.
  • Review zero-result and low-overlap cases separately from ordinary queries.
  • Use the same evaluation set when changing embeddings, tokenization, or retrieval depth.