Why raw score merging is fragile

A dense-search score and a BM25 score are produced by different ranking functions. Their ranges, distributions, and sensitivity to query length can differ substantially. A simple rule such as adding the two scores can make one retriever dominate for reasons unrelated to actual relevance.

Score normalization can help in carefully measured environments, but it adds assumptions: which normalization method to use, what historical query distribution represents production traffic, and how to handle score shifts after corpus or model changes. Rank-based fusion avoids much of that coupling.

  • BM25 rewards term-frequency and corpus-statistics signals.
  • Dense retrieval ranks by vector similarity in an embedding space.
  • The numeric scores need not be directly comparable.
  • A rank position is easier to combine across distinct retrieval methods.

Fuse the dense and sparse candidate lists with RRF

Run the same user query through the dense path and the sparse BM25 path, then take a bounded candidate list from each. For every document appearing in either list, add a reciprocal contribution based on its rank in each list: RRF(d) = Σ 1 / (k + rank_i(d)).

The constant k reduces the gap between adjacent top ranks and prevents a single first-place result from overwhelming the rest of the merged list. A commonly used starting value is 60, but it should be treated as a tuning parameter rather than a universal optimum.

  • Use one-based ranks: the top result has rank 1.
  • Assign no contribution when a document is absent from a list.
  • Deduplicate by a stable document identifier before returning results.
  • Keep the dense and sparse candidate depths explicit and configurable.

Evaluate the fused list against failure modes

Create an evaluation set that includes both semantic queries and exact-match-heavy queries. For example, natural-language questions can test semantic coverage, while queries containing error codes, SKU-like strings, or version names can test whether sparse retrieval retains important lexical matches.

Inspect more than an aggregate relevance metric. Look at queries where one retriever succeeds and the other fails, then confirm that fusion improves the combined ranking rather than masking a useful specialized signal. This review also helps determine whether candidate depth or the RRF constant deserves adjustment.

  • Compare dense-only, BM25-only, and fused rankings on the same judged queries.
  • Track whether relevant results appear in the candidate sets before fusion.
  • Review duplicate, stale, and identifier-mismatch cases in merged results.
  • Re-evaluate after meaningful corpus, embedding-model, or analyzer changes.