Why raw score blending is often fragile

Dense and sparse retrieval systems generally produce scores with different meanings and ranges. A dense similarity score reflects a relationship between embedding vectors, while BM25 is based on term frequency, document frequency, and document length. Treating those numbers as directly interchangeable can make a weighted sum sensitive to query type, corpus changes, and implementation details.

Score normalization can be useful in a controlled system, but it introduces choices about distributions, clipping, and calibration data. RRF takes a simpler route: it uses each retriever’s ranking order. This makes it a good baseline when building a hybrid endpoint or evaluating whether combining retrieval signals improves the candidate set.

  • Dense retrieval can surface paraphrases and conceptually related content.
  • BM25 can strongly reward distinctive query terms, identifiers, and exact phrases.
  • Rank positions are easier to combine than scores with unrelated scales.

Fuse the dense and BM25 lists with RRF

Run dense search against the vector index and BM25 search against the sparse index using the same document identifiers. For every returned document, add a contribution based on its rank in each list. A common form is RRF(d) = Σ 1 / (k + rank_i(d)), where rank starts at 1 and k is a positive constant.

The constant k reduces the difference between nearby positions and prevents the top result from dominating solely because it is ranked first in one list. Keep k fixed during an initial evaluation so that changes in relevance can be attributed to retrieval inputs or candidate depth rather than multiple moving parameters.

  • Request a candidate list from both dense and BM25 retrieval paths.
  • Deduplicate results by a stable document or chunk ID.
  • Add one RRF contribution for each list in which a document appears.
  • Sort documents by the combined RRF score and return the top results.

Evaluate by query class, not only aggregate relevance

Hybrid retrieval is most informative when evaluated across query classes. Include exact-name queries, error messages, product terminology, natural-language questions, and queries that use wording different from the source text. Compare dense-only, BM25-only, and RRF-fused candidate lists with the same result depth.

Inspect failures as well as wins. If BM25 results dominate identifier-heavy queries, that may be expected. If dense retrieval finds relevant paraphrases that BM25 misses, RRF can preserve both signals. Use those observations to decide candidate depths, metadata filters, and whether certain query types should receive specialized treatment.

  • Create a labeled query set representative of real user language.
  • Record whether a relevant document appears in the top candidate window.
  • Review duplicates caused by chunking or repeated source content.
  • Apply metadata filtering before fusion when a query has clear scope constraints.