Why raw-score blending is brittle

A dense retriever and a BM25 retriever can both return a value called a score, but those values need not mean the same thing. Their ranges, distributions, and sensitivity to query length may differ. A weighted expression such as dense_score + sparse_score can therefore change behavior substantially when either backend, corpus, or query mix changes.

Score normalization can help, but it introduces more choices: normalize per query or globally, use min-max or z-scores, and decide how to handle outliers. Before adding those tuning variables, use a method that relies only on an ordering that each retriever already produces.

  • Dense retrieval can surface semantically related wording.
  • BM25 can preserve exact-term matches for tokens such as IDs and error messages.
  • A rank is meaningful within one result list even when scores are not comparable across lists.

Fuse candidate lists with reciprocal rank fusion

Run the same query through dense retrieval and BM25, request a candidate list from each, and assign every document an RRF score. For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is its one-based position in retrieval list i. Sum only over lists that contain the document.

The constant k reduces the impact of a single first-place result and makes the fusion less dominated by small rank differences at the top. A commonly used starting value is 60, but it is a parameter to validate against representative queries rather than a universal optimum. Sort documents by their final RRF score, then return the top results or pass that shortlist to a later reranking stage.

  • Deduplicate documents by a stable document ID before emitting results.
  • Use one-based ranks: the first item has rank 1.
  • Retrieve enough candidates from both sources to allow overlap and complementary matches.
  • Keep the dense and BM25 lists available in logs for relevance debugging.

Evaluate complementarity, not just aggregate relevance

RRF is most valuable when the two retrieval paths contribute different useful documents. Build a small evaluation set containing paraphrase-heavy questions, exact-name lookups, identifier queries, and queries with ambiguous terminology. For each query, compare dense-only, BM25-only, and fused top-k results against the same relevance judgments.

When a fused result looks wrong, inspect its provenance. A document that ranks moderately in both lists may rightly rise through fusion; a document appearing only at the tail of one list may indicate that the candidate depth is too large, the source retrieval needs tuning, or the query class should be handled differently. This inspection turns hybrid retrieval from a black box into an engineering feedback loop.

  • Track whether a relevant result came from dense retrieval, BM25, or both.
  • Review zero-result and exact-token queries separately from natural-language questions.
  • Test candidate-list depth and k independently before adding more complex score calibration.
  • Use unchanged relevance judgments when comparing retrieval configurations.