Why raw-score blending is brittle

A dense-search score reflects the relationship between an embedded query and embedded documents. A BM25 score reflects lexical term matching, term frequency, document frequency, and length normalization. Even when both systems return numeric scores, the values have different meanings and may change with corpus composition, query wording, embedding models, or index settings.

A formula such as 0.5 × dense_score + 0.5 × bm25_score therefore creates an implicit assumption: that a one-point change in each score has equivalent ranking significance. That assumption is often difficult to justify. Rank fusion makes a smaller assumption: a high placement in either candidate list is useful evidence.

  • Dense retrieval helps when the query and relevant text use different language.
  • BM25 helps preserve exact-token matches such as part numbers, error codes, and names.
  • Raw scores should not be treated as interchangeable merely because both are numbers.

Fuse ranks with a simple RRF formula

For each query, retrieve a candidate list from dense search and another from BM25. Assign each document a rank in each list, where rank 1 is the first result. RRF gives a document a combined score based on the inverse of its rank in every list where it appears.

A common formulation is RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs over result lists, and k is a positive constant that reduces the difference between nearby ranks. Documents returned by both retrieval methods accumulate evidence; documents that rank very highly in one method can still remain competitive.

  • Request a fixed candidate depth from both dense and BM25 retrieval.
  • Use a stable document identifier to join results across the two lists.
  • Choose one k value, record it with the query configuration, and tune it with relevance judgments rather than intuition alone.
  • Apply a deterministic tie-breaker, such as document ID, for reproducible output.

Build the fusion layer as a testable retrieval step

Keep dense retrieval, BM25 retrieval, and fusion observable as separate stages. For every result, record whether it came from dense search, sparse search, or both, along with its source ranks and final fused rank. This makes it possible to diagnose why an exact-match result rose or why a semantically related result fell.

Evaluate representative query groups instead of relying on a single aggregate number. Include terminology-heavy queries, natural-language questions, short ambiguous queries, and queries containing identifiers. The goal is not to make both retrieval methods agree; it is to make their differences contribute useful candidates before the application presents or reranks them.

  • Log source ranks and fused ranks for debugging.
  • Maintain a small labeled query set before changing candidate depth or k.
  • Inspect queries where only one retrieval path produced relevant documents.
  • Treat metadata filters as part of the retrieval contract and apply them consistently to both paths.