Why raw-score fusion is fragile

Dense search and BM25 produce scores with different meanings. A dense score often represents a vector similarity measure, while BM25 reflects term frequency, document length, and corpus statistics. Even when both results are useful, adding their raw scores can over-weight one retriever for reasons unrelated to relevance.

Score distributions can also change as embeddings, analyzers, indexes, or document collections change. A fixed normalization rule may work for one query set and become unreliable after a routine indexing update. Rank-based fusion avoids requiring a shared score scale in the first place.

  • Dense retrieval can surface paraphrases and conceptually related content.
  • BM25 can preserve exact terms, identifiers, acronyms, and rare phrases.
  • Raw similarity and BM25 scores should not be assumed to be interchangeable.
  • Rank positions are easier to combine across independent retrieval systems.

Apply Reciprocal Rank Fusion to two candidate lists

For each query, request a bounded candidate list from the dense path and another from the sparse path. Assign each document an RRF score by summing 1 divided by k plus its one-based rank for every list in which it appears. The constant k reduces the influence of very high rank positions and is a tuning choice rather than a universal value.

For example, if a document is ranked second by dense retrieval and tenth by BM25, its fused score is 1/(k+2) + 1/(k+10). A document appearing in both lists gains evidence from both retrieval methods, while a strong result from only one list can still remain competitive.

  • Deduplicate candidates using a stable document identifier before returning results.
  • Use one-based ranks: the first result has rank 1.
  • Compute a contribution only when a document occurs in that retriever's candidate list.
  • Sort by fused score, then apply a deterministic tie-breaker such as document ID.

Make fusion observable and safe to iterate

Store enough retrieval metadata to explain every fused result: dense rank, sparse rank, fused score, and the query configuration used. This makes it possible to diagnose outcomes such as exact-match documents being excluded from the dense list or semantically relevant documents being buried by lexical matches.

Evaluate changes with representative queries that include natural-language questions, product names, error messages, codes, and multi-word exact phrases. Review both relevance and operational behavior, including candidate-list sizes and the consequences of missing or delayed results from either retrieval path.

  • Log ranks and source membership rather than relying only on final result order.
  • Keep dense and sparse candidate limits configurable.
  • Test fusion constants against a labeled or manually reviewed query set.
  • Define fallback behavior for cases where one retrieval path returns no candidates.