Why raw-score blending is fragile

A dense-search score and a BM25 score are produced by different ranking functions. Their ranges, distributions, and sensitivity to query length can differ substantially. Adding them together with fixed weights can work for a narrow evaluation set, but it often creates tuning work whenever the corpus, embedding model, analyzer, or query mix changes.

Reciprocal Rank Fusion (RRF) avoids comparing raw scores. Instead, it uses each system’s rank position. A document that appears near the top of both lists receives a stronger combined signal than a document that appears highly in only one list.

  • Dense retrieval captures semantic similarity and paraphrases.
  • BM25 preserves lexical precision for terms such as product names, IDs, and codes.
  • Rank positions are easier to combine than unrelated score scales.

Fuse independently retrieved candidate lists

For a query, retrieve a candidate list from dense search and another from BM25. Deduplicate documents by a stable document identifier, then calculate an RRF score for every document that appears in either list. The common formulation is RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the document’s one-based rank in result list i and k is a positive constant.

Choose the same candidate depth for both retrieval paths initially, then evaluate on representative queries. A larger candidate depth gives the fusion step more opportunities to recover documents that are useful to only one retriever, while also increasing downstream work if results are reranked or passed to an answer-generation system.

  • Use one stable ID across dense and sparse indexes.
  • Treat a missing document from a list as no contribution from that list.
  • Apply the same filters, such as tenant or access constraints, before fusion.
  • Sort by fused score and keep a final top-k result set.

Make fusion observable and testable

Log more than the final ranking. For each returned document, retain its dense rank, BM25 rank, and fused score. This makes it possible to identify whether a result won because both retrievers agreed or because one retriever surfaced an otherwise missed item. It also helps diagnose regressions after changing tokenization, embeddings, metadata filters, or document chunking.

Evaluate RRF with a small labeled query set that includes semantic questions, exact-term lookups, mixed queries, and queries with filters. Compare dense-only, BM25-only, and fused retrieval using the same relevance judgments. The goal is not to assume hybrid retrieval is always better, but to learn which query classes benefit from combining signals.

  • Include rare identifiers and error messages in evaluation queries.
  • Inspect queries where dense and sparse top results do not overlap.
  • Version retrieval settings alongside embedding and indexing changes.
  • Keep fusion deterministic so ranking changes can be explained.