Why raw-score blending is fragile

A dense-search score and a BM25 score do not necessarily represent the same quantity or range. Their values can change with the embedding model, similarity metric, corpus composition, analyzer settings, query length, and retrieval implementation. Adding them together with a fixed weight can therefore make ranking behavior difficult to predict.

Rank-based fusion avoids this calibration problem. Instead of asking whether a dense score of one value is equivalent to a BM25 score of another value, it asks a simpler question: which documents appeared near the top of each independently useful result list?

  • Use dense retrieval for semantic similarity and paraphrased intent.
  • Use BM25 for literal terms, identifiers, acronyms, and rare vocabulary.
  • Treat each retriever’s rank as the input to fusion, not its raw score.

Fuse candidate lists with RRF

Run the same user query through dense retrieval and BM25, keeping a bounded candidate list from each. For every document that appears in either list, calculate an RRF score: score(d) = Σ 1 / (k + rank_i(d)). The sum is taken across retrieval lists in which the document appears, and rank starts at one.

The constant k reduces the advantage of being ranked first and makes fusion less sensitive to small rank changes near the top. A document returned strongly by both retrievers will accumulate contributions from both lists. A document found by only one retriever can still remain in contention, which preserves complementary recall.

  • Deduplicate candidates by a stable document or chunk identifier.
  • Keep rank positions from each list before sorting the merged candidates.
  • Choose k deliberately and keep it consistent while evaluating changes.
  • Apply deterministic tie-breaking, such as document ID or the best individual rank.

Make fusion observable and test it on query slices

Log more than the final ranked list. Record which retrieval paths returned each result, its rank in each path, the fused score, and the query category when available. These fields make it possible to diagnose whether a poor result came from candidate generation, fusion, or a downstream reranker or answer-generation stage.

Evaluate representative query slices rather than relying on a single aggregate impression. Include exact product names, error messages, internal codes, short ambiguous queries, natural-language questions, and queries containing vocabulary absent from the embedding model’s training distribution. The goal is not to prove that one retriever always wins, but to verify that fusion preserves useful behavior from both.

  • Inspect dense-only, BM25-only, and overlap results separately.
  • Track whether relevant items enter the fused candidate set at all.
  • Use labeled judgments or carefully reviewed query sets before changing fusion settings.
  • Re-evaluate after changing embeddings, chunking, analyzers, or corpus content.