Why score addition is often fragile

Dense retrieval typically ranks documents by how closely their embeddings match a query embedding. Sparse BM25 retrieval ranks documents using query terms, term frequency, document frequency, and length normalization. Both produce an ordering, but their raw scores have different meanings and ranges.

Adding those scores together requires normalization choices that can shift as the corpus, embedding model, query mix, or BM25 configuration changes. A rank-based method instead asks a simpler question: how highly did each retrieval path place a document?

  • Dense search can surface semantic paraphrases and conceptually related passages.
  • BM25 can strongly reward exact terms, identifiers, error codes, and uncommon names.
  • Raw score scales should not be assumed to be comparable across retrieval methods.

Fuse two ranked lists with RRF

Run the same user query through the dense and sparse paths, then retain a candidate list from each. For every document that appears in either list, calculate an RRF score by adding 1 divided by k plus its rank for each list in which it appears.

The formula is RRF(d) = Σ 1 / (k + rank_i(d)). Here, rank_i(d) starts at 1, and k is a smoothing constant. A document returned near the top by either method receives useful credit, while a document ranked well by both receives more credit.

  • Use a consistent document identifier to join dense and BM25 results.
  • Treat documents absent from a list as contributing zero from that retrieval path.
  • Choose k deliberately and keep it fixed while evaluating changes.
  • Return the highest fused scores to the next stage, such as reranking or answer generation.

Make fusion observable and testable

Log the dense rank, sparse rank, and final fused rank for each returned document. These fields make it possible to understand whether a result was supported by semantic matching, lexical matching, or both. They also help identify cases where one path dominates unexpectedly.

Evaluate with representative queries rather than relying on a single relevance style. Include exact-match queries such as product names or codes, semantic questions expressed in varied language, and mixed queries that contain both domain terms and natural-language intent.

  • Inspect overlap between dense and sparse candidate lists.
  • Track which retrieval path introduced each final result.
  • Review failures caused by missing exact terms or weak semantic matches.
  • Change one variable at a time: candidate depth, k, query preparation, or ranking logic.