Why raw-score blending is fragile

It is tempting to add a dense-search score to a BM25 score and sort the result. The problem is that the values are not guaranteed to mean the same thing. A dense similarity score depends on the embedding model and similarity measure, while BM25 is driven by term frequency, document frequency, and length normalization.

Even if a weighted sum appears effective for one query set, it can become unstable when query language, document length, corpus composition, or embedding models change. Score calibration is possible, but it requires evaluation data and ongoing maintenance. Rank-based fusion is a simpler baseline because it avoids assuming that one system’s numerical scale is interchangeable with another’s.

  • Dense retrieval can surface semantically related wording.
  • BM25 can reward exact terms, identifiers, names, and rare vocabulary.
  • A score of 0.8 in one retriever is not inherently comparable to 0.8 in another.
  • Rank positions are easier to combine consistently across retrieval methods.

Fuse dense and sparse lists with RRF

Run the same user query through dense retrieval and BM25 retrieval, then retain a bounded candidate list from each. For every document that appears in either list, calculate an RRF score by adding 1 divided by k plus its rank for each list where it appears. The document with the largest combined value is ranked first.

The constant k reduces the advantage of a document that happens to be ranked first in only one list. A commonly used starting value is 60, but it is a tuning parameter rather than a universal rule. Keep ranks one-based, deduplicate documents by a stable document ID, and apply a deterministic tie-breaker such as document ID or a stored timestamp.

  • For each result list, assign ranks 1, 2, 3, and so on.
  • Compute: RRF(document) = Σ 1 / (k + rank).
  • Add contributions only from lists in which the document appears.
  • Sort documents by the fused score and return the desired top results.

Make fusion observable before tuning it

Start with a small, repeatable evaluation set: representative queries, expected documents or relevance judgments, and notes about failure modes. Include queries that contain exact product codes or names, paraphrased questions, short ambiguous requests, and longer natural-language queries. This makes it easier to see whether sparse, dense, or fused retrieval is helping.

Log the document IDs and ranks from each retriever alongside the final fused rank. Those fields explain a result far better than a fused score alone: a document may win because both retrievers ranked it well, or because one retriever found something the other missed. Only after this baseline is visible should you vary candidate depth, k, or downstream reranking logic.

  • Record dense rank, BM25 rank, fused rank, and document ID for each returned result.
  • Measure retrieval quality on a fixed query set before changing parameters.
  • Inspect queries where the two retrievers disagree most strongly.
  • Treat candidate-list depth as a recall decision: fusion cannot promote documents that neither list returned.