Why raw dense and BM25 scores should not be added directly

Dense retrieval scores and BM25 scores have different meanings, ranges, and distributions. A dense score generally reflects proximity in an embedding space, while BM25 rewards query-term evidence using term frequency, document length, and corpus statistics.

Adding those values together without calibration can make ranking sensitive to score scale rather than relevance. A change in embedding model, corpus composition, or BM25 configuration can shift one score distribution and unintentionally overwhelm the other.

  • BM25 is especially useful for exact names, identifiers, error codes, and rare terms.
  • Dense search can surface semantically related content when wording differs.
  • Score magnitudes alone do not establish that one result is more relevant than another across retrieval systems.

Apply reciprocal rank fusion to two candidate lists

Run dense and BM25 retrieval independently for the same query, then retain a ranked candidate list from each. For every document, calculate an RRF score by summing 1 divided by k plus its rank for each list in which it appears: RRF(d) = Σ 1 / (k + rank_i(d)).

The constant k dampens the advantage of the first few positions. A commonly used starting point is 60, but it is a tuning parameter rather than a universal answer. The important property is that RRF uses ordinal rank, so it does not require dense and sparse scores to share a common scale.

  • Use a stable document ID to deduplicate candidates across dense and BM25 results.
  • Assign ranks starting at 1, not 0.
  • Give a document credit for every result list in which it appears.
  • Sort by the summed RRF score and return the top fused results.

Evaluate fusion with queries that expose each retrieval mode

Build a small evaluation set from realistic traffic before tuning k or candidate depths. Include exact-match queries, natural-language questions, ambiguous requests, and queries containing product names or identifiers. Label which documents are useful at the top of the result list.

Inspect failures by retrieval source. If BM25 finds the right document but dense retrieval does not, the issue may be vocabulary or embedding coverage. If dense search succeeds while BM25 misses, the query may rely on paraphrase. Fusion is most valuable when those strengths are complementary.

  • Track relevance at a fixed cutoff such as the first 5 or 10 results.
  • Compare dense-only, BM25-only, and fused rankings on the same query set.
  • Test candidate-list depth separately from the final number of results shown.
  • Keep the fusion step observable: record source ranks and the final fused rank for debugging.