Why raw dense and BM25 scores should not be added directly

A dense-search score reflects the relationship between embedded query and document vectors. A BM25 score is derived from lexical term statistics, including term frequency and document frequency. Even when both systems return numeric scores, those numbers describe different ranking models.

Adding the two scores with a fixed formula can make ranking sensitive to query shape, corpus composition, and implementation details. A score range that appears reasonable for one query may be unrepresentative for another. Before combining scores directly, a team would need a calibration process and ongoing evaluation to show that the scales remain meaningfully aligned.

  • BM25 rewards matching query terms, especially informative or rare terms.
  • Dense retrieval can surface semantically related text with little word overlap.
  • A numeric score is not automatically a probability or a universal relevance scale.
  • Rank position is often more stable across retrieval methods than raw score magnitude.

Fuse ranked lists with reciprocal rank fusion

Reciprocal rank fusion (RRF) combines result lists using position rather than raw retrieval scores. For each document, add a contribution from every list where it appears: 1 divided by k plus its rank. The constant k reduces the impact of small rank differences near the top of a list.

For a hybrid query, request a candidate list from dense retrieval and another from BM25. Assign ranks starting at 1, aggregate each document’s RRF contributions, then sort documents by the fused total. A document appearing near the top of both lists rises naturally, while a strong result from only one method can still be retained.

  • Formula: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Use a stable document identifier to deduplicate candidates across lists.
  • Treat a missing document from one list as contributing zero from that list.
  • Start with one shared k value, then validate choices with representative relevance judgments.

Make fusion an observable retrieval stage

Keep the dense and sparse retrieval stages visible in application telemetry. Record which source retrieved each final document, its rank in each source, and its fused rank. This makes it possible to investigate failures such as exact identifiers being missed by sparse retrieval or semantically relevant passages being absent from dense candidates.

Candidate depth matters as much as the fusion formula. If each retriever returns too few candidates, fusion cannot rescue documents that never enter either list. Select candidate limits from query logs and relevance evaluation, while considering latency and the number of documents passed to any later reranking or answer-generation stage.

  • Log dense rank, BM25 rank, fused score, and final position per returned document.
  • Evaluate identifier-heavy, natural-language, short, and multi-concept queries separately.
  • Inspect disagreements: documents retrieved by only dense search or only BM25 are useful diagnostics.
  • Version fusion settings so ranking changes can be compared against prior behavior.