Why raw dense and sparse scores should not be added

A dense-search score and a BM25 score are produced by different ranking functions. Their numeric ranges, distributions, and sensitivity to query length can differ substantially. A score of 0.7 from one retriever does not inherently mean the same thing as a score of 0.7 from another.

Adding raw scores can therefore make a hybrid ranker unstable. A change in embedding model, vector similarity configuration, analyzer behavior, or corpus composition may alter one score distribution and unexpectedly dominate the combined ranking.

  • Dense retrieval ranks semantic proximity in an embedding space.
  • BM25 ranks lexical term matches using term frequency and corpus statistics.
  • Raw-score fusion requires calibration that may need ongoing maintenance.

Fuse ranked lists with reciprocal rank fusion

Reciprocal rank fusion, commonly abbreviated as RRF, combines result lists by position rather than by raw score. Retrieve a candidate list from dense search and another from BM25, then assign each document a fused score based on where it appears in each list.

For each document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the one-based position in retriever i and k is a positive constant. A document appearing near the top of either list receives meaningful credit; a document appearing in both lists receives credit twice.

  • Use a deterministic tie-breaker, such as a document ID, after computing fused scores.
  • Deduplicate documents before returning the final ranked list.
  • Choose the same candidate depth for both retrievers as an initial baseline.
  • Treat k as a ranking-control parameter and validate it with representative queries.

Build an observable hybrid retrieval path

A useful implementation starts by issuing the same user query to both retrieval paths: an embedded query to the dense index and a text query to Quickwit BM25. Keep the retrieved rank, source, and original score for every candidate, even if RRF is the only score used for final ordering.

Those fields make ranking behavior inspectable. When a result looks surprising, an engineer can determine whether it was carried by semantic similarity, lexical matching, or agreement between both retrievers. This is more actionable than inspecting only a single opaque fused score.

  • Log query text or an approved query identifier, subject to privacy requirements.
  • Record dense rank, BM25 rank, fused rank, and source membership for returned candidates.
  • Evaluate semantic queries, exact-name queries, acronym queries, and mixed-intent queries separately.
  • Revisit candidate depth and k when indexes, analyzers, or embedding models change.