Why raw dense and BM25 scores should not be added

A dense-search score and a BM25 score are produced by different ranking models with different scales, distributions, and meanings. Even when both are exposed as numeric values, a score of 0.7 from one system does not imply the same relevance as 0.7 from the other.

Adding raw scores can make one retriever dominate simply because its numeric range is wider. Normalizing scores can help in some systems, but it requires observing score distributions over representative queries and revisiting the approach as embeddings, analyzers, or corpora change.

  • Dense retrieval emphasizes proximity in embedding space.
  • BM25 emphasizes term frequency, document frequency, and length normalization.
  • Score magnitudes are model- and implementation-specific.
  • Rank positions are easier to compare across retrieval methods.

Fuse ranked lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in each ranked list, then sums those contributions. For a document d, the fused score is: RRF(d) = Σ 1 / (k + rankᵢ(d)), where rankᵢ(d) is the one-based rank of d in retrieval list i.

The constant k softens the difference between nearby ranks. A commonly used starting value is 60, but it is a tuning parameter rather than a universal rule. The important property is that a document appearing near the top of both dense and sparse results receives a strong combined signal without requiring either system’s raw score.

  • Request a fixed candidate count from dense search and BM25 search.
  • Use a stable, shared document identifier to deduplicate candidates.
  • Treat a missing document in a list as contributing zero from that list.
  • Sort by fused score, with a deterministic tie-breaker such as document ID.

Implement the fusion layer at the retrieval boundary

For each user query, issue a dense query against the vector index and a BM25 query against the sparse index. Convert each response into an ordered list of document IDs, then maintain a map keyed by ID. As each list is processed, add its reciprocal-rank contribution to the map entry.

Keep the fusion logic small and observable. Log the candidate count from each retriever, the rank at which each returned document appeared, and whether a final result came from dense retrieval, sparse retrieval, or both. These signals make it possible to investigate failures such as exact identifiers being missed by dense search or paraphrases being absent from sparse results.

RRF is especially useful as a reliable baseline. Once it is in place, changes such as query rewriting, metadata filtering, embedding updates, or reranking can be evaluated against a clear hybrid retrieval path rather than against a single retriever alone.

  • Apply required metadata and access-control filters consistently before fusion.
  • Use the same document IDs in the S3 Vectors and Quickwit indexing paths.
  • Fetch only the top fused candidates needed by the next stage.
  • Test with semantic queries, exact names, error codes, and mixed natural-language-plus-identifier queries.