Why raw dense and BM25 scores should not be added

A dense-search score and a BM25 score are produced by different retrieval models with different scales and distributions. Even when both are useful indicators within their own result lists, a score of 0.8 from one system does not inherently mean the same thing as a score of 0.8 from another.

Adding raw scores can make ranking sensitive to query length, corpus changes, embedding model choices, and implementation details. Score normalization can help, but it introduces assumptions that need validation. Rank-based fusion avoids the need to treat the two score scales as directly comparable.

  • Dense retrieval can recover semantically related language that does not share terms.
  • BM25 can strongly match exact terms, identifiers, names, and rare phrases.
  • Their score ranges may change independently as indexes or models evolve.

Fuse two result lists with reciprocal rank fusion

RRF assigns each document a contribution based on its rank in each list. For a document at rank r, the contribution is 1 divided by k plus r. Sum that contribution across the dense and sparse lists, then sort documents by the resulting total.

The constant k reduces the difference between adjacent top ranks and limits the influence of any single list. A common starting value is 60, but it is a tuning choice rather than a universal default. Evaluate it against representative queries and relevance judgments where possible.

  • Request a candidate list from S3 Vectors for the embedded query.
  • Request a candidate list from Quickwit BM25 for the text query.
  • Deduplicate by a stable document or chunk identifier.
  • Compute RRF scores and return the highest-ranked merged candidates.

Keep fusion observable and tune it with real queries

Store the source ranks used to produce each fused result. This makes it possible to see whether a result was supported by dense retrieval, sparse retrieval, or both. It also helps diagnose failures, such as an exact identifier being absent from the sparse query or a semantically relevant chunk being missing from the vector candidates.

Start with equal weighting for both lists, then adjust only when evaluation shows a consistent need. If your workload contains product codes or error messages, sparse retrieval may deserve more influence. If users ask broad natural-language questions over varied wording, dense retrieval may contribute more often.

  • Log query text, candidate identifiers, source ranks, and the final fused rank.
  • Evaluate exact-term, semantic-paraphrase, and mixed-intent query sets separately.
  • Choose candidate depths large enough that each retriever can contribute useful results.
  • Revisit tuning after changing chunking, embeddings, analyzers, or corpus composition.