Why raw dense and sparse scores should not be added blindly

A dense-search score and a BM25 score are produced by different retrieval methods. Their numeric ranges, distributions, and meanings may differ by index, query, embedding model, analyzer configuration, and implementation details.

Adding those raw values together can make one retrieval path dominate for reasons unrelated to relevance. A safer default is to combine rank positions rather than treating independently generated scores as a shared measurement scale.

  • Dense search ranks documents by vector similarity.
  • BM25 ranks documents using query-term and document-term statistics.
  • Score ranges can shift when data, models, or indexing settings change.
  • Rank-based fusion avoids requiring score calibration up front.

Fuse candidate lists with reciprocal rank fusion

Reciprocal rank fusion (RRF) assigns each document a contribution based on its position in each ranked list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus the document's rank in every list where it appears. The constant k dampens the influence of very high positions.

In practice, query regional S3 Vectors for a dense candidate list and Quickwit BM25 for a sparse candidate list. Deduplicate document identifiers, calculate the fused score, sort descending, and return the top results. A document found by both retrievers receives contributions from both lists.

  • Choose a candidate depth for each retriever, such as the top N dense and top N sparse results.
  • Use stable document identifiers to deduplicate candidates.
  • Treat ranks as one-based: the first result has rank 1.
  • Keep the source ranks in response metadata for debugging and evaluation.

Make fusion observable before tuning it

RRF is a practical starting point, not a substitute for relevance evaluation. Record which retrieval path contributed each returned document, its dense rank, its BM25 rank, and its final fused rank. This makes it possible to identify cases where lexical matches or semantic matches are being underrepresented.

Evaluate with representative queries that include exact identifiers, product names, short natural-language questions, and ambiguous terms. Adjust candidate depths and the RRF constant only against a defined relevance set or reviewed query sample, rather than optimizing from a small number of memorable examples.

  • Include exact-match queries, where BM25 may be particularly valuable.
  • Include paraphrased queries, where dense retrieval may add useful candidates.
  • Inspect documents returned by only one retriever as well as documents returned by both.
  • Version fusion settings alongside embedding and indexing configuration changes.