Why raw dense and BM25 scores should not be added directly

Dense retrieval and BM25 calculate relevance differently. Dense search ranks documents by proximity in an embedding space, while BM25 ranks documents using query terms, document terms, and corpus-level statistics. Their numeric outputs are therefore not automatically interchangeable.

Adding the two raw scores can make one retrieval path dominate simply because its scoring range is wider. That can hide useful exact matches or, in the opposite direction, suppress semantically relevant documents that use different wording from the query.

  • Treat each result list as an ordered ranking first.
  • Do not infer that a score of 0.8 from one system equals 0.8 from another.
  • Validate score normalization separately before relying on weighted score sums.

Start with reciprocal rank fusion

Reciprocal rank fusion, often called RRF, combines lists by rewarding documents that appear near the top of either list. Instead of using a retrieval system’s raw score, the method uses the document’s position in each ranking. A document retrieved by both dense search and BM25 receives contributions from both lists.

For each document, add a value based on one divided by a constant plus its rank. The constant reduces the difference between adjacent positions and prevents the first few ranks from overwhelming the rest of a candidate list. Documents absent from a list contribute nothing from that list.

  • Retrieve a candidate set from S3 Vectors for semantic coverage.
  • Retrieve a candidate set from Quickwit BM25 for lexical coverage.
  • Deduplicate documents by a stable document identifier.
  • Sort the merged candidates by their combined rank-fusion value.

Make fusion observable and tune it with query examples

Keep the dense rank, sparse rank, and final fused rank alongside each returned candidate during development. These fields make it possible to explain why a document appeared and to identify cases where one retrieval path contributes little value.

Build a small evaluation set from representative queries, including exact identifiers, product names, abbreviations, natural-language questions, and queries with synonyms. Review whether dense search, BM25, or their combination supplies the useful candidates. Tune candidate depths and fusion parameters against those examples rather than relying on a single query type.

  • Log whether each final candidate came from dense search, sparse search, or both.
  • Inspect misses before changing fusion weights or rank constants.
  • Use stable IDs so duplicate records can be merged safely.
  • Keep retrieval fusion separate from any later application-specific reranking step.