Why raw dense and BM25 scores should not be added directly

A dense retrieval score and a BM25 score are produced by different models and scoring functions. Their numerical ranges, distributions, and meaning can vary by embedding model, query length, document collection, analyzer configuration, and search settings. A score of 0.7 from one dense-search setup does not inherently have the same meaning as a BM25 score of 7.

Adding raw scores can therefore create unstable behavior. One retriever may dominate merely because its scores have a larger numeric range, not because its results are more useful for the query. Min-max normalization or hand-tuned weighting can help in specific environments, but those approaches need ongoing validation as the corpus and retrieval configuration change.

  • Dense retrieval is useful for semantic similarity and paraphrases.
  • BM25 is useful for exact wording, rare terms, codes, and names.
  • Score magnitudes are not a shared relevance scale.
  • A merge strategy should preserve useful results from both lists.

Merge result lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in each ranked list. For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the one-based position of d in retrieval list i. Sum the contribution for every list in which the document appears, then sort documents by the resulting total.

The constant k reduces the difference between adjacent high ranks and prevents the first result from overwhelming the rest of the list. A commonly used starting value is 60, but it is a configuration choice rather than a universal optimum. The important property is that RRF uses ordering, so it avoids treating S3 Vectors dense scores and Quickwit BM25 scores as directly comparable.

  • Run dense and BM25 retrieval independently for the same query.
  • Request a sufficiently deep candidate list from each retriever.
  • Deduplicate documents by a stable document or chunk identifier.
  • Add RRF contributions for every list where a document appears.

Choose candidate depth and evaluate the merged results

Candidate depth matters because fusion can only promote documents that appear in at least one input list. If each retriever returns too few candidates, a relevant document that ranks just below a cutoff cannot benefit from the other signal. Start with a depth that matches the size of the final result set and the diversity of your corpus, then test changes with representative queries.

Evaluate hybrid retrieval with query groups, not just an aggregate number. Include exact-lookup queries such as product names or error codes, semantic questions phrased differently from the source text, and ambiguous queries that need multiple perspectives. Inspect whether fusion recovers useful documents that either dense or BM25 retrieval alone missed.

  • Keep the source rank and retrieval method as debugging metadata.
  • Test several k values and candidate depths on a fixed query set.
  • Check for duplicate chunks and canonicalize identifiers before fusion.
  • Use relevance judgments or review samples before changing defaults.