Why raw dense and BM25 scores should not be added

A dense-search score and a BM25 score are produced by different models and scoring rules. Their numeric ranges, distributions, and meanings can vary by index configuration, query length, embedding model, and corpus composition.

Adding raw scores can make one retrieval method dominate simply because it emits larger numbers. Even normalizing scores per query has edge cases: a narrow score distribution, a single strong result, or a sparse query with few matches can change the effect of the normalization.

  • Dense retrieval is useful for semantic similarity and paraphrases.
  • BM25 is useful for exact terms, codes, names, and uncommon vocabulary.
  • A combined ranking should not assume score values are interchangeable.

Fuse ranks instead of scores with RRF

Reciprocal rank fusion (RRF) combines ranked lists using position rather than the original score. For each document, add 1 divided by k plus its rank for every list in which it appears. Documents that rank well in either list receive useful credit, while documents supported by both lists rise naturally.

A common form is RRF(d) = sum over result lists of 1 / (k + rank(d)). The constant k reduces the impact of small rank differences near the top of a list. Choose a fixed value, document it, and evaluate it against representative queries rather than changing it ad hoc per request.

  • Request a bounded top-N candidate list from S3 Vectors.
  • Request a bounded top-N candidate list from Quickwit BM25.
  • Assign ranks starting at 1 within each list.
  • Sum RRF contributions by stable document identifier, then sort descending.

Build a reliable application-side fusion step

Use the same canonical document identifier in both retrieval paths. If dense vectors represent chunks while BM25 indexes whole documents, define the intended retrieval unit before fusion. Otherwise, one document may occupy many dense ranks and receive unintended influence over the final result set.

Keep the fusion stage observable. Log query class, candidate counts, overlap between dense and sparse results, and the source lists contributing to each final result. These signals help distinguish a retrieval issue from an indexing mismatch or an identifier-join problem.

  • Deduplicate candidates by the canonical retrieval-unit ID before returning results.
  • Preserve source ranks for debugging and relevance review.
  • Evaluate exact-term, paraphrase, mixed-intent, and low-result queries separately.
  • Apply any downstream filtering consistently to both candidate sources.