Why raw dense and BM25 scores should not be added

Dense retrieval scores and BM25 scores are produced by different ranking functions. Their ranges, distributions, and sensitivity to query length can differ substantially. Even when both systems return a numeric score, a score of 0.8 from one system does not inherently carry the same relevance meaning as 0.8 from the other.

Direct score addition can therefore create fragile behavior. A change in embedding model, index configuration, analyzer, corpus composition, or query wording may shift one score distribution and silently alter the balance of the hybrid ranker.

  • Use dense retrieval for semantic similarity and paraphrases.
  • Use BM25 for exact vocabulary, product names, codes, and rare terms.
  • Treat scores from separate retrieval systems as local ranking signals unless they have been explicitly calibrated.

Fuse ranked lists with Reciprocal Rank Fusion

RRF avoids cross-system score comparison. Retrieve a ranked candidate list from dense search and another from BM25, then assign each document a contribution based on its rank in each list. The combined score is the sum of those contributions: RRF(d) = Σ 1 / (k + rank_i(d)).

The constant k reduces the gap between adjacent top ranks and makes fusion less dominated by a single first-place result. It is a tuning parameter, not a universal truth: choose a value, inspect representative queries, and revise it only with evidence from relevance judgments or observed search outcomes.

  • Deduplicate by a stable document or chunk identifier before final sorting.
  • Give no contribution to a document that is absent from a retrieval list.
  • Keep the source ranks and fusion contribution in logs for debugging.
  • Begin with equal weighting; introduce source weights only when evaluation supports them.

Build an evaluable hybrid retrieval path

For each query, issue dense retrieval to the relevant regional S3 Vectors collection and sparse retrieval to the Quickwit BM25 index over the same retrieval unit, such as a document chunk. Fetch enough candidates from each side that fusion has meaningful overlap and coverage, then apply RRF and return the highest-ranked fused results.

Evaluation matters more than a clever-looking formula. Create a small set of real queries, including exact-name lookups, multi-term troubleshooting questions, paraphrases, and ambiguous requests. Record whether the desired source appears in the candidate set and at what rank, then compare dense-only, BM25-only, and fused retrieval before adjusting depth, k, or weights.

  • Ensure both indexes use the same chunk IDs and current document version.
  • Measure candidate recall separately from final ranking quality.
  • Inspect queries where dense and BM25 disagree; they reveal corpus and analyzer issues.
  • Keep retrieval, fusion, and any later reranking as separate observable stages.