Why raw dense and sparse scores should not be added directly

A dense-search score reflects similarity in an embedding space. A BM25 score reflects term frequency, inverse document frequency, document length normalization, and query terms. Even when both systems return a numeric score, the numbers have different meanings, ranges, and distributions.

Adding those scores together can make ranking sensitive to accidental scale differences. A change in embedding model, chunking strategy, corpus composition, or BM25 configuration can alter one score distribution without indicating that the underlying relevance signal became more or less valuable.

Treat dense and sparse search as independent rankers unless you have a deliberately trained and monitored score-calibration process. Rank fusion provides a safer default for many retrieval pipelines.

  • Dense search is useful for paraphrases and concept-level similarity.
  • BM25 is strong for exact names, error codes, part numbers, and uncommon terms.
  • Raw score ranges can change after indexing or model changes.
  • Rank positions are easier to combine across retrieval methods.

Apply reciprocal rank fusion to two candidate lists

Run the same user query through both retrieval paths: dense retrieval against vectors stored in the regional S3 Vectors layer and sparse retrieval through Quickwit BM25. Request a sufficiently broad candidate set from each path so that the fusion stage has meaningful overlap and alternatives to consider.

For every document returned by either path, compute an RRF contribution from its position in each list. The common form is RRF(d) = Σ 1 / (k + rank_i(d)), where rank 1 is the first result, the sum spans available rankers, and k is a positive smoothing constant.

Documents found by both rankers receive two contributions. Documents found by only one ranker can still appear in the merged ranking, which is useful when a query is either highly lexical or highly semantic.

  • Deduplicate candidates with a stable document or chunk identifier.
  • Use one-based ranks: first place has rank 1.
  • Keep the dense and BM25 retrieval depths explicit in configuration.
  • Choose k as a tunable application parameter and evaluate it on representative queries.

Make fusion observable and validate it with query slices

Log the fused rank, final RRF score, source ranks, and source-list membership for each returned result. These fields make it possible to explain why a result appeared: it may have ranked well in one path, or it may have been consistently strong across both.

Evaluate more than an aggregate relevance metric. Create query slices for exact identifiers, short ambiguous queries, long natural-language questions, newly introduced terminology, and multilingual content when applicable. A fusion approach that improves semantic questions but hides exact error-code matches may need different retrieval depths or query routing.

RRF is a first-stage merge, not a substitute for all ranking decisions. If your application needs stronger ordering among the top results, pass the fused candidates to a separate reranking step only when you can measure its effect and operate it reliably.

  • Inspect examples where only BM25 or only dense retrieval contributed.
  • Track overlap between candidate lists as a diagnostic, not a quality target by itself.
  • Re-evaluate fusion after changing embeddings, chunking, analyzers, or corpus content.
  • Use relevance judgments or task outcomes to select retrieval depths and k.