Why raw-score blending is fragile

A dense retrieval score and a BM25 score do not naturally share a common meaning. Their ranges, distributions, and sensitivity to query length can differ, even when both systems return useful documents.

A weighted formula such as dense_score + BM25_score can work only after deliberate normalization and ongoing validation. Changes to embedding models, document fields, analyzers, or index configuration can alter score behavior and quietly invalidate a previously chosen weight.

RRF avoids this direct comparison. It treats each retriever as a ranked-vote source and rewards documents that appear near the top of one or both lists.

  • Dense retrieval helps with semantic similarity and paraphrase.
  • BM25 helps with exact terms, identifiers, and rare vocabulary.
  • Rank positions are easier to combine than unrelated raw scores.

Apply reciprocal rank fusion at the application layer

For a query, request the top N dense results from regional S3 Vectors and the top N sparse results from Quickwit BM25. Join the lists by a stable document or chunk identifier, then assign every returned item an RRF score.

The common formula is RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs across retrieval lists where document d appears. The constant k reduces the advantage of being first in a single list and is commonly chosen as a moderate positive value such as 60.

If a document is absent from one list, it simply receives no contribution from that list. Sort the merged candidates by their RRF score, then pass the leading results to a reranker, answer-generation stage, or user-facing result page.

  • Use the same canonical ID in dense and sparse indexes.
  • Retrieve enough candidates from each source to allow useful overlap and recovery.
  • Deduplicate before presenting or reranking results.
  • Retain per-source ranks for debugging and relevance analysis.

Make fusion observable before making it complex

Start by logging which retrieval source contributed to each fused result, its rank in each source, and its final RRF score. Those fields make it possible to inspect cases where exact-match content wins, semantic matches win, or both retrievers agree.

Evaluate the fused list against representative queries rather than relying on a single query style. Include acronym-heavy searches, product names, error strings, natural-language questions, and queries that should return no result.

RRF is a strong baseline, not a substitute for relevance judgment. Once its behavior is visible, teams can test candidate depths, the k constant, field-specific BM25 queries, filtering rules, or a downstream reranker while keeping a stable fusion baseline.

  • Track overlap between dense and BM25 candidate sets.
  • Inspect queries where only one retriever contributes useful results.
  • Keep filters consistent across both retrieval paths.
  • Version retrieval settings alongside embedding and index changes.