Why rank fusion is a practical starting point

A single retrieval method rarely handles every query equally well. A query such as "reset my password" may benefit from semantic similarity, even if the best document uses different wording. A query such as "ERR-1842" or a specific SKU, however, often depends on exact lexical matching.

It is tempting to add dense and sparse scores together. In practice, those scores are produced by different models and ranking systems, so their numeric scales and distributions need not be comparable. Rank-based fusion avoids assuming that a dense score and a BM25 score mean the same thing.

  • Use dense retrieval to capture semantic relatedness.
  • Use BM25 to preserve exact-token and rare-term behavior.
  • Treat each retriever's output as an ordered list rather than a directly comparable score scale.
  • Start with fusion before introducing learned rerankers or custom score calibration.

Fuse the two candidate lists with reciprocal rank fusion

Reciprocal rank fusion assigns each document a contribution based on its position in each result list. For a document d, the fused score is the sum of 1 divided by k plus its rank in every list where it appears: RRF(d) = Σ 1 / (k + rank_i(d)). Lower rank numbers contribute more, so documents that rank well in either retrieval path rise naturally.

The constant k reduces the difference between nearby ranks and keeps a single first-place result from overwhelming the rest of the evidence. Choose k as a configurable application parameter, then evaluate it against representative queries. The important property is not a universal value of k, but consistent treatment of ranks across both lists.

  • Request a candidate set from dense search in regional S3 Vectors.
  • Request a candidate set from Quickwit BM25 using the same access and filtering constraints.
  • Deduplicate by a stable document or chunk identifier.
  • Sum each document's reciprocal-rank contributions, sort descending, and return the top fused results.

Make fusion reliable with aligned documents, filters, and evaluation

Fusion works best when both indexes represent the same retrieval unit. If dense search indexes document chunks while BM25 indexes entire documents, the result merger needs a deliberate policy: fuse at the chunk level, or map chunks to a parent document before presentation. Without that decision, duplicates and inconsistent ranking can obscure useful results.

Apply eligibility rules consistently to both branches. Tenant boundaries, document status, language, permissions, and time-based constraints should be enforced before fusion whenever possible. Otherwise, a highly ranked candidate from one branch may be discarded late, reducing the effective depth of the final result set.

Evaluate the fused system on a query set that includes natural-language questions, exact identifiers, misspellings, and mixed queries. Review not only aggregate relevance, but also whether one branch consistently contributes unique useful results. That evidence helps determine candidate depth, filtering behavior, and whether later reranking is justified.

  • Use stable IDs shared by dense and sparse records.
  • Keep metadata and access-control filters semantically equivalent across both retrieval paths.
  • Log source ranks and fused rank for debugging, without exposing sensitive query or document content unnecessarily.
  • Measure result quality by query category, not only by one blended average.