Why rank fusion is safer than score fusion

Dense-search scores and BM25 scores are not naturally comparable. Their ranges, distributions, and meanings depend on the embedding model, query text, corpus, analyzer settings, and retrieval implementation. Adding the two raw scores can make one retriever dominate for accidental numerical reasons rather than because it found better evidence.

RRF avoids that comparison entirely. Instead of combining scores, it combines document positions. If a chunk appears near the top of either ranked list, it earns a useful contribution to its final fused rank. This makes the merge strategy easier to reason about and less sensitive to score-scale changes.

  • Use dense retrieval for semantic similarity and paraphrases.
  • Use BM25 for literal wording, rare terms, and structured identifiers.
  • Treat each retriever's output as an ordered list, not as directly compatible scores.

Use one canonical chunk identity across both indexes

Hybrid retrieval depends on recognizing that a dense result and a BM25 result refer to the same retrievable unit. Assign every chunk a stable canonical ID before indexing it. Store that ID in the metadata associated with the dense vector and in the sparse document indexed for BM25.

Keep the chunking pipeline shared as well. If the sparse side indexes whole pages while the dense side indexes small passages, fusion can still work, but deduplication and downstream citation selection become more complicated. Matching chunk boundaries make the final result set more coherent for retrieval-augmented generation.

  • Generate a stable ID from a document ID, version, and chunk position.
  • Index the same searchable text or a deliberately documented variant in both paths.
  • Carry source, tenant, access-control, and document-version metadata with each chunk.
  • Apply equivalent filters before merging results.

Fuse candidates, then inspect the final evidence

At query time, request a candidate set from regional S3 Vectors and another from Quickwit BM25. Deduplicate by canonical chunk ID, then calculate an RRF score such as 1 divided by k plus rank for every list in which a chunk appears. Sum those contributions and sort descending. The constant k dampens the difference between adjacent high ranks; choose it as a configuration value and evaluate it on representative queries.

Return more candidates from each retriever than the application ultimately displays or sends to a language model. Fusion needs room for overlap and complementary results. After fusion, apply any required authorization checks, optional reranking, and context-budget selection. Log the component ranks alongside the fused rank so relevance issues can be traced to the dense path, the sparse path, or the merge policy.

  • Retrieve separate top-N lists from dense and sparse search.
  • For each chunk, sum 1 / (k + rank) across the lists where it appears.
  • Deduplicate by canonical chunk ID before building final context.
  • Evaluate with queries containing paraphrases, exact product terms, IDs, and mixed intent.