Why raw-score blending is fragile

Dense retrieval ranks documents by similarity in embedding space. It is useful when a query and a document use different wording but express related meaning. BM25 is a sparse lexical method that rewards matching query terms and related term-frequency signals.

The scores returned by these methods are not naturally interchangeable. A cosine-like similarity value and a BM25 score have different ranges, distributions, and meanings. Adding them directly requires calibration that can change as the corpus, embedding model, or query mix changes.

Rank fusion avoids that score-normalization problem. It only asks where a document appeared in each result list.

  • Dense search can recover semantic matches with limited exact term overlap.
  • BM25 can preserve exact identifiers, names, error codes, and uncommon terms.
  • Rank-based fusion does not require dense and sparse scores to share a scale.

Fuse independent dense and sparse candidates with RRF

For each query, run a dense search against the regional S3 Vectors index and a BM25 search against the Quickwit index. Request a sufficiently deep candidate list from each path, then assign every returned document an RRF contribution based on its rank.

A common formula is RRF(d) = Σ 1 / (k + rankᵢ(d)), where rankᵢ(d) is the one-based rank of document d in retrieval list i. The constant k reduces the difference between adjacent top ranks; 60 is a commonly used starting value, but it should be treated as a tunable application choice.

Sum contributions for each document ID, sort documents by the final fused score, and return the top results. A document found by both retrieval methods receives two contributions, while a document found by only one method can still be retained.

  • Use a stable document ID shared by the dense and sparse indexes.
  • Treat rank as one-based when implementing the formula.
  • Deduplicate document IDs before producing the final ordered result set.
  • Keep the source ranks in response metadata for debugging and evaluation.

Make fusion observable and evaluate it on real queries

Start with a small labeled query set drawn from the intended workload. Include semantic questions, exact-title searches, identifiers, abbreviations, and multi-concept requests. Compare dense-only, BM25-only, and fused rankings using the same candidate depth and relevance judgments.

Logging is especially important because a fused result can be hard to explain without source details. Record whether each returned document came from dense retrieval, BM25 retrieval, or both, along with its original ranks and fused score.

If relevant documents are missing before fusion, increasing the candidate depth from one or both retrieval paths may help more than changing the fusion formula. If the candidates are present but poorly ordered, adjust k or investigate query processing and index content.

  • Evaluate recall-oriented measures at the candidate stage before judging final ranking.
  • Inspect queries where dense and sparse retrieval disagree substantially.
  • Version the embedding model, document-processing pipeline, and evaluation set.
  • Use production query logs to expand the evaluation set over time.