Why raw-score blending is fragile

A dense retrieval system and a BM25 engine produce scores with different meanings. A vector similarity score depends on the embedding model, distance metric, and index configuration. A BM25 score depends on term frequency, document length, corpus statistics, and query terms. Adding the two values together without calibration can make one retrieval path dominate for reasons unrelated to relevance.

Score distributions can also move over time. A corpus update may change BM25 statistics, while an embedding-model change may alter dense score ranges. If an application relies on fixed score weights, a previously acceptable blend can quietly become unbalanced after an operational change.

RRF avoids this direct comparison. It uses each document's position in each ranked list, making it useful when dense and sparse scores are not naturally comparable.

  • Dense retrieval helps with semantic similarity and paraphrases.
  • BM25 helps preserve exact-token behavior for names, codes, and rare terms.
  • Raw scores should not be assumed to share a common scale.
  • Rank-based fusion reduces dependence on score calibration.

Fuse two candidate lists with RRF

Run the same user query through both retrieval paths: dense search against vectors and sparse search with BM25. In a Talqora deployment, this means obtaining a ranked dense candidate list from regional S3 Vectors and a ranked sparse candidate list from Quickwit BM25, then merging them in the application retrieval layer.

For each document d, calculate an RRF score as the sum of 1 divided by k plus its rank in every list where it appears. In notation: RRF(d) = Σ 1 / (k + rank_i(d)). A document returned by both systems gains evidence from both lists, while a strong result from only one path can still remain competitive.

The constant k dampens the influence of the first few ranks. Start with a fixed value such as 60 as a conventional baseline, then evaluate it against representative queries before changing it. The important property is consistency: rank lists should use one-based positions and should be truncated to a deliberate candidate depth.

  • Retrieve a bounded top-N list from dense search and BM25.
  • Deduplicate by a stable document or chunk identifier.
  • Assign ranks independently within each result list.
  • Sort documents by descending fused RRF score and return the desired top results.

Make fusion observable and testable

Hybrid retrieval is easier to operate when the response pipeline records provenance. For every returned item, retain whether it came from dense search, BM25, or both, along with its rank in each source list. This makes it possible to investigate why a result appeared and to detect cases where one retrieval path contributes little.

Build a small evaluation set from real query patterns, including exact identifiers, short ambiguous queries, natural-language questions, and domain-specific terminology. Review relevance at the final fused ranks, but also inspect dense-only and BM25-only candidate lists. The goal is not to prove that one method always wins; it is to identify which query classes benefit from each signal.

Keep the initial design simple. Use RRF before introducing learned rerankers or complex score normalization. A transparent baseline gives engineering teams a reliable way to validate indexing changes, candidate depths, and query processing decisions.

  • Log source membership and per-source rank for fused results.
  • Evaluate exact-match and semantic query classes separately.
  • Watch for duplicate chunks and inconsistent document identifiers.
  • Revisit candidate depth when relevant results rarely reach the fusion stage.