Why fuse ranks instead of raw scores

Dense search and BM25 produce scores with different meanings and ranges. A cosine-like similarity score is not directly comparable to a BM25 score, and attempting to add the two without calibration can let one retriever dominate for accidental numerical reasons.

RRF avoids that comparison. It uses only each document's position in a ranked list. This makes it a useful first hybrid strategy when an application can obtain a top-k list from dense retrieval and another top-k list from sparse retrieval.

  • Dense retrieval can surface semantic matches with different wording.
  • BM25 can prioritize exact identifiers, names, error codes, and rare terms.
  • Rank-only fusion avoids assuming that two scoring systems share a scale.

Apply RRF to two candidate lists

For every document appearing in either list, compute an RRF score by adding 1 divided by k plus its rank from each list. A document missing from a list contributes nothing from that retriever. Rank positions should begin at 1, not 0.

The constant k reduces the difference between nearby rank positions. A commonly used starting value is 60, but it is a tuning choice rather than a universal rule. Keep the dense and BM25 candidate depths explicit so changes to either retrieval path are easy to evaluate.

  • Formula: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Use one stable document ID across S3 Vectors results and Quickwit BM25 results.
  • Deduplicate IDs before sorting the fused candidates.
  • Return the highest fused scores, with deterministic tie-breaking such as document ID.

Make the fusion path observable

Store lightweight retrieval metadata alongside each response: whether a result came from dense search, BM25, or both; its rank in each list; and its final fused rank. This is more actionable than retaining only a final score because it shows which retrieval path actually contributed the candidate.

Start with a small evaluation set containing semantic paraphrases, exact-term queries, and mixed queries. Review failures by query type. If exact identifiers are missed, inspect sparse candidate depth and indexing. If paraphrases are missed, inspect embeddings, chunking, and dense candidate depth before changing fusion weights or constants.

  • Log candidate counts from each retriever before fusion.
  • Track overlap between dense and sparse result lists.
  • Inspect source ranks for clicked, accepted, or otherwise validated results.
  • Change one retrieval variable at a time during evaluation.