Why raw-score blending is fragile

A tempting hybrid-search implementation is to add a dense score to a BM25 score after choosing weights. This is often unreliable because the scores have different meanings. A similarity value from a vector search and a relevance score from BM25 are produced by different models, distributions, and query-dependent behavior.

Even normalizing each list does not fully solve the issue. A query with one highly distinctive keyword may produce a sharply peaked BM25 ranking, while a broad conceptual query may produce a flatter dense ranking. Score-based weighting can change substantially from one query shape to another.

  • Dense search rewards semantic proximity rather than exact token overlap.
  • BM25 rewards informative term matches and document term statistics.
  • The same numeric score in two systems should not be assumed to represent the same relevance confidence.

Fuse ranks with RRF

RRF ignores raw scores and assigns credit based on rank position. For each candidate document, add a contribution from every retrieval list in which it appears: 1 divided by k plus its rank. The final RRF score is the sum of those contributions, and documents are sorted by that sum.

Use a positive constant k to reduce the gap between adjacent top ranks and to keep lower-ranked results from having too much influence. A commonly used starting point is k = 60, but it is a tuning parameter, not a universal answer. Evaluate it against representative queries and relevance judgments when available.

  • Run dense retrieval against the S3 Vectors-backed index and sparse retrieval with Quickwit BM25.
  • Request a candidate depth from each retriever, such as the top N results.
  • Deduplicate candidates by a stable document or chunk identifier.
  • Compute the RRF sum, then return the highest-ranked merged candidates.

Make fusion observable and testable

Keep the source ranks alongside the fused result. A record such as dense_rank, sparse_rank, and rrf_score makes it possible to inspect why a result appeared. It also reveals whether one retrieval path is rarely contributing useful candidates for a particular content type or query class.

Start with a small query set that includes exact IDs, quoted names, domain-specific terminology, paraphrases, and multi-concept questions. Review not only the first result but also whether the merged candidate set contains the passages an answer-generation or reranking stage would need.

  • Log query text, retrieval depth, source ranks, and the final fused rank.
  • Measure candidate recall separately from final-answer quality.
  • Test failures caused by stale metadata, inconsistent chunk IDs, and filters applied differently across retrieval paths.
  • Tune candidate depth and k together; a larger candidate pool can change fusion behavior.