Why score-based merging is often fragile

A dense search score and a BM25 score are not naturally comparable. They can have different ranges, distributions, and sensitivity to query length. Adding the raw values together can make one retriever dominate simply because its scoring scale is numerically larger.

This is especially noticeable across mixed query traffic. A short product-code query may benefit from sparse retrieval, while a natural-language question may be better served by dense retrieval. A fixed score normalization scheme can be difficult to maintain as data, analyzers, embeddings, or indexes change.

  • Dense similarity scores depend on the embedding model and similarity measure.
  • BM25 scores depend on term frequency, document length, and corpus statistics.
  • Rank positions are easier to compare than raw scores from different systems.

Fuse ranked lists with RRF

RRF combines result lists by awarding a document credit based on its position in each list. For every document, add 1 divided by k plus its rank for each retriever that returned it. The document’s final score is the sum of those credits.

For example, an application can query Talqora’s regional S3 Vectors-backed dense search and its Quickwit BM25-backed sparse search, collect the top results from each, then fuse those ranked lists in the application ranking layer. A document that appears near the top of both lists will rise, while a strong result from only one list can still remain competitive.

  • Use one-based ranks: rank 1 is the first result.
  • A common formula is: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Choose a positive k to reduce the gap between adjacent top ranks.
  • Deduplicate by a stable document or chunk identifier before returning results.

Make fusion observable and query-aware

Start with the same candidate depth from both retrievers, such as retrieving the top N dense results and top N BM25 results, then fuse and return the final top K. Candidate depth should be large enough to let either method contribute, but bounded to control request work and downstream reranking cost.

Log which retrieval paths contributed to each final result, along with dense rank, sparse rank, fused rank, query type, and user outcome signals when available. These records make it possible to inspect failures: an identifier query missing from BM25 candidates is a different problem from a relevant candidate being pushed down during fusion.

  • Evaluate exact-token, semantic, mixed, and ambiguous queries separately.
  • Keep a small judged query set when tuning candidate depth or k.
  • Consider rules for clear identifiers, such as requiring an exact sparse match to remain visible.
  • If adding a reranker later, apply it to the fused candidate set rather than every indexed document.