Why rank fusion is safer than score fusion

Dense retrieval and BM25 produce scores with different meanings and scales. A vector similarity score depends on the embedding model and similarity method, while a BM25 score depends on term frequency, document frequency, and query structure. Adding those raw values directly can create unstable rankings.

Reciprocal rank fusion (RRF) avoids that comparison. Instead of using each system’s score, it uses an item’s position in each ranked list. Documents that rank well in either list receive credit, while documents that appear near the top of both lists receive an especially useful boost.

  • Use dense search for semantic matches and paraphrases.
  • Use BM25 for exact terms, product names, codes, and rare vocabulary.
  • Fuse ranks rather than assuming dense and sparse scores are numerically comparable.

Implement RRF in the application layer

For each user query, request a candidate list from Talqora’s dense retrieval path and another from the Quickwit BM25 path. Keep enough candidates from each list to allow useful overlap and recovery of results that only one method finds. The appropriate candidate depth depends on the corpus and the number of final results your application needs.

Assign every returned document an RRF score: score(document) = sum of 1 divided by k plus rank, across the lists where the document appears. The constant k reduces the difference between nearby ranks; a commonly used starting value is 60, but it should be treated as a tuning parameter rather than a universal default.

  • Deduplicate candidates with a stable document or chunk identifier.
  • Use one-based ranks: the first result has rank 1.
  • Sort by the fused score and return the top N results.
  • Preserve source ranks in logs to make relevance investigations easier.

Tune retrieval as a system, not as two isolated searches

Evaluate the fused list against representative queries, including exact-name lookups, natural-language questions, acronym-heavy searches, and queries with ambiguous wording. Review whether the relevant chunk is present in the candidate lists before assessing any later generation or reranking step. If it is absent, the issue is retrieval coverage rather than answer composition.

RRF is intentionally simple, which makes it a strong baseline. After establishing it, teams can experiment carefully with query-aware routing, such as giving more attention to BM25 for identifier-like queries or increasing dense candidate depth for conversational questions. Keep a consistent evaluation set so changes are attributable to the retrieval adjustment rather than anecdotal examples.

  • Track recall at the candidate stage, not only final answer quality.
  • Inspect queries where dense and sparse results have no overlap.
  • Test changes by query category rather than relying on aggregate averages alone.
  • Version embedding, chunking, and indexing changes alongside retrieval configuration.