Why raw dense and BM25 scores are difficult to merge

Dense retrieval ranks documents by proximity between vector representations. BM25 ranks documents from sparse term matching and term statistics. Even when both outputs are represented as numeric scores, those numbers are produced by different retrieval models and should not be assumed to share a common relevance scale.

A direct weighted sum, such as 0.5 times the dense score plus 0.5 times the BM25 score, requires meaningful calibration. Without it, one retriever can dominate simply because it emits a wider numeric range. This can change as corpus composition, query language, embedding models, or index settings change.

  • Dense retrieval can surface semantic matches that do not share exact query terms.
  • BM25 can strongly reward exact identifiers, product names, and rare terms.
  • Score magnitude alone is not evidence that scores from separate retrievers are comparable.

Fuse ranked lists with RRF

RRF assigns each document a contribution based on its position in each ranked list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) for every list in which d appears. The constant k reduces the difference between nearby positions and is commonly selected as a fixed tuning value.

Retrieve a bounded candidate list from both Talqora retrieval paths, identify documents by a shared stable ID, and add their RRF contributions. Sort the merged documents by the resulting total. A document appearing in both lists receives support from both retrieval methods, while a high-ranking document from only one list can still be retained.

  • Use one-based ranks: the first result has rank 1.
  • Deduplicate by the canonical document or chunk ID before returning results.
  • Choose a candidate depth appropriate for the downstream context or reranking stage.
  • Keep the dense and BM25 retrieval requests independently observable.

Implement, inspect, and tune the merge

Start with equal treatment of the dense and sparse lists. For each query, retain diagnostic fields such as dense rank, BM25 rank, fused score, and which retrievers returned the document. These fields make it possible to investigate why a result was included without treating the fused score as a universal relevance probability.

Then evaluate with representative queries, especially queries containing exact strings alongside natural-language requests. Examples include error codes, part numbers, policy names, and questions expressed with synonyms. If one retrieval path is intentionally more important for a query class, use a controlled variation such as repeating its RRF contribution or routing that class to a different retrieval policy.

  • Test exact-match queries and semantic paraphrases separately.
  • Review queries where only one retriever contributes results.
  • Track result overlap between dense and BM25 lists.
  • Version fusion parameters alongside the retrieval configuration that produced them.