Why raw dense and BM25 scores should not be added

Dense retrieval ranks documents by semantic similarity in an embedding space. BM25 ranks documents from lexical term statistics, including term frequency and document frequency. Although both produce numbers called scores, those numbers do not share a stable meaning or scale.

Adding the two scores can make ranking sensitive to index settings, query length, embedding model changes, and score distributions. A document can win simply because one retrieval system emits numerically larger values, not because it is more relevant.

  • Dense search is useful for paraphrases, concepts, and semantically related language.
  • BM25 is useful for exact identifiers, rare terms, quoted phrases, and domain-specific vocabulary.
  • Score ranges can change independently as either retrieval system evolves.

Fuse ranks with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in each ranked result list. For a document at rank r, the contribution is 1 divided by k plus r. Sum that contribution across the dense and BM25 lists, then sort documents by the resulting total.

The constant k reduces the difference between adjacent positions near the top of a list. A commonly used starting value is 60, but it is a tuning parameter rather than a universal rule. The key property is that RRF uses ordering, so it does not require dense and sparse scores to be calibrated against each other.

  • Retrieve a bounded candidate list from dense search and another from BM25.
  • Use a stable document identifier to detect the same document in both lists.
  • For each result list, add 1 / (k + rank) to the document’s fused score.
  • Sort by fused score and return the top combined results.

Implement the fusion layer deliberately

Treat fusion as a small, explicit retrieval stage in your application. Query Talqora’s dense path, backed by regional S3 Vectors, and its sparse path, backed by Quickwit BM25, then merge the returned document IDs and ranks. Preserve source-specific metadata during development so that you can inspect why a result appeared.

Evaluate representative queries before changing candidate depths or k. Include semantic questions, exact product names, error messages, abbreviations, and mixed queries. If a document is strong in either retrieval mode, RRF gives it an opportunity to reach the final ranking without forcing both systems to agree.

  • Log the dense rank, BM25 rank, and fused rank for returned documents.
  • Deduplicate by canonical document ID before presenting results.
  • Keep candidate depth high enough for each retrieval mode to contribute.
  • Review queries where one retrieval mode consistently supplies the useful result.