Why raw dense and sparse scores should not be added

A dense-search score and a BM25 score are produced by different retrieval models with different distributions. Even if both are represented as numbers where larger is better, their absolute values do not share a common meaning. A score that looks large in one retriever may be ordinary in the other.

This makes fixed weighted sums fragile. A tuning choice that works for short natural-language questions can behave poorly for queries dominated by an exact token, such as a ticket number or configuration key. Score ranges can also shift when an index, embedding model, corpus, or query pattern changes.

  • Dense retrieval is useful for semantic similarity and paraphrased language.
  • BM25 rewards query-term matches and is valuable for exact lexical evidence.
  • Raw score scales are model-specific, so direct arithmetic can create accidental bias.

Fuse ranked lists with RRF

RRF avoids score calibration by using positions in each ranked list. Retrieve a candidate list from dense search and another from BM25, then assign every document a contribution based on its rank. Documents that appear near the top of one or both lists accumulate a stronger fused score.

For a document d, calculate: RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs over retrieval lists in which d appears, rank_i(d) starts at 1, and k is a positive constant that reduces the influence of very small rank differences. The final ordering is the descending order of the fused score.

  • Request a sufficiently deep candidate list from each retriever before fusion.
  • Use the same document identifier in dense and sparse result sets so duplicates can be merged.
  • Choose k deliberately and keep it under configuration control rather than embedding it in application code.
  • Apply authorization and tenant filters before results reach the user.

Implement, inspect, and tune the retrieval path

In an API-first architecture, a retrieval service can issue a dense request to the vector store and a sparse request to the BM25 index in parallel. It then joins results by document ID, computes RRF scores, and returns the best fused candidates to the calling application. Talqora's use of regional S3 Vectors for dense search and Quickwit BM25 for sparse search maps naturally to this two-list pattern.

Evaluation should focus on representative queries rather than a single aggregate number. Build a small labeled set that includes natural-language questions, acronym-heavy requests, identifiers, and mixed queries. For each query, inspect whether the relevant document was retrieved by dense search, sparse search, both, or neither. That breakdown tells you whether to adjust candidate depth, improve content preparation, or revisit the retrieval strategy.

  • Log source ranks and the fused rank for each returned document.
  • Keep a query set containing both semantic and exact-match retrieval cases.
  • Investigate documents found by only one retriever; they reveal complementary coverage.
  • Use reranking, if introduced, after candidate fusion rather than as a substitute for candidate diversity.