Why raw dense and sparse scores should not be added

A dense retriever returns scores based on its embedding representation and similarity measure. A BM25 retriever returns scores based on term frequency, inverse document frequency, document length, and query terms. Even when both systems rank useful documents highly, a score of 0.8 from one system does not have a universal relationship to a score of 8 from the other.

Score normalization can be useful when it is measured and validated against representative relevance judgments. But simple approaches such as min-max scaling on each result page can be unstable: the same document may receive a different normalized value as the query, corpus, or candidate set changes. Rank-based fusion avoids treating unlike scores as though they share a common scale.

  • Dense retrieval helps with paraphrases, related concepts, and vocabulary mismatch.
  • BM25 helps with literal phrases, product names, codes, and rare tokens.
  • Raw score ranges and distributions are retrieval-system specific.
  • Rank positions are easier to combine than uncalibrated scores.

Apply reciprocal rank fusion to the two result lists

Reciprocal rank fusion, commonly abbreviated RRF, assigns each document a contribution based on its position in each ranked list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus its rank for every list in which it appears. The constant k reduces the difference between nearby ranks and prevents the first result from dominating the fused score.

For example, an application can request the top candidates from Talqora's dense retrieval path, backed by regional S3 Vectors, and the top candidates from its sparse BM25 path, backed by Quickwit. It can then deduplicate document IDs, calculate an RRF score for each candidate, sort descending, and return the leading fused results. A document appearing in both lists receives two contributions, while a strong exact-match-only or semantic-only result can still remain competitive.

  • Choose a shared candidate depth, such as the top 50 or top 100 results from each retriever.
  • Use a stable document ID to deduplicate candidates across lists.
  • Treat rank as one-based: the first result has rank 1.
  • Start with a conventional k value such as 60, then tune it using relevance feedback.

Make fusion observable and test it with real queries

RRF is deliberately simple, but it is still a retrieval policy that should be evaluated. Build a small query set from search logs, support questions, or representative workflows. For each query, record whether the expected document appears near the top for dense-only, BM25-only, and fused retrieval. Include queries with quoted phrases, acronyms, natural-language questions, and domain-specific identifiers.

Log the source ranks that produced each fused result. This makes failures diagnosable: a result may have won because it ranked well in both systems, because BM25 found an exact token, or because dense retrieval recognized related language. Those observations can guide later work on chunking, metadata filters, candidate depth, embedding selection, or query-specific routing.

  • Compare dense-only, BM25-only, and fused rankings on the same query set.
  • Inspect queries where the expected result is absent from both candidate lists.
  • Track document IDs and source ranks alongside the final fused rank.
  • Keep retrieval fusion separate from any later reranking step so each stage can be evaluated.