Why raw dense and BM25 scores should not be added directly

Dense retrieval and BM25 rank documents using different mathematical signals. A dense score reflects a relationship in embedding space, while BM25 is driven by term frequency, document frequency, and length normalization. Even when both systems return numeric scores, a score of 0.8 from one system does not inherently represent the same relevance strength as 0.8 from the other.

Adding or averaging raw scores can therefore produce unstable behavior. A small scoring-distribution change in either retrieval path may disproportionately alter the final ordering, even when the underlying relevance of documents has not changed.

  • Dense retrieval is useful for semantic similarity and paraphrases.
  • BM25 is useful for exact language, rare terms, codes, and names.
  • Score normalization requires assumptions that may not hold across queries.
  • Rank-based fusion avoids treating unlike scores as equivalent.

Fuse ranked lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in each ranked list. For every document returned by dense search or BM25, add 1 divided by k plus its rank. The final fused score is the sum of those contributions, and documents are sorted by that total.

The constant k reduces the difference between adjacent high ranks and lower ranks. This helps prevent one retrieval method from dominating solely because it places a document slightly higher, while still rewarding documents that appear near the top of either list or in both lists.

  • For a document d, use: RRF(d) = sum of 1 / (k + rank_i(d)).
  • Use one ranked list from dense search and one from BM25.
  • Choose a consistent rank convention, such as rank 1 for the first result.
  • Deduplicate documents by a stable document identifier before returning results.

Apply RRF at the retrieval boundary

In a Talqora retrieval workflow, an application can request dense candidates from regional S3 Vectors and sparse candidates from Quickwit BM25, then fuse the two ranked result sets before presenting a final candidate list to downstream logic. This keeps the fusion step explicit and makes its behavior inspectable.

Start by retrieving a fixed candidate depth from each path, recording each document’s dense rank, sparse rank, and fused rank. Review representative queries that include natural-language questions, exact identifiers, product terminology, and mixed queries. The goal is not to make every query look semantic or lexical, but to ensure that either retrieval signal can contribute useful candidates.

  • Retrieve enough candidates from both paths for overlap and complementary matches to emerge.
  • Log per-source ranks alongside the fused rank for debugging.
  • Use stable tie-breaking, such as a document ID, when fused scores match.
  • Evaluate changes with labeled queries or structured relevance review before adjusting k or candidate depth.