Why adding raw dense and sparse scores is fragile

Dense retrieval and BM25 solve related but different matching problems. Dense vectors can retrieve semantically similar content even when wording differs. BM25 favors terms that appear in both the query and document, which is often valuable for identifiers, product names, error codes, and exact phrases.

The problem begins when an application tries to create one score by adding the two values directly. A score of 12 from BM25 does not have an inherent meaning relative to a dense similarity of 0.82. Even after normalization, score distributions can vary by query, corpus composition, and the number of matching terms.

A safer first fusion strategy is to treat each retriever as a source of ordering rather than as a source of directly comparable numeric evidence.

  • Dense search is useful when the query and relevant text use different language.
  • BM25 is useful when exact terminology carries meaning.
  • Raw score scales may change across indexes and queries.
  • Ranking positions are easier to combine than uncalibrated scores.

Fuse two ranked lists with reciprocal rank fusion

Reciprocal rank fusion assigns a contribution to a document based on its position in each result list. For every dense and BM25 result list, add 1 divided by k plus the document rank. The document's final RRF score is the sum of those contributions across lists.

In formula form, RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is document d's one-based rank in result list i. The constant k reduces the difference between nearby top positions and prevents rank one from overwhelming all other evidence. A common starting value is 60, but it should be treated as an application parameter, not a universal default.

For Talqora-based applications, retrieve a bounded candidate list from regional S3 Vectors for dense search and another from Quickwit BM25 for sparse search. Deduplicate documents by a stable document ID, compute the fused rank in application logic, then return the top fused results.

  • Choose a candidate depth for each retriever, such as the top 20 to top 100 results.
  • Use the same stable document ID in both retrieval paths.
  • Assign rank 1 to the first result in each list.
  • Documents appearing in both lists receive evidence from both retrieval methods.

Make fusion observable before tuning it

RRF is simple enough to implement, but its effectiveness depends on the queries and relevance expectations of the product. Log the dense rank, BM25 rank, and final fused rank for each returned document. These fields make it possible to explain why a result appeared and to identify whether one retriever is consistently missing useful candidates.

Start with a small evaluation set containing representative searches: natural-language questions, exact names, abbreviations, mixed queries, and queries with little context. For each query, record a few relevant documents and compare dense-only, BM25-only, and fused rankings. This is more informative than judging fusion from a handful of memorable examples.

If exact-match queries need stronger treatment, increase sparse candidate depth or apply a narrowly scoped business rule for verified identifiers. If semantic queries lack coverage, inspect embedding input construction and dense candidate depth before changing fusion weights.

  • Log per-source ranks alongside the fused rank.
  • Evaluate retrieval quality separately for query classes.
  • Watch for duplicate content represented by different document IDs.
  • Keep fusion logic deterministic so ranking changes are traceable.