Why raw dense and sparse scores should not be added blindly

A dense retrieval score reflects the relationship between query and document embeddings. A BM25 score reflects term-frequency and corpus-statistics signals. Even when both are useful relevance indicators, their numeric ranges and distributions are produced by different ranking functions.

Adding those scores together assumes that a one-point change means the same thing in both systems. That assumption can make a hybrid ranker sensitive to query length, vocabulary rarity, embedding behavior, and index-specific scoring details. The result may look reasonable for one query class and fail quietly for another.

  • Exact product names, error codes, and identifiers often benefit from sparse retrieval.
  • Paraphrases and conceptually related language often benefit from dense retrieval.
  • Score scales can shift across queries, documents, and retrieval implementations.
  • A rank-based merger avoids requiring a shared score scale.

Fuse ranks with a simple RRF calculation

Run the query against both retrieval paths, then retain each document's position in each result list. For every document returned by at least one path, calculate an RRF score by summing 1 divided by k plus the document rank for each list where it appears. Sort documents by the combined result.

In notation, RRF(d) = Σ 1 / (k + rank_i(d)). The constant k reduces the difference between adjacent top ranks and keeps a single rank-one placement from overwhelming the rest of the evidence. A document returned by both dense search and BM25 gains support from both lists, while a strong result from only one list can still appear.

  • Use rank 1 for the first result in each list.
  • Choose one k value and evaluate it against representative queries before changing it.
  • Deduplicate on a stable document or chunk identifier before calculating the final order.
  • Fetch more than the final display count from each retriever so fusion has useful candidates.

Apply RRF to Talqora retrieval pipelines

Talqora uses regional S3 Vectors for dense search and Quickwit BM25 for sparse search. A client or application retrieval layer can issue the corresponding dense and sparse requests, collect their ranked candidates, and apply RRF before passing the merged context to a user interface or downstream generation step.

Start with observability rather than aggressive tuning. Record which branch retrieved each final result, its rank in that branch, and whether it was selected by both. Reviewing those traces against a small, labeled query set makes it easier to spot cases where exact-match queries need more sparse candidates or semantic queries need more dense candidates.

  • Use the same filtering and access-control rules for both retrieval branches.
  • Keep document chunking and identifiers consistent across dense and sparse indexes.
  • Test queries containing both natural-language questions and exact domain terms.
  • Measure relevance on a held-out query set before modifying candidate counts or fusion settings.