Why raw dense and BM25 scores should not be added

A dense retrieval score reflects the similarity function and embedding model used to represent a query and document. A BM25 score reflects term frequency, document frequency, field length, and the BM25 configuration. Even when both systems rank the same document highly, the numeric values do not share a natural unit.

Adding raw scores can create unstable behavior. A small change to an embedding model, vector normalization approach, BM25 parameters, or indexed corpus can change score ranges without meaningfully changing relevance. The result is a fusion rule that appears simple but must be continually recalibrated.

  • Treat dense and BM25 scores as ranking signals, not interchangeable measurements.
  • Avoid choosing a fixed weight until score distributions have been examined over representative queries.
  • Preserve each retriever's rank position and document identifier when collecting candidates.
  • Use a deterministic tie-breaker, such as a stable document ID, for reproducible results.

Merge candidate lists with reciprocal rank fusion

RRF assigns each document a contribution based on its position in each ranked list rather than its original score. For a document d, the fused score is the sum of 1 divided by k plus its rank in every list where it appears. The constant k reduces the difference between adjacent positions near the top of a list.

In a Talqora-oriented retrieval pipeline, an application can request dense candidates from regional S3 Vectors and sparse candidates from Quickwit BM25, then merge the returned document IDs with RRF. This keeps the fusion logic explicit in the application layer and avoids requiring dense and sparse scores to be normalized together.

  • Use the formula: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Start with the same candidate depth for both retrievers so one list is not accidentally underrepresented.
  • Choose a k value deliberately and evaluate it against real queries; k controls how strongly top ranks are emphasized.
  • Deduplicate by a canonical document or chunk ID before returning the fused ranking.

Make fusion useful with clean identities and evaluation slices

Fusion only works when both retrieval paths refer to the same logical content. A chunk indexed for dense retrieval and its BM25 counterpart should share a canonical identifier, even if the systems store different fields or serialization formats. Without that identity mapping, relevant candidates cannot receive credit from both lists.

Evaluate the fused list by query type rather than relying on a single aggregate metric. Exact product names, log messages, and part numbers often reveal the value of sparse retrieval. Paraphrased questions and concept searches often reveal the value of dense retrieval. The hybrid result should improve the overall query mix without hiding regressions in either slice.

  • Maintain a stable mapping from source document to retrieval chunk IDs.
  • Record which retriever contributed each returned candidate for debugging.
  • Create evaluation slices for exact-match, semantic, short-query, and long-query behavior.
  • Inspect failed queries before changing candidate depth, k, or downstream reranking.