Why raw dense and sparse scores should not be added blindly

A dense retriever and a BM25 retriever produce scores through different mechanisms. A dense-search score reflects a relationship between vector representations, while a BM25 score is derived from term statistics and document length normalization. Even when both scores increase with relevance within their own result lists, equal numeric values do not imply equal evidence.

Adding raw scores can therefore make ranking behavior depend on incidental details such as embedding similarity conventions, analyzer changes, corpus growth, or query vocabulary. Score normalization can help in some systems, but it also creates another set of assumptions that must be monitored and retuned.

For Talqora applications that use regional S3 Vectors for dense search and Quickwit BM25 for sparse search, a rank-based fusion stage provides a useful baseline: retrieve independently, then merge by position rather than by raw score.

  • Dense retrieval is useful for semantic similarity and paraphrased intent.
  • BM25 is useful when exact words, product codes, names, and rare terms matter.
  • Raw score ranges are retrieval-method-specific rather than inherently comparable.

Fuse two candidate lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its rank in each retrieval list. For a document d, the fused score is the sum of 1 divided by k plus its rank, across the lists in which it appears: RRF(d) = Σ 1 / (k + rank_i(d)). A document that ranks well in either list receives meaningful credit; one that ranks well in both is promoted further.

The constant k reduces the gap between adjacent top positions and prevents the first few ranks from overwhelming the rest of the candidate pool. A commonly used starting point is k = 60, but it is a policy parameter, not a universal optimum. Evaluate it against representative queries and relevance judgments when those are available.

Use stable document IDs as the fusion key. If the same source document can be split into chunks, decide before implementation whether fusion should operate at chunk level or parent-document level. Mixing these levels can create duplicate-looking results and obscure why an item ranked highly.

  • Run dense and BM25 retrieval independently for the same query.
  • Request a candidate depth from each retriever, such as the top N results.
  • Deduplicate candidates by a stable ID before returning the fused ranking.
  • Sort by fused score, with a deterministic tie-breaker such as document ID.

Make fusion observable before making it more complex

Log the dense rank, BM25 rank, and final fused rank for every returned result. These fields make it possible to diagnose common outcomes: exact-match documents rescued by BM25, semantically relevant documents rescued by dense search, and documents that are consistently strong in both lists.

Evaluate hybrid retrieval with a query set that includes more than natural-language questions. Include acronym-heavy queries, known-item lookups, error messages, model numbers, short ambiguous queries, and paraphrases. A fusion policy that appears strong on one query shape may be weak on another.

Only add weighting, query classification, or score-based reranking after the simple fused baseline is understood. RRF is valuable not because it eliminates tuning, but because it gives retrieval teams an interpretable reference point for deciding whether added complexity is justified.

  • Record per-retriever ranks alongside the final result order.
  • Inspect queries where dense-only and BM25-only results disagree.
  • Track duplicate chunks and parent-document collisions.
  • Treat candidate depth and k as versioned retrieval configuration.