Why raw-score blending is fragile

Dense retrieval ranks documents by semantic proximity between an embedding for the query and embeddings for stored content. Sparse retrieval with BM25 ranks documents from lexical evidence such as matching terms, their frequency, and their distribution in the indexed corpus. Both methods return a ranking, but the numerical scores have different meanings.

A weighted expression such as `0.5 * dense_score + 0.5 * bm25_score` assumes that score ranges are stable and comparable. That assumption can fail as query wording changes, the corpus grows, analyzers change, or embedding models are updated. A document may be genuinely strong in one system while its score is numerically smaller than a weaker result from the other.

  • Use dense retrieval for semantic phrasing, paraphrases, and concept-level matches.
  • Use BM25 for exact identifiers, uncommon names, error codes, and literal terms.
  • Treat scores from separate retrieval systems as system-specific signals unless they have been deliberately calibrated.

Fuse ranks with Reciprocal Rank Fusion

RRF replaces each result score with a contribution based on its position in a ranked list. For every document returned by either search path, calculate `RRF(d) = Σ 1 / (k + rank_i(d))`, where `rank_i(d)` is the one-based position of document `d` in result list `i`. Documents appearing near the top of one or both lists receive the highest combined score.

The constant `k` dampens the advantage of the first few positions. A commonly used starting value is 60, but it is a tuning parameter rather than a universal rule. The important property is that RRF relies on ordering, not on a claim that a BM25 score of one value equals a dense similarity score of another value.

  • Request a candidate list from dense search and a candidate list from BM25 search.
  • Deduplicate results using a stable document identifier.
  • Assign each document one reciprocal-rank contribution per list in which it appears.
  • Sort documents by the sum of their contributions and retain the top results.

Implement fusion as a retrieval-layer contract

In an application using Talqora's dense and sparse retrieval paths, perform the two searches for the same user query, then fuse their returned identifiers in the application or retrieval service. Fetch enough candidates from each path that the fusion step has useful overlap and alternatives; fusing only a very short list can hide documents that one retriever ranked just below its initial cutoff.

Keep the fused result explainable. Store or log the dense rank, BM25 rank, and final RRF score for each returned document. Those fields make it easier to investigate why a result was selected, spot missing identifiers, and evaluate whether a change to query construction or candidate depth improves the result set.

  • Use the same document ID across dense and sparse indexes so deduplication is deterministic.
  • Apply metadata filters consistently to both retrieval paths when the product requires filtered search.
  • Choose candidate depths and the RRF constant using representative queries and relevance judgments.
  • Consider a reranking stage only after establishing a reliable fused candidate set.