Why raw dense and sparse scores are difficult to combine

A dense retrieval score reflects the relationship between query and document embeddings. A BM25 score is derived from term overlap, term rarity, and document-length normalization. Even when both systems return numeric scores, those values do not have a shared interpretation.

Adding or averaging uncalibrated scores can produce unstable rankings. A small change in embedding model, corpus composition, BM25 configuration, or query wording may alter one score distribution without changing actual relevance. The result is a fusion rule that appears simple but is difficult to reason about.

Talqora’s use of regional S3 Vectors for dense search and Quickwit BM25 for sparse search makes this distinction especially relevant: each retrieval path can contribute useful candidates, but their native score scales should be treated as separate signals.

  • Dense search helps with paraphrases and conceptual similarity.
  • BM25 helps with exact phrases, product names, codes, and rare terms.
  • Native scores are ranking signals within their own retrieval method, not automatically across methods.

Fuse ranked lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in each result list. For a document d, the fused score is the sum of 1 divided by k plus its rank: RRF(d) = Σ 1 / (k + rank_i(d)). The sum is taken over the lists in which the document appears.

The constant k reduces the gap between adjacent positions and prevents the first few ranks from overwhelming every other result. A commonly used starting value is 60, but it is a tuning choice rather than a universal rule. Keep it explicit in configuration so that ranking changes can be evaluated and rolled back.

Because RRF consumes ranks, not raw similarity values, it avoids the need to normalize dense and BM25 scores into an artificial common range. A document that performs well in both lists rises naturally; a document that is strong in only one list can still be retained.

  • Retrieve a bounded candidate list from dense search.
  • Retrieve a bounded candidate list from BM25 search.
  • Assign ranks independently within each list, starting at 1.
  • Sum RRF contributions by document ID, then sort by the fused score.

Make the fusion layer observable and query-aware

Start with equal treatment for the dense and sparse lists, then inspect representative queries before adding complexity. Queries containing quoted text, error codes, SKUs, or names often benefit from sparse retrieval. Broad natural-language questions may receive more useful candidates from dense retrieval. These are hypotheses to test against relevance judgments, not fixed laws.

Log the dense rank, sparse rank, and final fused rank for every returned document. Those fields make it possible to diagnose why an item appeared: it may have ranked highly in one system, moderately in both, or entered only after fusion. This is more actionable than recording a final score alone.

Keep candidate depth separate from the number of results shown to a user. Fusion needs enough candidates from each retrieval path to discover overlap and complementary matches, while the application may display only a small final set. If relevant documents are absent from both input lists, no fusion method can recover them.

  • Record per-source rank and whether a document appeared in one or both lists.
  • Evaluate exact-term and semantic-paraphrase query sets separately.
  • Tune candidate depth and k independently.
  • Apply any later reranking step after building the fused candidate set.