Why raw dense and sparse scores should not be added

Dense retrieval and BM25 answer different signals. Dense search can surface semantically related text even when wording differs, while BM25 rewards term overlap and distinctive query terms. Both are valuable, but their numerical scores are not inherently interchangeable.

A weighted sum such as dense_score + bm25_score can behave unpredictably unless both systems have been carefully normalized and monitored across query types. A score distribution may also shift when embeddings, document length patterns, analyzers, or corpus contents change. Ranking positions are usually a safer common representation than raw scores.

  • Use dense retrieval for semantic paraphrases and conceptual matches.
  • Use BM25 for exact terms, identifiers, names, and rare vocabulary.
  • Treat each retriever's score as local to that retriever unless calibration is explicitly validated.

Fuse ranked lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its rank in each retrieval list, then sums those contributions. For a document d, the fused score is: RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs over retrieval lists in which d appears, and k is a positive constant that reduces the influence of very high ranks.

In a Talqora-style hybrid pipeline, retrieve a top-N list from regional S3 Vectors and a top-N list from Quickwit BM25. Deduplicate documents by a stable document identifier, calculate the RRF score for each identifier, and sort descending. A document appearing in both lists is rewarded, while a strong result from only one retriever can still remain competitive.

  • Choose a stable, shared document ID before retrieval results are merged.
  • Request enough candidates from each retriever to create useful overlap and coverage.
  • Apply the same rank convention everywhere: rank 1 is the highest-ranked result.
  • Keep the per-retriever ranks and fused score in logs for debugging.

Make fusion observable and tune it by query class

RRF has few moving parts, but it still benefits from evaluation. Build a small relevance set from representative searches: exact product names, error strings, natural-language questions, acronym-heavy queries, and queries with ambiguous terminology. Compare dense-only, BM25-only, and fused rankings using the same judged documents.

Start with equal participation from both retrieval paths. If one source is systematically less useful for a known query class, adjust candidate counts, introduce a query-routing rule, or use a weighted version of RRF. Make changes based on observed relevance, not on the apparent magnitude of either system's raw scores.

  • Track which retriever contributed each returned document.
  • Inspect failures where relevant documents were outside both initial candidate lists.
  • Evaluate identifier-heavy and semantic queries separately.
  • Re-run relevance checks when embeddings, indexing settings, or document processing change.