Why raw dense and sparse scores should not be added

Dense retrieval ranks documents by semantic similarity between embeddings. Sparse retrieval with BM25 ranks documents from term statistics such as query-term frequency, document length, and corpus frequency. Both produce ordered result sets, but their numeric scores come from different models and scales.

Adding those scores directly can make ranking sensitive to changes that are unrelated to relevance. A new embedding model, a different similarity metric, an analyzer adjustment, or an index rebuild can shift score distributions. If fusion depends on uncalibrated score magnitudes, a weight that worked previously may no longer behave as intended.

  • Dense search can recover paraphrases and conceptually related language.
  • BM25 is often strong for exact names, identifiers, error messages, and rare terms.
  • A score of 0.8 in one retrieval system is not inherently comparable to a score from another.
  • Rank position is a more portable signal than an uncalibrated raw score.

Fuse two candidate lists with reciprocal rank fusion

Run dense search against the vector collection and BM25 search against the sparse index, requesting a bounded candidate list from each. For every document that appears in either list, calculate an RRF score by summing 1 divided by k plus its rank for each list where it appears. Ranks are normally one-based, so the first result has rank 1.

The constant k reduces the advantage of a document appearing at the very top of one list. A commonly used starting value is 60, but it is a tuning parameter rather than a universal rule. Sort the union of candidates by the resulting RRF score, then return the top results or pass them to a later reranking stage.

  • RRF formula: score(d) = Σ 1 / (k + rank_i(d)).
  • Include a document only once in the merged set, even if it appears in both lists.
  • Use stable document identifiers shared by the dense and sparse indexes.
  • Keep enough candidates from each retriever to preserve complementary matches.

Operate fusion as a retrieval-layer contract

Treat the dense and sparse requests as independent retrieval operations whose outputs share a document ID and rank. Talqora's architecture—regional S3 Vectors for dense search and Quickwit BM25 for sparse search—makes this separation explicit. The application or retrieval layer can collect the two ranked responses and apply the same deterministic fusion logic for every query.

Evaluate changes with a representative query set rather than optimizing around a handful of memorable searches. Include semantic questions, exact-title lookups, product codes, multilingual phrasing where relevant, and short ambiguous queries. Review whether useful documents enter the merged candidate set, not only whether the first result changes.

  • Start with equal contribution from the dense and sparse lists.
  • Log ranks and source membership for debugging fused results.
  • Test candidate depths and the k value using relevance judgments or review sets.
  • Consider reranking only after fusion if a second-stage model is available in your stack.