Why raw dense and BM25 scores should not be added

A dense retrieval score and a BM25 score are produced by different models and scoring systems. Their ranges, distributions, and meanings can vary by query, corpus, embedding model, analyzer configuration, and index settings. A score of 0.7 from one retrieval path does not inherently mean the same thing as a score of 0.7 from another.

Adding raw scores can therefore create unstable behavior. One retriever may dominate simply because its numerical scale is larger, not because its result is more useful. Normalization can help in some systems, but it introduces its own assumptions and requires careful monitoring as data and retrieval configurations change.

Rank-based fusion avoids this direct score comparison. Instead of asking whether two scores have the same meaning, it asks a simpler question: how highly did each retrieval method rank a document?

  • Dense search helps with semantic similarity and paraphrased intent.
  • BM25 helps preserve exact-match signals for product names, codes, and rare terms.
  • Raw score scales are not automatically comparable across retrieval methods.
  • A fusion layer can combine candidate lists after each retriever has ranked them.

Apply Reciprocal Rank Fusion at the candidate stage

RRF assigns a document a contribution based on its rank in each result list. For a document d, the fused score is the sum of 1 divided by k plus its rank for every list in which it appears: RRF(d) = Σ 1 / (k + rank_i(d)). The constant k reduces the difference between adjacent ranks and keeps the head of a single list from overwhelming every other signal.

In a Talqora-oriented architecture, an application can retrieve a candidate set from regional S3 Vectors for dense search and another from Quickwit BM25 for sparse search. It can then deduplicate document IDs, calculate an RRF score over the two ranked lists, and return the fused ordering. This keeps dense and sparse retrieval independent while making the combination logic explicit.

For example, a document ranked 2nd by dense retrieval and 12th by BM25 receives credit from both paths. A document found only by one path can still rank well, but agreement between methods is naturally rewarded.

  • Request a bounded top-N list from each retriever before fusion.
  • Use a stable document or chunk identifier for deduplication.
  • Start with the same candidate depth for both retrieval paths.
  • Keep the original dense rank and BM25 rank in logs for debugging.

Choose parameters and evaluate the fused ranking

There is no universal candidate depth or RRF constant. A common starting point is to retrieve several times more candidates than the number ultimately shown or passed to a later stage, then test whether relevant documents are being missed before fusion. The appropriate depth depends on corpus size, chunking strategy, query mix, and downstream context limits.

Evaluate fusion on representative queries rather than relying on a few memorable examples. Include semantic questions, exact identifier lookups, acronym-heavy queries, multi-term queries, and queries with ambiguous vocabulary. Compare dense-only, BM25-only, and fused rankings using judged relevance where available.

RRF is deliberately simple, which makes it a useful baseline. If it improves coverage and ranking quality, retain its transparency. If it exposes consistent query-specific weaknesses, those observations can guide later work such as query routing, field-aware retrieval, or a separate reranking stage.

  • Measure whether relevant items appear in the candidate set before judging final order.
  • Inspect failures by query type, not only aggregate metrics.
  • Version fusion settings alongside embedding, chunking, and analyzer changes.
  • Log retrieval-source membership to see when dense and sparse search agree or diverge.