Why raw-score mixing is fragile

Dense retrieval scores commonly reflect vector similarity, while BM25 scores reflect lexical term matching and document statistics. Even when both systems rank relevant results well, their numeric outputs are not calibrated to a shared meaning.

Min-max normalization can appear to solve the issue, but it makes a query’s result range determine the weighting. One unusually strong or weak candidate can reshape the normalized distribution. That can make ranking behavior difficult to reason about and difficult to debug.

  • A dense score and a BM25 score should not be assumed to be directly comparable.
  • Per-query normalization can change the influence of a retriever unexpectedly.
  • Rank positions are simpler to interpret than uncalibrated relevance values.

Build two candidate lists, then apply reciprocal rank fusion

Run the same retrieval intent through both paths: a dense query against the vector index and a lexical query against BM25. Keep a bounded candidate list from each path, deduplicate documents by a stable document identifier, and assign a fused score based on each document’s rank.

Reciprocal rank fusion, or RRF, gives a document credit for appearing near the top of either list. For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i is its one-based position in retriever i and k is a smoothing constant. A larger k reduces the difference between adjacent ranks; choose it deliberately and keep it stable while evaluating changes.

  • Retrieve a fixed candidate depth from dense and sparse search.
  • Use one stable ID to deduplicate documents before final ordering.
  • Add each available rank contribution; a missing result contributes nothing.
  • Use deterministic tie-breaking, such as a stable document ID, for reproducible results.

Treat fusion as a measurable retrieval layer

RRF is not a substitute for checking whether the right documents are entering the candidate sets. Inspect queries with exact tokens, acronyms, long natural-language questions, and mixed queries containing both concepts and identifiers. These cases reveal whether one retriever is consistently absent from results that should benefit from it.

Keep filtering rules consistent across both retrieval paths. If a query is restricted to a tenant, corpus, language, or document state, the dense and sparse candidate lists should represent the same eligible document universe before fusion. Otherwise, the fusion layer can promote a result that should never have been considered.

  • Log each document’s dense rank, sparse rank, and final fused rank.
  • Review failures by query type instead of relying only on aggregate relevance judgments.
  • Change candidate depth, k, or query construction one variable at a time.
  • Apply equivalent eligibility filters before combining ranked lists.