Why raw-score blending is fragile

A dense retrieval score represents similarity within an embedding space. A BM25 score is produced from lexical term statistics, including term frequency and document-frequency effects. Even when both systems return a numeric score, those values do not share a stable unit or range.

A weighted formula such as dense_score + bm25_score can therefore make one retriever dominate for reasons unrelated to relevance. Score distributions can also shift as embeddings, document lengths, analyzers, or corpus composition change. Calibration is possible, but it requires evaluation data and ongoing maintenance.

  • Dense search can recover semantic matches that use different wording.
  • BM25 can preserve exact terms, identifiers, error codes, and rare phrases.
  • Raw scores should not be assumed comparable across retrieval methods.

Fuse ranks with a small, explicit algorithm

Reciprocal rank fusion assigns each document a contribution based on its position in every ranked list. For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is its one-based position in list i and k is a constant chosen by the application. Sum the contributions for documents returned by dense and sparse retrieval, then sort by the resulting total.

The method rewards documents that appear near the top of either list and gives an additional boost to documents found by both. Because it uses rank positions rather than score magnitudes, the application does not need to force S3 Vectors similarity values and Quickwit BM25 scores onto a common scale.

  • Request a bounded candidate list from dense retrieval and from BM25 retrieval.
  • Use a stable document ID as the key when deduplicating candidates.
  • Treat absent documents as contributing zero from that retriever.
  • Keep the per-retriever ranks for debugging and evaluation.

Implement fusion at the retrieval boundary

In an API-first retrieval flow, issue the dense and sparse queries for the same user request, collect their ranked document IDs, and apply RRF before returning final candidates to the caller or sending them to a reranker. The fused layer should also apply the same eligibility constraints to both paths, such as tenant boundaries, document status, or access filters.

Start with a fixed candidate depth and a fixed k value, then evaluate on representative queries. Inspect queries where the fused ranking differs from each individual retriever: these are often the clearest examples of whether semantic recall and lexical precision are complementing one another. Change one parameter at a time and retain query-level examples alongside aggregate relevance judgments.

  • Log dense rank, BM25 rank, and fused rank for every returned document.
  • Evaluate exact-match queries separately from natural-language questions.
  • Confirm filters are enforced before a document can enter the fused set.
  • Consider a later reranking stage only after validating the fused candidate set.