Why raw-score blending is fragile

Dense retrieval scores and BM25 scores are produced by different models and have different distributions. A dense similarity score reflects the relationship between embeddings, while BM25 is driven by term frequency, document frequency, and query-term matching.

Adding those values together requires calibration choices: normalization method, weighting, clipping, and behavior for unusual queries. Those choices can change which documents win even when the underlying dense and sparse retrievers have not changed.

  • Dense retrieval can surface semantically related wording.
  • BM25 can preserve exact identifiers, rare terms, and literal phrases.
  • Raw score ranges should not be assumed to be comparable.
  • A merge strategy should handle documents returned by only one retriever.

Fuse ranked lists with reciprocal rank fusion

Reciprocal rank fusion (RRF) combines ranked lists without requiring their scores to share a scale. Run a dense query against the vector index and a BM25 query against the sparse index, then assign each returned document a contribution based on its rank.

For a document d, a common form is RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is its one-based position in result list i and k is a positive constant. Documents found by both retrievers accumulate contributions, while a high placement in either list remains useful.

  • Use the same document identifier in both retrieval paths.
  • Request a bounded candidate list from each retriever before fusion.
  • Deduplicate by document identifier before producing final results.
  • Choose k as a configuration value and validate it with representative queries.

Make fusion observable and testable

A fused ranking is easier to operate when each result retains its provenance. Store whether a document came from dense retrieval, BM25 retrieval, or both, along with its per-source rank and final fused score. This makes surprising results inspectable without treating the final rank as a black box.

Evaluate hybrid retrieval with a query set that reflects production language: natural-language questions, short keyword queries, product names, identifiers, acronyms, and misspellings where relevant. Review not only the final top results, but also whether each retrieval path contributes useful candidates.

  • Log dense rank, BM25 rank, and fused rank for returned documents.
  • Separate retrieval failures from metadata or filtering failures.
  • Test candidate depths independently from the final result count.
  • Revisit the query set as corpus vocabulary and user behavior change.