Why raw score blending is fragile

A dense retrieval score and a BM25 score are produced by different ranking functions. Their numeric ranges, distributions, and sensitivity to query length can differ substantially. Adding them directly can make one retriever dominate merely because its scores have a larger scale, not because its result is more relevant.

Score normalization can help, but it introduces choices about calibration windows, query classes, and handling outliers. For many retrieval systems, it is simpler to first combine rankings rather than attempting to interpret unrelated score values as a shared relevance scale.

  • Dense retrieval favors semantic similarity and paraphrases.
  • BM25 favors exact terms, uncommon vocabulary, and token overlap.
  • Raw scores from separate retrieval methods should not be assumed comparable.
  • Rank fusion keeps each retriever’s internal scoring model independent.

Fuse ranked lists with reciprocal rank fusion

Run a dense query against the vector index and a sparse query against the BM25 index. Keep a ranked list from each response, using a stable document identifier to identify duplicates. RRF assigns each document a contribution based on its position in every list where it appears.

For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d, r), for each retriever r. The constant k reduces the difference between adjacent top positions and prevents a single rank-one result from overwhelming all other evidence. Choose k deliberately, document it, and evaluate it against representative queries rather than assuming one value is universally best.

  • Request a sufficiently deep candidate list from both retrievers before fusion.
  • Use one canonical document or chunk ID across dense and sparse indexing paths.
  • Add each list’s reciprocal-rank contribution for duplicate IDs.
  • Sort by fused score, then apply a deterministic tie-breaker such as document ID.

Evaluate by query behavior, not only aggregate relevance

Hybrid fusion is most useful when evaluated across the query patterns users actually submit. Build a small labeled set that includes exact identifiers, short ambiguous searches, natural-language questions, terminology variants, and queries containing both a named entity and a descriptive need. Compare dense-only, BM25-only, and fused rankings on the same set.

Inspect failures at the result level. If BM25 repeatedly retrieves stale or overly broad lexical matches, the issue may be chunking, field selection, or index content rather than the fusion formula. If dense retrieval misses important exact terms, preserving those terms in sparse indexing and retaining BM25 in the fused path is often more reliable than trying to encode every lexical requirement into an embedding.

  • Track recall-oriented measures at the candidate depth used before reranking.
  • Review results separately for exact-match and semantic queries.
  • Version the fusion rule alongside index and embedding changes.
  • Log which retriever contributed each final result for debugging.