Why raw score blending is fragile

Dense retrieval represents semantic proximity between a query embedding and document embeddings. BM25 ranks documents from lexical evidence such as matching terms and their frequency. Even when both produce a numeric score, the numbers do not inherently mean the same thing.

A weighted expression such as dense_score + sparse_score can therefore be unstable. A change to embedding models, text chunking, BM25 configuration, or query wording may alter one score distribution without an equivalent change in the other.

  • Dense search can surface semantically related passages that omit the exact query terms.
  • BM25 can strongly reward identifiers, product names, error codes, and other exact language.
  • Score ranges alone are not a reliable measure of cross-system relevance.

Fuse ranks with Reciprocal Rank Fusion

Run the same user query through the dense and sparse retrieval paths, retain a candidate list from each, and assign every result an RRF contribution based on its rank. For a document d, the fused score is the sum of 1 divided by k plus its rank in each list where it appears.

The constant k reduces the impact of tiny rank differences near the top of a list. RRF does not require dense and BM25 scores to be calibrated; it only requires each retrieval path to provide an ordered list of document identifiers.

  • Use a shared document or chunk identifier across dense and sparse indexes.
  • Deduplicate candidates before producing the final ranking.
  • Choose a candidate depth that gives both retrievers a chance to contribute useful results.
  • Keep the fusion logic explicit in the application layer when experimenting with retrieval policy.

Evaluate disagreement, not just aggregate relevance

RRF is especially useful when the two retrievers disagree for good reasons. Inspect queries where BM25 finds an exact operational term while dense retrieval finds explanatory language, and queries where dense retrieval recovers paraphrases that sparse matching misses.

Build a small evaluation set from real query classes: exact identifiers, short keyword searches, natural-language questions, and domain-specific phrasing. Compare dense-only, sparse-only, and fused rankings using the same relevance judgments before changing fusion settings.

  • Log which retrieval path contributed each fused result.
  • Review the top results for queries with zero overlap between candidate lists.
  • Test chunking changes separately from fusion changes.
  • Treat k and candidate depth as evaluation parameters, not universal constants.