Why raw score blending is fragile

It is tempting to retrieve a dense list and a BM25 list, normalize both score ranges, and calculate a weighted total. This can work in a tightly controlled experiment, but it creates a maintenance burden. Score distributions can shift when embeddings change, when the corpus grows, or when a query contains rare terms that BM25 rewards strongly.

Rank-based fusion makes a narrower and more stable assumption: an item near the top of either list is useful evidence. It does not require a dense score of 0.8 to mean the same thing as a BM25 score of 8.0. That makes it a practical default when the two retrieval systems are independently scored.

  • Dense retrieval helps with conceptual and paraphrased queries.
  • BM25 helps preserve exact-term, code, name, and identifier matching.
  • Raw scores should be considered system-specific signals, not a shared unit of relevance.

Apply reciprocal rank fusion to a shared candidate set

For each query, request a bounded candidate list from dense retrieval and another from BM25. Match returned records using a stable document or chunk identifier. For every unique result, add a contribution from each list in which it appears: 1 divided by k plus its one-based rank. The final fused score is the sum of those contributions.

The constant k reduces the gap between adjacent top ranks. A commonly used starting point is 60, but it is a tuning choice rather than a universal rule. More importantly, keep candidate depths and the value of k explicit in configuration, then inspect representative queries before changing them.

  • Use the same chunk IDs and metadata filters for both retrieval paths where possible.
  • Deduplicate before presenting or reranking results.
  • Record source ranks alongside the fused score for debugging.
  • Start with equal contributions from dense and sparse lists before adding weights.

Evaluate failures, not just the fused top result

A useful review set contains queries with different retrieval needs: natural-language questions, product names, error messages, abbreviations, and long-tail identifiers. For each query, compare the dense-only, BM25-only, and fused candidate lists. The goal is to learn whether fusion recovers relevant items that either individual retriever misses.

Also inspect disagreement. If BM25 repeatedly finds the correct document for identifier-heavy queries, do not dilute that behavior with an arbitrary dense-score conversion. If dense retrieval repeatedly finds useful paraphrases that BM25 misses, ensure the dense candidate depth is large enough for those results to enter fusion. RRF is simple, but its quality still depends on the candidates it receives.

  • Label a small set of representative queries and expected relevant documents.
  • Review queries where only one retriever returns the relevant item.
  • Check that filtering and access-control rules are applied consistently before fusion.
  • Treat fusion parameters as versioned retrieval configuration, not hidden application logic.