Why raw-score blending is fragile

A dense retriever and a BM25 retriever can both return a number called a score, but the values reflect different ranking functions. A dense score is associated with the relationship between vector representations, while BM25 scores are driven by term frequency, document frequency, and document-length normalization. Adding those values directly assumes they are calibrated to one another.

That assumption can break as the corpus changes, embedding models change, analyzers change, or query types vary. A score weight that looks reasonable for natural-language questions may behave poorly for part numbers, quoted strings, or short acronym-heavy queries. Rank fusion avoids requiring a universal conversion between the two score distributions.

  • Use raw scores within each retriever to produce its own ordered list.
  • Treat dense and sparse scores as retriever-specific evidence, not a shared relevance currency.
  • Keep the source rank and source name for every returned candidate.
  • Log score distributions separately if you need to diagnose either retriever.

Fuse the dense and sparse lists with RRF

For a query, request a candidate list from dense search and another from BM25. Assign each document a contribution based on its rank in each list: 1 divided by k plus rank. Sum the contributions for documents that appear in one or both lists. The document with the larger combined total ranks higher.

The constant k reduces the difference between adjacent top ranks and prevents a single first-place result from overwhelming all other evidence. Its exact value is a product decision rather than a universal truth, so choose it deliberately and evaluate it against representative queries. The important property is that RRF consumes positions, not incomparable score magnitudes.

  • Formula: RRF(document) = Σ 1 / (k + rank) across result lists containing that document.
  • Use a stable, shared document ID to deduplicate candidates across dense and BM25 results.
  • Decide whether rank starts at 1 and apply that convention consistently.
  • Request enough candidates from each retriever that the fused list can contain useful overlap and useful disagreements.

Make fusion observable and deterministic

A hybrid ranker is easier to improve when every result carries an explanation. Record whether a result came from dense retrieval, BM25, or both; its rank in each source list; and its final RRF score. This makes it possible to inspect cases where exact-match content is lost, semantic matches are promoted, or a source contributes little to a query class.

Define deterministic behavior for ties and missing metadata before production traffic exposes them. For example, break equal RRF scores using the best source rank, then a stable document ID. Apply authorization and tenant filters consistently to both source candidate lists before fusion, so the fused result cannot reintroduce a document that should not be eligible.

  • Store per-result provenance: dense rank, BM25 rank, and fused rank.
  • Apply the same corpus, tenant, and access filters to both retrieval paths.
  • Use a deterministic tie-break rule for reproducible pagination and debugging.
  • Evaluate query groups separately, including natural-language questions, exact identifiers, and short keyword queries.