Why raw dense and sparse scores should not be added directly

A dense retriever ranks documents by proximity in an embedding space. BM25 ranks documents from lexical evidence such as matching terms, term rarity, and document length. Both systems can produce a useful ordering, but their numeric scores are generated by different models and are not naturally comparable.

Adding those values together can make retrieval sensitive to changes that have little to do with relevance. A new embedding model, a different similarity convention, a revised analyzer, or an index rebuild may shift score ranges. The fusion behavior then changes even when the relative ordering within each retriever remains good.

  • Treat dense and sparse scores as model-specific signals.
  • Use held-out relevance judgments before relying on score normalization.
  • Prefer a rank-based baseline when score semantics are uncertain.

Fuse two ranked lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in every result list where it appears. For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) across the dense and sparse lists. The constant k reduces the impact of small rank differences near the top of a list.

For example, request a candidate set from dense search and another from Quickwit BM25, preserve each list's one-based rank, then aggregate contributions by document identifier. Sort the merged documents by their RRF score and return the highest-ranked entries. A document retrieved by both methods is naturally rewarded, while a strong result unique to one method can still remain competitive.

  • Use the same stable document ID in both retrieval paths.
  • Start with a fixed candidate depth for each list, such as the top N results.
  • Choose k as a tunable relevance parameter rather than assuming one universal value.
  • Deduplicate before presenting results to the application.

Evaluate the fusion policy as part of the retrieval pipeline

RRF is intentionally simple, but it is still a retrieval decision that should be tested. Build a query set that includes exact identifiers, short keyword queries, natural-language questions, synonyms, and terminology that appears rarely in the corpus. For each query, define one or more relevant documents and compare dense-only, BM25-only, and fused rankings.

Inspect failures rather than only aggregate metrics. Dense retrieval may surface conceptually related material that misses a required product code, while BM25 may find the code but miss paraphrased documentation. These patterns help determine candidate depths, the RRF constant, and whether specific query classes need routing or additional filtering.

  • Measure ranking quality at the result depth users actually inspect.
  • Keep a regression set when embeddings, analyzers, or document chunking change.
  • Log which retriever contributed each fused result for debugging.
  • Apply metadata filters consistently before fusion when a query requires them.