Why raw-score blending is fragile

A dense search score reflects the similarity function and vector representation used for a query and document. A BM25 score reflects term statistics, document length normalization, and query-term matches. Even when both scores increase for better results, a score of 0.7 from one retriever has no inherent relationship to a score of 0.7 from the other.

Direct score addition therefore introduces an implicit calibration problem. It can require per-index tuning, may change as content evolves, and can produce surprising behavior when a query contains both an exact product code and a broader natural-language intent. Rank-based fusion avoids assuming that the underlying values are comparable.

  • Use BM25 to preserve exact lexical matches such as names, codes, and quoted phrases.
  • Use dense retrieval to recover meaning when relevant documents use different wording.
  • Treat each retriever's output as an ordered list before combining results.

Fuse dense and sparse lists with RRF

RRF assigns each document a contribution based on its position in each result list. For a document d, compute RRF(d) = sum of 1 / (k + rank_i(d)) across the lists where d appears. The rank is one-based, and k is a positive constant that reduces the difference between nearby ranks.

For example, retrieve the top N candidates from S3 Vectors and the top N candidates from Quickwit BM25. Deduplicate documents by a stable document identifier, add their RRF contributions, then sort by the fused score. A document that ranks well in both lists rises naturally, while a strong result from only one retriever can still remain competitive.

  • Start with the same candidate depth for both retrievers to make evaluation easier.
  • Use a stable identifier when deduplicating results returned by both paths.
  • Keep the dense rank and BM25 rank for debugging and relevance analysis.
  • Choose k as a deliberate configuration value and validate it on representative queries.

Evaluate fusion by query class, not only averages

Hybrid retrieval is most useful when query types vary. Build a small evaluation set that includes exact identifiers, short keyword queries, natural-language questions, synonym-heavy requests, and ambiguous terms. Label one or more relevant documents for each query, then compare dense-only, BM25-only, and fused rankings at the result depth your application presents.

When a fused result looks wrong, inspect the two source ranks before changing weights or retrieval depth. If an identifier is missing from the BM25 list, investigate tokenization and indexing. If conceptually relevant documents are absent from dense candidates, inspect document chunking and embedding input. Fusion can combine available evidence, but it cannot recover documents neither retriever returns.

  • Track whether relevant documents appear in dense candidates, sparse candidates, or both.
  • Review failures separately for exact-match and semantic-intent queries.
  • Log fused rank alongside source ranks to make ranking decisions explainable.