Why raw score blending is fragile

Dense retrieval scores and BM25 scores represent different ranking systems. Their ranges, distributions, and meanings can change with the embedding model, corpus composition, analyzer settings, query length, and retrieval configuration. Adding the two scores directly can therefore make one retriever dominate for reasons unrelated to relevance.

Score normalization can help in tightly controlled systems, but it requires careful calibration and ongoing evaluation. For an initial hybrid design, combining ranks is often easier to reason about because each retriever contributes an ordered list rather than a supposedly universal relevance number.

  • Dense search is useful for semantic similarity and paraphrases.
  • BM25 preserves signal from exact tokens and rare terms.
  • Raw score scales should not be assumed to be interchangeable.
  • Rank-based fusion reduces dependence on score calibration.

Fuse dense and sparse candidates with RRF

RRF assigns each document a contribution based on its position in each result list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) across the lists in which d appears. The constant k dampens the difference between adjacent ranks and is commonly chosen as a fixed tuning value.

In a Talqora-oriented architecture, an application can obtain a dense candidate list from regional S3 Vectors and a sparse candidate list from Quickwit BM25, then perform this small fusion step in its retrieval service. The fused list becomes the candidate set for presentation, filtering, or a later reranking stage.

  • Retrieve a bounded top-N list from dense search.
  • Retrieve a bounded top-N list from BM25 using the same query context.
  • Deduplicate by a stable document or chunk identifier.
  • Sum RRF contributions and sort documents by the resulting score.

Make fusion observable and testable

Log more than the final rank. For each returned item, record whether it came from dense retrieval, BM25, or both, along with its rank in each source list. This makes it possible to diagnose cases where exact-match content disappears, semantic matches are missing, or one source contributes very little.

Evaluate the fused system with queries that reflect real retrieval work. Include identifier-heavy queries, short ambiguous queries, paraphrased questions, and queries using terminology absent from a target document but present in related language. Review both relevance and diversity in the final candidate set before changing list depths or the RRF constant.

  • Keep source ranks and fusion inputs in retrieval logs.
  • Measure recall at the candidate stage, not only final-answer quality.
  • Test documents returned by only one retrieval method.
  • Tune candidate depths and k against a labeled query set.