Why raw-score hybrid search is fragile

A common hybrid-search design runs a dense search and a sparse BM25 search, then adds or weights their scores. This can work only after careful calibration. BM25 scores vary with corpus composition, document length, analyzers, and query terms. Dense similarity values depend on the embedding model, distance metric, and index configuration.

A fixed rule such as “70% dense plus 30% sparse” can therefore change behavior as documents are added or either retrieval pipeline changes. It may overvalue one system simply because its numerical range is larger, rather than because it found more useful documents.

  • Do not assume a score of 0.8 from dense retrieval has a meaning comparable to a BM25 score of 8.
  • Treat score distributions as implementation details unless they have been explicitly calibrated.
  • Keep each retrieval path independently observable before introducing a fusion layer.

Fuse ranked lists with RRF

RRF combines documents based on where they appear in each ranked list. For every document, add 1 divided by k plus its rank for each list in which it appears. The document with the highest combined total ranks first. Here, rank starts at 1 and k is a positive smoothing constant chosen by the application.

In a Talqora-based architecture, an application can retrieve a dense candidate list from regional S3 Vectors and a sparse candidate list from Quickwit BM25, then perform this small merge step in its own service. The method uses ordering rather than raw scores, so the two engines do not need a shared scoring scale.

  • Formula: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Use the same document identifier in both indexes so results can be deduplicated during fusion.
  • Start with equal contribution from dense and sparse lists; add weighting only when evaluation supports it.
  • Request enough candidates from each retriever to give fusion meaningful overlap and coverage.

Make fusion debuggable before making it sophisticated

Log the dense rank, BM25 rank, and final fused rank for each returned document. These fields make it possible to see whether a result was supported by both retrievers or rescued by only one. They also reveal failures such as mismatched document IDs, stale indexing, or a sparse query that returns too few candidates.

Evaluate RRF with a small, representative query set that includes exact-match queries and semantic paraphrases. Review the top results, not just a single aggregate metric. For many teams, the first useful improvement is better document chunking and metadata consistency, rather than a more complicated fusion formula.

  • Include product names, codes, acronyms, and quoted phrases in the evaluation set.
  • Track candidate counts from each retrieval path and the amount of overlap between them.
  • Use filters consistently across dense and sparse retrieval so fusion does not mix incompatible result sets.
  • Version the embedding model, chunking strategy, and sparse analyzer configuration alongside evaluation results.