Why raw hybrid scores are difficult to merge

A dense retriever typically returns a similarity-based score derived from an embedding comparison. A BM25 retriever returns a lexical relevance score based on term frequency, document statistics, and query terms. Even when both systems rank useful documents highly, their numeric outputs do not automatically mean the same thing.

Adding raw scores can make one retriever dominate for reasons unrelated to relevance. A score range can vary with the embedding model, similarity metric, corpus composition, analyzer settings, or query length. Manual weighting may work for a narrow evaluation set, but it often becomes another parameter that needs ongoing calibration.

  • Dense retrieval helps when intent is expressed with different wording.
  • BM25 helps when literal terms must be preserved.
  • A score of 2 in one ranking system is not inherently comparable to a score of 2 in another.
  • Rank position is often more portable than an uncalibrated score.

Fuse rank positions with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in each result list. For a document d, sum 1 divided by k plus its rank for every list in which it appears: RRF(d) = Σ 1 / (k + rank_i(d)). The constant k reduces the difference between nearby rank positions and keeps a single first-place result from overwhelming the combined ranking.

Run dense search against Talqora Vector's regional S3 Vectors-backed dense retrieval path and run a BM25 query through its Quickwit-backed sparse retrieval path. Request a candidate list from each, deduplicate by a stable document or chunk identifier, calculate the RRF score in the application, and sort descending. The fused list can then feed your answer-generation, recommendation, or result-display layer.

  • Use one-based ranks: the first result has rank 1.
  • Choose the same candidate depth for both lists as a straightforward starting point.
  • Deduplicate before returning results, but retain contributions from every list where a document appeared.
  • Keep the original dense and BM25 ranks in logs for debugging.

Make fusion observable before making it complex

RRF is valuable because it gives both retrieval signals a chance to contribute without requiring score normalization. It is not a substitute for evaluation. Build a small set of representative queries that includes natural-language questions, exact identifiers, abbreviations, product names, and queries with mixed semantic and lexical intent.

For each query, inspect the dense list, the BM25 list, and the fused list. Look for relevant results rescued by the other retriever, duplicate chunks from the same source, and cases where one retriever consistently supplies poor candidates. If needed, improve chunking, metadata filters, lexical analysis, or query construction before introducing more complicated ranking logic.

  • Record which retrieval path contributed each final result.
  • Evaluate result quality at the rank depth your application actually uses.
  • Apply metadata and tenant filters consistently to both retrieval paths.
  • Consider reranking only after fusion if your workflow needs a more selective final ordering.