Why raw hybrid scores are difficult to merge

Dense retrieval and BM25 answer different relevance signals. A dense vector search can surface documents with similar meaning even when they use different wording, while BM25 rewards overlap between query terms and document terms. Both signals are useful, but their numeric outputs have different scales and distributions.

A cosine-like similarity value, a distance-derived score, and a BM25 score should not be added together without careful calibration. Their ranges can change with embedding models, query length, corpus composition, indexing settings, and score transformations. A weight that appears reasonable for one query class may distort ranking for another.

This is especially important when a query contains both a precise identifier and broader natural-language intent. Sparse retrieval may strongly favor the identifier, while dense retrieval may find explanatory or semantically related material. The objective is not to force their scores onto one scale; it is to preserve useful evidence from each ranked list.

  • Dense search contributes semantic matches and paraphrases.
  • BM25 contributes exact terms, identifiers, and uncommon vocabulary.
  • Raw score addition assumes score comparability that often does not exist.
  • Rank-based fusion avoids making that assumption.

Fuse dense and sparse candidates with RRF

Reciprocal Rank Fusion assigns each document a contribution based on its position in each result list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus the rank of d across the retrieval lists where it appears. The constant k reduces the difference between nearby ranks and keeps the method focused on consistently well-placed candidates.

In a Talqora retrieval path, issue the dense query against the S3 Vectors-backed index and the lexical query against Quickwit BM25. Keep a bounded candidate list from each side, deduplicate documents by a stable document ID, then calculate an RRF score for the union. Sort by that fused score before returning the final top results.

For example, a document ranked 2nd by dense retrieval and 8th by BM25 receives contributions from both lists. A document appearing only at rank 1 in one list can still rank highly, but a document with strong placement in both lists gains a useful advantage. This makes RRF a good default when neither retrieval signal should dominate globally.

  • Retrieve a fixed top-N candidate set from dense search.
  • Retrieve a fixed top-N candidate set from BM25.
  • Join candidates using a stable canonical document ID.
  • Compute fused scores from ranks, then select the final top-K results.

Operational details that make fusion reliable

Use the same filtering rules for both branches whenever possible. If a request is restricted by tenant, region, document type, or access policy, apply that constraint before candidates are fused. Otherwise, one branch can introduce documents that the other branch would never have been allowed to consider.

Choose candidate depths deliberately. The final page may contain only a small number of results, but each retriever should provide enough candidates for overlap and recovery. If the dense and sparse lists are too shallow, fusion has little opportunity to surface a document that is moderately strong in both lists.

Evaluate the fused ranking with representative query groups rather than relying only on aggregate relevance judgments. Include exact-ID queries, short keyword queries, long questions, terminology-heavy queries, and queries with synonyms. When reviewing failures, inspect each branch's ranks first: fusion can only combine candidates that at least one retriever returned.

  • Apply authorization and metadata filters before fusion.
  • Log document IDs and ranks from each retrieval branch.
  • Test candidate depth separately from final result count.
  • Treat RRF as a transparent baseline before adding learned reranking.