Why raw-score mixing is usually brittle

Dense and sparse systems commonly produce scores with different meanings. A BM25 score reflects term statistics and document length effects, while a dense-search similarity score reflects the relationship between query and embedding vectors. Adding those values directly assumes that a score of one unit means the same thing in both systems, which is rarely a safe assumption.

Score normalization can help in controlled settings, but it introduces operational questions: which sample defines the distribution, how often should it be refreshed, and what happens when the document corpus or embedding model changes? For an initial hybrid retrieval implementation, combining ranks is often easier to reason about than attempting to calibrate incomparable scores.

  • Use dense retrieval for semantic alternatives and paraphrases.
  • Use BM25 for exact wording, product codes, names, and rare terms.
  • Treat each retrieval path as a candidate generator rather than a final answer.

Fuse two candidate lists with RRF

Run the same query against Talqora's dense and sparse retrieval paths, requesting a sufficiently deep candidate list from each. Then assign every returned document a fusion contribution based on its rank: 1 divided by k plus its rank. Add contributions when a document appears in both lists.

The constant k reduces the gap between adjacent top ranks. A commonly used starting point is 60, though it is a tuning parameter rather than a universal rule. With one-based ranks, the fused score is: RRF(d) = Σ 1 / (k + rank_i(d)). Sort documents by that total and return the top results.

  • Deduplicate by a stable document or chunk identifier before final sorting.
  • Use one-based rank positions: the first result has rank 1.
  • Include a document's contribution from every list where it appears.
  • Keep the original dense and BM25 ranks for debugging.

Make fusion observable and tune it with real queries

Log more than the final order. For each result, record whether it came from dense retrieval, BM25 retrieval, or both, along with its per-source ranks. This makes it possible to distinguish a genuinely useful hybrid result from one path overwhelming the other because of shallow candidate depth or a query-processing issue.

Evaluate with a small, representative query set before adding more complexity. Include exact identifier queries, acronym-heavy queries, natural-language questions, and queries whose wording differs from likely document wording. Review whether relevant documents enter either candidate list before judging the fusion rule; reranking cannot recover documents that were never retrieved.

  • Start by retrieving the same candidate depth from both paths.
  • Test several k values while holding the query set and candidate depth fixed.
  • Inspect failures by source: missing from dense, missing from BM25, or poorly fused.
  • Consider a later reranking stage only after candidate recall is understood.