Why raw score blending is fragile

Dense retrieval and BM25 produce scores with different meanings and distributions. A cosine-like similarity score, a distance-derived score, and a BM25 score should not be added together merely because they are numeric. Even within one retrieval method, score ranges can shift as the corpus, query language, analyzers, or embedding model changes.

For Talqora workloads, dense search is backed by regional S3 Vectors and sparse search by Quickwit BM25. Treat those retrieval paths as independent candidate generators. Their job is to return useful ranked lists; a fusion step can decide which candidates deserve to move forward.

  • Dense search helps when relevant text uses different wording from the query.
  • BM25 helps when exact names, codes, quoted phrases, and rare terms matter.
  • Rank position is often safer to combine than raw retrieval scores.

Fuse the two ranked lists with RRF

Reciprocal Rank Fusion assigns each document a contribution based on its rank in each list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus its rank, across every list where it appears. The constant k reduces the advantage of a document that is only a few positions higher in one list.

Start with one dense list and one BM25 list, deduplicate by a stable document or chunk identifier, calculate the fused score, and sort descending. A document that ranks well in both paths will naturally rise, while a strong result from only one path can still survive.

  • Use one-based ranks: rank 1 is the first result in a list.
  • Choose a fixed k for an initial implementation; 60 is a commonly used starting point, not a universal optimum.
  • Request enough candidates from each path to create a meaningful union before fusion.
  • Apply metadata filters consistently to both retrieval paths when the application requires them.

Make fusion observable and easy to tune

Log the dense rank, BM25 rank, and fused rank for each selected candidate. These fields make retrieval behavior explainable: an engineer can see whether a result won because both methods agreed, because an exact-term match was strong, or because semantic similarity recovered different language.

Evaluate fusion with a representative query set rather than tuning from a few memorable examples. Include identifier-heavy queries, natural-language questions, short ambiguous queries, and queries whose relevant content uses synonyms. If later stages use a reranker or language model, measure both candidate-set recall and final answer quality.

  • Track how often final candidates come from dense only, BM25 only, or both.
  • Inspect failures where relevant documents appear in neither candidate list.
  • Tune candidate depth and k separately; they affect different parts of the retrieval pipeline.
  • Keep chunking, document IDs, and filtering rules aligned across the dense and sparse indexes.