Why raw dense and sparse scores should not be added

A dense-search score is derived from a vector similarity calculation, while a BM25 score is produced from term-frequency and corpus-statistics signals. Even when each system returns a higher score for a more relevant document, the numerical ranges and distributions are not inherently aligned.

Adding the two scores directly can make ranking sensitive to implementation details rather than relevance. A query with a narrow BM25 score range may be dominated by dense scores, while another query may behave in the opposite direction. The result is difficult to reason about and can shift as indexes or scoring settings evolve.

  • Dense retrieval helps with paraphrases, related concepts, and natural-language intent.
  • BM25 helps with exact names, error codes, product terms, and rare vocabulary.
  • A score of 0.8 in one retrieval system does not automatically have the same meaning as 0.8 in another.

Use reciprocal rank fusion as a stable first fusion strategy

Reciprocal rank fusion, commonly abbreviated as RRF, combines ranked result lists instead of comparing their raw scores. For each document, add a contribution based on its position in every list where it appears: 1 divided by k plus the document rank. The constant k reduces the influence of small rank differences near the top of a list.

In a Talqora-oriented architecture, an application can retrieve a candidate list from regional S3 Vectors for dense search and another from Quickwit BM25 for sparse search. It can then merge results by document identifier and calculate an RRF score before returning the final ordered set.

  • For each result list, rank positions should start at 1.
  • A document returned by both systems receives contributions from both ranks.
  • A document found by only one system can still be included in the fused ranking.
  • Choose k deliberately and keep it consistent while evaluating retrieval quality.

Implement fusion around a shared document identity

Fusion is only reliable when dense and sparse indexes refer to the same logical document identity. Store a stable identifier with each vector record and ensure the BM25 document carries that same identifier. If content is chunked, use a chunk identifier for ranking and retain a parent-document identifier for grouping or display.

Retrieve more candidates from each source than the number of results shown to the user. Fusion needs enough overlap and enough independent candidates to make useful ordering decisions. After scoring, deduplicate by the intended display unit, apply any authorization or metadata filters consistently, and return the top results.

  • Define whether retrieval operates on full documents, passages, or another chunk unit.
  • Apply tenant, access-control, and lifecycle filters before exposing fused results.
  • Log source ranks and the fused rank to make relevance investigations explainable.
  • Evaluate dense-only, BM25-only, and fused result sets against the same query set.