Why raw-score fusion is fragile

A dense retriever and a BM25 retriever produce scores with different meanings. A dense score may reflect vector similarity, while a BM25 score is driven by term frequency, document frequency, and field-length normalization. Adding those values directly can make one retriever dominate simply because its score range is numerically larger.

That issue becomes harder to manage as documents, embedding models, tokenization rules, or sparse-search configuration change. A weight that looked reasonable for one collection may behave differently for another. Rank-based fusion avoids treating these scores as interchangeable measurements.

  • Dense retrieval helps when query and document wording differs.
  • BM25 helps preserve exact-match behavior for terms and identifiers.
  • Raw score ranges are not inherently comparable across retrieval methods.
  • Rank positions are easier to combine than unrelated score scales.

Fuse result lists with RRF

With RRF, run dense and sparse searches independently, collect a candidate list from each, and assign every document a fused score based on its rank in each list. A common form is: RRF(d) = sum of 1 / (k + rank_i(d)), where rank_i(d) is the document's one-based position in result list i and k is a positive constant.

Documents returned by both retrieval methods receive contributions from both lists. Documents found by only one method can still appear in the final ranking. The constant k reduces the difference between adjacent ranks near the top of a list, preventing a single first-place result from overwhelming all other evidence.

  • Request the same candidate depth from dense and sparse retrieval, such as the top N from each.
  • Deduplicate candidates by a stable document or chunk identifier.
  • Use one-based ranks consistently when calculating the fusion score.
  • Sort candidates by fused score, then apply a deterministic tie-breaker if needed.

Apply the pattern in a Talqora retrieval flow

In Talqora, a client can treat regional S3 Vectors dense search and Quickwit BM25 sparse search as two candidate generators. Send the embedded query to the dense path and the text query to the sparse path, then perform RRF in the application layer over the returned identifiers. Fetch or retain the metadata needed to present the final chunks to the user or to supply context to a downstream system.

Evaluate the fused results using queries that reflect production behavior. Include semantic questions, exact identifiers, partial product names, acronyms, and misspellings where relevant. Review not only whether a useful result appears, but whether it appears high enough in the final list to be selected by the next stage of the application.

  • Keep the dense and sparse indexes aligned on document IDs and chunk boundaries.
  • Log each candidate's dense rank, sparse rank, and final fused rank for debugging.
  • Start with a fixed RRF constant and candidate depth before introducing additional tuning.
  • Use judged queries to compare dense-only, sparse-only, and fused rankings.