Why raw dense and sparse scores should not be added directly

Dense retrieval ranks records by the relationship between an embedded query and embedded content. BM25 ranks records using lexical evidence such as matching terms and their distribution in the indexed corpus. Both produce useful orderings, but their numeric scores arise from different calculations.

Adding those scores directly can make one retrieval path dominate for accidental reasons: score ranges may differ by query, index, embedding model, or BM25 configuration. Even normalizing scores per request can be fragile when the result set is small or when one method has a sharply different score distribution.

Rank-based fusion avoids treating either score as a universal measure of relevance. Instead, it rewards records that appear near the top of one or both ranked lists.

  • Dense search can recover semantic matches that use different wording.
  • BM25 can preserve precision for exact terms, identifiers, and rare phrases.
  • Ranks are comparable as positions even when underlying scores are not.

Run two retrieval paths and retain enough candidates

For each user query, create the dense query representation and search the appropriate Talqora dense collection backed by S3 Vectors. In parallel, send the original text query to the corresponding Quickwit BM25 index. Apply the same access, tenant, document-status, and other hard filters to both paths.

Request more candidates than the application will ultimately display. Fusion needs room for each retriever to contribute unique documents; if each path returns only the final page size, a useful result from one path can be excluded before fusion begins.

Use a stable document identifier across dense and sparse indexes. The fusion layer needs that identifier to recognize that two hits represent the same record and to return one deduplicated result.

  • Retrieve independently from dense and sparse indexes.
  • Keep filter logic consistent across both searches.
  • Use a shared document ID for deduplication.
  • Choose a candidate depth larger than the final result count.

Apply reciprocal rank fusion in the application layer

For each result list, assign rank 1 to its first item, rank 2 to its second item, and so on. For every document, sum 1 divided by k plus its rank for every list in which it appears. The constant k reduces the difference between adjacent top positions and prevents a single rank-one appearance from overwhelming all other evidence.

A common initial value for k is 60, but it is a tuning parameter rather than a guarantee. Sort documents by the fused score, use a deterministic tie-breaker such as document ID or a source timestamp, and return the top results. Store the source ranks alongside the response during evaluation so relevance changes remain explainable.

RRF is particularly useful as a baseline: it is easy to implement, does not require labeled data, and makes no assumption that a BM25 score can be numerically compared with a vector similarity score. If later evaluation data supports a more specialized reranker, RRF still provides a clear reference point.

  • Fused score: sum of 1 / (k + rank) across result lists.
  • Deduplicate before sorting the final candidate set.
  • Evaluate using representative queries, including exact-term and paraphrased searches.
  • Log dense rank, BM25 rank, and fused rank for debugging.