Why raw dense and sparse scores should not be added

Dense retrieval scores are produced from the relationship between embeddings. BM25 scores are produced from term statistics and document-level matching behavior. Even when both searches return numbers that look similar, their scales, distributions, and sensitivity to a query can differ.

Adding those numbers directly creates a hidden weighting decision. A BM25 score that is numerically larger is not necessarily stronger evidence than a dense score that is numerically smaller. The result can be a ranking dominated by whichever system happens to emit larger values for a particular query class.

This is especially noticeable when a query contains an exact identifier, error code, product name, or quoted phrase. Sparse search may surface exact matches effectively, while dense search can retrieve semantically related documents. Both signals matter, but their score scales should not be treated as interchangeable.

  • Dense scores reflect embedding-space similarity, not term frequency.
  • BM25 scores reflect lexical matching and corpus statistics.
  • Score ranges can vary across queries, indexes, and retrieval configurations.
  • A fixed raw-score weight can behave unpredictably over time.

Use reciprocal rank fusion as a stable first baseline

Reciprocal rank fusion, often shortened to RRF, combines ranked lists instead of attempting to normalize their underlying scores. For each document, assign a contribution from every list where it appears: 1 divided by k plus its rank. Sum those contributions and sort documents by the resulting total.

In a Talqora-backed application, a practical flow is to issue a dense query against regional S3 Vectors and a sparse query against Quickwit BM25. Request a candidate list from each system, identify documents by a shared stable ID, then fuse the two lists in application code.

The constant k reduces the difference between adjacent top ranks and lower ranks. It also prevents the first result in one list from overwhelming every other result solely because of its position. Treat k and each candidate-list depth as explicit configuration values, then evaluate them with representative queries.

  • Dense list: retrieve the top N semantic candidates.
  • Sparse list: retrieve the top N lexical candidates.
  • Join candidates using a stable document or chunk identifier.
  • Compute RRF score: sum of 1 / (k + rank) for every list containing the item.

Design the evaluation set around retrieval disagreements

The most informative test queries are not the easy ones where dense and sparse search return the same documents. Build a small evaluation set that includes exact terms, abbreviations, natural-language questions, ambiguous phrases, and queries that mix a named entity with an intent.

For each query, inspect the dense list, the BM25 list, and the fused list. Record whether relevant material appears, where it ranks, and whether fusion introduces duplicate chunks or near-duplicate content. This review reveals whether candidate depth, chunking, or document identifiers need adjustment before more complicated ranking logic is added.

Rank fusion is a retrieval-stage technique, not a substitute for clean content preparation. Stable IDs, consistent chunk boundaries, and a clear policy for updating both dense and sparse representations make the final merged ranking easier to reason about.

  • Include identifier-heavy queries such as ticket IDs, API fields, and error strings.
  • Include conceptual questions that may not share wording with the answer.
  • Review duplicate and near-duplicate results after fusion.
  • Track changes to k, candidate depth, and chunking as separate experiments.