Why raw dense and sparse scores should not be added

A dense retrieval score reflects the relationship between a query embedding and document embedding. A BM25 score reflects term frequency, document frequency, document length, and exact query-term matches. Even when both are useful signals, the values do not share a common meaning.

Adding the two scores directly can make ranking sensitive to query wording, corpus composition, embedding model changes, or BM25 configuration. A weighting scheme that appears reasonable for one query may behave differently for another because score distributions can shift independently.

  • Dense retrieval helps with paraphrases and conceptually related language.
  • BM25 helps when exact terminology, error codes, names, or product identifiers matter.
  • Raw score ranges may change without indicating a change in relevance.
  • Rank positions are usually easier to compare than raw scores.

Fuse ranked lists with reciprocal rank fusion

Run dense search against regional S3 Vectors and sparse search through Quickwit BM25, then retain an ordered list of document identifiers from each retriever. RRF assigns each document a contribution based on its position in a list, rather than its source score.

For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) across the lists where d appears. The constant k dampens the effect of very high positions; a commonly used starting value is 60, but it should be treated as a tunable policy choice rather than a universal optimum.

  • Retrieve a sufficiently broad candidate list from both dense and sparse search.
  • Use stable document IDs to deduplicate documents appearing in both lists.
  • Assign ranks starting at 1, not 0.
  • Sort documents by descending fused score and return the desired top N.

Make fusion observable and easy to revise

Store or log the dense rank, sparse rank, and final fused rank for sampled queries. These fields make it possible to see whether useful results are being carried by one retrieval mode, supported by both, or lost because candidate lists are too shallow.

Evaluate with a query set that reflects real search behavior. Include semantic questions, exact-name lookups, partial identifiers, abbreviations, and cases where vocabulary differs between the query and the relevant document. Review failures by query type before changing fusion parameters.

  • Inspect documents returned by only one retriever as well as documents returned by both.
  • Check whether relevant documents appear in either candidate list before blaming fusion.
  • Version the embedding model, BM25 configuration, and fusion policy together.
  • Use reranking only after establishing that first-stage candidates contain the needed documents.