Why raw-score merging is risky

A dense retriever and a BM25 retriever usually produce scores with different meanings and ranges. A vector similarity score is shaped by the embedding model and similarity metric. A BM25 score is shaped by term frequency, document length, and corpus-level term statistics.

Sorting one combined list by those raw values assumes the scores are comparable. That assumption can make one retrieval method dominate simply because its scoring scale is numerically larger, rather than because its results are more useful for the query.

  • Dense retrieval helps with paraphrases and related concepts.
  • BM25 helps with exact names, codes, error messages, and uncommon terms.
  • Score ranges can vary across indexes, queries, and retrieval methods.

Fuse ranks instead of scores

Reciprocal rank fusion (RRF) combines independently ranked lists without requiring score normalization. Retrieve a candidate list from regional S3 Vectors and another from Quickwit BM25, then assign each document a contribution based on its position in each list.

For each document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the document's one-based rank in list i. The constant k reduces the effect of small rank differences near the top of a list. A commonly used starting value is 60, but it should be treated as a configurable application choice.

  • Deduplicate documents by a stable document or chunk identifier.
  • Assign no contribution when a document is absent from a list.
  • Sort by the summed RRF score after processing both lists.
  • Keep the source ranks for debugging and relevance analysis.

Implement RRF as a retrieval orchestration step

At query time, issue the same user query to both retrieval paths. The dense path uses the query embedding against S3 Vectors, while the sparse path sends the text query to Quickwit BM25. Ask each system for more candidates than the final number shown to the user, because fusion needs room to reward results that appear in either list.

After fusion, return the top documents or chunks to the next stage of the application, such as a result page or an answer-generation context builder. Log the query, final identifiers, source ranks, and fusion score so relevance reviews can reveal whether a miss came from dense retrieval, sparse retrieval, or the fusion policy.

  • Start with equal candidate counts from both retrieval paths.
  • Use a shared identifier across dense and sparse representations.
  • Test exact-term, semantic, and mixed-intent queries separately.
  • Adjust candidate depth and k only after reviewing representative queries.