Why raw dense and BM25 scores should not be added directly

A dense-search score and a BM25 score are not automatically comparable. Their ranges, distributions, and meanings depend on the embedding model, similarity metric, document collection, tokenization, and query. A BM25 value of 12 does not inherently mean a result is twice as relevant as a vector result with a similarity score of 0.6.

Adding raw scores can make one retrieval path dominate for accidental numerical reasons. A change in embedding model, BM25 configuration, or index contents may alter score distributions without representing a real change in relevance. Rank-based fusion avoids treating these internal scoring scales as a common measurement.

  • Dense retrieval helps with paraphrases, conceptual similarity, and vocabulary mismatch.
  • BM25 helps with exact wording, codes, names, quoted phrases, and uncommon terms.
  • Score scales may shift as models, analyzers, or collections change.
  • Use ranked positions when no validated cross-system score calibration exists.

Fuse two result lists with Reciprocal Rank Fusion

Run the same user query through dense retrieval in S3 Vectors and sparse retrieval with Quickwit BM25. Keep a stable document or chunk identifier in both indexes so that the same item can be recognized during merging. Then assign each candidate a fusion contribution based on its position in each list.

For a document d, a common RRF formula is: RRF(d) = sum over lists of 1 / (k + rank(d)). The rank is one-based, and k is a positive constant that reduces the difference between adjacent top positions. If a document appears in both lists, it receives two contributions; if it appears in only one, it still remains eligible for the final result set.

  • Fetch a modest candidate set from each retriever before fusion, such as the top N results.
  • Deduplicate candidates by a canonical chunk or document ID.
  • Use the same query text initially; query rewriting can be evaluated later as a separate change.
  • Sort by fused score and pass the highest-ranked candidates to the next stage, such as a reranker or answer generator.

Make fusion observable before tuning it

RRF has few moving parts, but it still needs evaluation against real queries. Build a small judgment set that includes exact-lookup queries, natural-language questions, acronym-heavy requests, and queries with ambiguous terminology. Record whether the intended document appears in the final top results and which retrieval path contributed it.

Start with equal weight for the dense and BM25 lists. If later evidence shows that one list deserves more influence for a particular query class, introduce weights deliberately rather than changing several variables at once. Logging ranks, source lists, and fused outcomes makes those decisions explainable.

  • Track dense rank, BM25 rank, fused rank, and canonical document ID for every returned candidate.
  • Inspect queries where a relevant result was retrieved by only one method.
  • Test candidate depth separately from the RRF constant k.
  • Evaluate retrieval quality before optimizing downstream generation behavior.