Why raw-score merging is unreliable

A dense search result may be ordered by vector similarity, while a sparse result is ordered by BM25 relevance. Both rankings can be useful, but their numeric scores are produced by different methods. A score of 0.82 from a dense search is not inherently comparable to a BM25 score of 8.4.

Sorting one combined list by raw score can therefore over-favor whichever retriever happens to emit larger numbers. Score normalization can help in some systems, but it introduces decisions about distributions, query behavior, and edge cases. Rank-based fusion avoids treating unlike scores as though they were measurements in the same unit.

  • Dense retrieval can capture semantic similarity and paraphrases.
  • BM25 can reward exact terms, identifiers, and rare vocabulary.
  • Raw scores should be treated as retriever-specific unless calibrated for comparison.

Fuse candidate lists with Reciprocal Rank Fusion

RRF assigns each document a contribution based on its position in every result list where it appears. For a document d, the fused score is the sum of 1 divided by k plus its rank: RRF(d) = Σ 1 / (k + rank_i(d)). The constant k reduces the gap between nearby ranks and keeps the fusion focused on consistently high placements.

In practice, request a bounded candidate list from regional S3 Vectors and another from Quickwit BM25. Use a stable document identifier to deduplicate the lists, calculate an RRF score for every seen document, then sort descending by that fused score. A document found by both retrievers receives two contributions; a strong result from only one retriever can still remain competitive.

  • Use one canonical document ID across dense and sparse indexes.
  • Keep ranks one-based when implementing the formula.
  • Choose a candidate depth that leaves room for complementary results.
  • Apply a deterministic tie-breaker, such as document ID, for stable responses.

Make fusion observable before tuning it

Start with a small evaluation set built from real query patterns: natural-language questions, product names, error strings, abbreviations, and identifier-heavy searches. Inspect not only whether the final top result is relevant, but also which retriever contributed it and whether a relevant result was recovered by one path but lost during fusion.

Treat k, candidate depth, and any post-fusion filters as explicit configuration. Logging the dense rank, sparse rank, and final fused rank for returned documents makes retrieval behavior debuggable. This is especially useful when an exact-match query should lean on BM25 while an ambiguous or paraphrased query benefits from dense retrieval.

  • Record each document's source ranks alongside its fused score.
  • Test exact terms and semantic paraphrases separately.
  • Check filters and access constraints before returning fused results.
  • Re-evaluate after changes to chunking, embeddings, or indexed fields.