Why raw-score fusion is fragile

A dense retrieval score and a BM25 score do not necessarily share a common meaning or range. A score from a vector similarity calculation may be influenced by embedding choice, vector normalization, and similarity metric. A BM25 score is driven by term frequency, document frequency, field length, and query terms. Even if both happen to be positive numbers, a value of 0.8 from one system is not inherently comparable to a value of 0.8 from the other.

Min-max normalization can make score ranges look compatible, but it is sensitive to the particular candidate set returned for a query. A single unusually high or low result can change the normalized distribution. Thresholds can also be difficult to maintain when content, analyzers, embeddings, or query patterns evolve.

  • Do not assume dense and BM25 scores have equivalent semantics.
  • Avoid fixed cross-engine score thresholds unless they are continuously validated.
  • Keep each backend responsible for producing a ranked candidate list.
  • Treat fusion as a separate retrieval decision.

Fuse ranked lists with RRF

RRF assigns each document a contribution based on its rank in a result list. For a document at rank r, the contribution is 1 divided by k plus r. The final score is the sum of contributions across lists. The constant k reduces the difference between adjacent positions and keeps the method focused on consistently high-ranking results rather than raw score magnitude.

For example, if a document ranks second in the dense list and tenth in the BM25 list, it receives contributions from both appearances. A document that ranks first in only one list can still perform well, but a document supported by both retrievers often rises naturally. Documents absent from a list simply receive no contribution from that list.

  • Formula: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Use one-based ranks: the first result has rank 1.
  • Deduplicate candidates by a stable document or chunk identifier before returning results.
  • Choose a k value deliberately and evaluate it on representative queries rather than copying a default blindly.

Implement a predictable hybrid retrieval path

A useful request path starts by issuing the dense query to regional S3 Vectors and the sparse query to Quickwit BM25. Request enough candidates from each backend to create room for fusion; returning only a tiny list can hide useful results that would otherwise be promoted by the second retriever. The application layer then converts each response into ordered identifiers, applies RRF, and fetches the metadata needed by the caller.

Keep the diagnostic data. For each fused result, record its dense rank, sparse rank, fused score, and the retrieval configuration used for the request. This makes relevance investigations concrete: an engineer can see whether a result won because of semantic similarity, exact-term matching, or agreement between both retrieval paths.

  • Use the same chunking and stable IDs across dense and sparse indexes where possible.
  • Run both retrieval requests independently, then fuse only successful result sets according to your application’s failure policy.
  • Log per-source rank positions alongside the final ordering.
  • Evaluate queries containing paraphrases, product names, codes, and rare terminology before changing fusion settings.