Why raw score addition is fragile

A dense-search similarity score and a BM25 relevance score are produced by different retrieval models with different scales and distributions. Even if both systems rank useful documents highly, adding their scores directly can give disproportionate influence to whichever score has the larger numeric range.

Score behavior can also vary across queries. A short identifier lookup may produce a sharply peaked BM25 ranking, while a broad natural-language question may create a flatter dense ranking. A fixed weighting scheme can therefore behave differently from one query type to another.

  • Dense similarity and BM25 scores are not inherently comparable.
  • Score ranges may differ across indexes and queries.
  • Manual score weights require ongoing evaluation and tuning.

Retrieve separate candidate lists

Start by sending the same user query through two retrieval paths. Generate an embedding for the dense path and retrieve a bounded candidate list from S3 Vectors. Send the text query to Quickwit BM25 and retrieve a separate bounded candidate list from the sparse index.

Both paths must refer to the same logical document universe. Store a stable document or chunk identifier in each index, and retain metadata needed for filtering, display, and later reranking. The retrieval lists do not need to contain identical items; their differences are often the reason hybrid retrieval helps.

  • Use one stable chunk or document ID across dense and sparse indexes.
  • Apply compatible tenant, access-control, and content-state filters to both paths.
  • Choose a candidate depth large enough to preserve useful overlap and complementary results.
  • Log each source rank separately for debugging and relevance analysis.

Fuse by reciprocal rank, then inspect the result

RRF assigns each document a contribution based on its position in each ranked list. For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i is the one-based rank from retrieval source i and k is a positive constant. Documents returned by both S3 Vectors and Quickwit BM25 receive contributions from both lists.

Because RRF only uses ordering, it avoids treating a BM25 score as if it were a dense similarity score. It also rewards agreement between retrieval methods while allowing a strong result from either method to remain competitive. After fusion, the application can return the fused list directly or pass a small top set to a separate reranking step.

  • Deduplicate candidates by stable ID before returning results.
  • Use one-based ranks when calculating the fusion contribution.
  • Keep the dense rank, sparse rank, and fused score in retrieval logs.
  • Evaluate representative semantic, exact-match, and mixed-intent queries before changing candidate depth or fusion parameters.