Why raw-score blending is fragile

A dense retriever and a BM25 retriever produce scores with different meanings. Dense similarity reflects proximity in an embedding space, while BM25 reflects term frequency, document length, and corpus-level term statistics. Even if both systems return a number called a score, the ranges and distributions can vary by query.

A fixed formula such as dense_score + bm25_score can therefore produce unstable behavior. A rare exact identifier may deserve strong lexical weight, while a conceptual question may benefit more from semantic neighbors. Score normalization can help, but it introduces another set of assumptions that must be monitored as the corpus and query mix change.

  • Keep the dense and sparse queries semantically aligned: use the same user intent and compatible filters.
  • Request a candidate depth larger than the final page size from each retriever.
  • Apply access-control, tenant, and document-status filters before fusion whenever possible.

Fuse ranks with reciprocal rank fusion

RRF assigns each document a contribution based on its position in each ranked list. For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is its one-based position in result list i and k is a positive smoothing constant. Documents returned by both retrievers accumulate contributions; a document can also rank well by appearing near the top of one list.

The important property is that RRF consumes ranks, not incompatible score scales. Your application can query regional S3 Vectors for dense candidates and Quickwit BM25 for sparse candidates in parallel, deduplicate by a stable document or chunk identifier, compute fused ranks, and return the top results.

  • Start with a conventional smoothing constant such as k = 60, then validate it against representative queries.
  • Use one-based ranks and define deterministic tie-breaking, such as a stable document ID.
  • Fuse at the chunk level when answers depend on passages; fuse at the document level when the result is a document listing.
  • Preserve source ranks in logs so relevance investigations can explain a fused result.

Build a small evaluation loop before tuning

Create a query set that includes exact names, error codes, abbreviations, natural-language questions, and multi-topic requests. For each query, record whether the desired item appears in the top results for dense retrieval, BM25 retrieval, and the fused list. This reveals where the systems complement each other rather than relying on aggregate impressions.

Candidate depth is often the first parameter to inspect. If a relevant item never reaches either candidate list, fusion cannot recover it. If it appears in one list but is pushed down after fusion, inspect its rank contribution, duplicate handling, filters, and whether the query should be rewritten or supplemented with structured constraints.

  • Evaluate dense-only, BM25-only, and RRF side by side on the same frozen query set.
  • Record query text, applied filters, source ranks, fused rank, and relevance judgment.
  • Review failure clusters separately: identifiers, freshness-sensitive content, synonyms, and long questions.
  • Version embeddings, indexing settings, and fusion parameters so changes remain reproducible.