Why use two retrieval signals?

Dense retrieval represents the meaning of a query and document as vectors. It is useful when the query and the relevant content use different vocabulary, such as a search for “reset account access” finding documentation titled “recovering your login credentials.”

BM25 uses term statistics and document text. It can be especially valuable when a query contains literal strings that should remain prominent: error codes, product names, API parameters, ticket IDs, filenames, or rare technical terms. Treating these signals as complementary is usually more reliable than assuming one ranking method is universally best.

  • Use dense retrieval to broaden semantic recall.
  • Use BM25 to preserve exact-term sensitivity.
  • Keep the same document identifier available in both indexes.
  • Evaluate on representative queries, including identifier-heavy queries.

Retrieve separately, then apply Reciprocal Rank Fusion

RRF merges ranked lists without requiring dense similarity scores and BM25 scores to share a scale. This matters because the numerical score from one retrieval system is not automatically comparable to the score from another. Instead, RRF gives each document credit based on its position in each result list.

For a document d, calculate RRF(d) as the sum of 1 divided by k plus rank(d) across every list where it appears. The constant k reduces the influence of small rank differences near the top of a list. A commonly used starting point is 60, but it should be treated as a tunable application choice rather than a universal setting.

  • Run a dense query against the S3 Vectors-backed path.
  • Run a text query against the Quickwit BM25-backed path.
  • Request a candidate depth larger than the final number of results.
  • Deduplicate candidates by stable document ID before returning the fused ranking.

Make fusion operationally useful

Start with a small evaluation set before changing ranking logic in production. Include natural-language questions, terse keyword searches, known-item lookups, and queries containing codes or quoted phrases. For each query, record whether the expected document appears in the candidate set and whether it appears near the top after fusion.

Instrumentation should show more than the final result. Record which retrieval paths returned each document, its rank in each path, its fused score, and the query type when available. This makes it easier to diagnose whether a poor result came from missing indexing content, weak semantic recall, lexical mismatch, or an overly aggressive merge policy.

  • Use stable IDs and consistent deletion/update handling across both indexes.
  • Log dense rank, BM25 rank, and final fused rank for returned documents.
  • Test candidate depth and the RRF k value against a labeled query set.
  • Apply metadata filters consistently before fusion when the application requires them.