Why one retrieval signal is rarely enough

Dense retrieval represents the meaning of a query and document as vectors. It is useful when a person asks for a concept using different wording than the source material. A query such as “how do I rotate an access key?” can still surface material titled “credential renewal procedure.”

BM25 is a lexical ranking method that rewards overlap between query terms and indexed terms. It remains valuable for product names, error codes, API fields, quoted phrases, acronyms, and uncommon identifiers. If a user searches for an exact error string, lexical retrieval often supplies an important precision signal.

  • Use dense search to widen semantic recall.
  • Use BM25 to retain exact and terminology-sensitive matches.
  • Treat the two result lists as complementary evidence, not interchangeable rankings.

Retrieve independently before combining results

Issue the same user query to both retrieval paths, while applying the same access controls and content filters to each. Request a limited number of candidates from regional S3 Vectors for dense retrieval and from Quickwit BM25 for sparse retrieval. The candidate count should be large enough to allow overlap and recovery from a weak signal, but bounded enough to keep downstream work predictable.

Do not assume a dense similarity score and a BM25 score share a common scale. Their raw values are produced by different ranking systems and should not be added directly without a validated normalization approach. A safer first implementation uses ranks rather than raw scores.

  • Apply tenant, permission, document-state, and language filters before fusion.
  • Keep document IDs stable across dense and sparse indexes.
  • Fetch more than the final display count from each path.
  • Log which retrieval path contributed each candidate.

Use reciprocal rank fusion as a practical default

Reciprocal rank fusion (RRF) combines result lists using position instead of score magnitude. For each document, add 1 divided by k plus its rank for every list in which it appears. The constant k reduces the impact of small rank differences near the top of a list. Documents found by both dense and sparse retrieval naturally receive a stronger combined score.

After fusion, deduplicate by canonical document ID, sort by the fused score, and return the top results. Evaluate this policy using representative queries that include natural-language questions, exact identifiers, short ambiguous searches, and domain-specific vocabulary. Review failures by query type before changing candidate counts or adding more complex reranking.

  • RRF avoids directly comparing incompatible score scales.
  • Set a deterministic tie-breaker, such as a stable document ID or recency field.
  • Record ranks, source lists, and final position for debugging.
  • Tune the fusion constant and candidate-pool sizes against relevance judgments.