Why raw dense and BM25 scores should stay separate

A dense search score reflects the similarity function and embedding representation used by the vector search path. BM25, meanwhile, is a lexical ranking function whose score is influenced by term frequency, document length, and corpus statistics. Even when both results are relevant, a score of 0.8 from one path does not inherently mean the same thing as a score of 0.8, 8, or 80 from the other.

Trying to add or compare those raw values directly can make ranking behavior hard to reason about. It also creates a maintenance burden: a change to embeddings, tokenization, document fields, or search configuration can shift score distributions. Rank-based fusion avoids requiring a shared score interpretation in the first place.

  • Use regional S3 Vectors to retrieve dense candidates for semantic matches.
  • Use Quickwit BM25 to retrieve sparse candidates for exact words, codes, names, and uncommon terms.
  • Preserve each source’s original rank and score for debugging, but do not equate the scores.

Fuse candidate lists with reciprocal rank fusion

RRF assigns each document a contribution based on its position in each ranked list. For a document at rank r, its contribution is 1 divided by k plus r. The final RRF score is the sum of that contribution across the lists where the document appears. The constant k reduces the difference between very high ranks and slightly lower ranks, preventing the first few positions in one list from overwhelming every other signal.

For two lists, the application can retrieve a bounded set of dense candidates and a bounded set of BM25 candidates, deduplicate them by document identifier, calculate the RRF score, and sort descending. A document that appears near the top of both lists will naturally rise, while a document found by only one method can still be retained.

  • Formula: RRF(document) = Σ 1 / (k + rank in list).
  • Choose one stable document ID for deduplication across dense and sparse results.
  • Use the same rank convention consistently, such as rank 1 for the first result.
  • Keep the source ranks in response metadata to make merged rankings explainable.

Make fusion observable before tuning it

Start with a small evaluation set made from real user queries and judged relevant documents. Include queries that depend on exact text, such as product IDs or quoted phrases, alongside queries that express an idea in varied language. Inspect not only whether a relevant document appears, but also whether the fused ordering puts it at a useful position.

When results are surprising, inspect the dense list and BM25 list independently before changing fusion parameters. This separates candidate-generation issues from merge issues. If neither list retrieves the needed document, adjusting RRF will not solve the underlying problem; the document content, indexing path, query construction, or retrieval depth may need attention.

  • Log the dense rank, BM25 rank, and final fused rank for returned documents.
  • Track which retrieval path contributed each final result.
  • Test candidate-list depth and the RRF k value against a fixed query set.
  • Review failures by query type rather than relying only on an aggregate metric.