Why score values should not be merged naively

Dense retrieval and BM25 answer different matching questions. Dense search can surface semantically related content even when wording differs, while BM25 rewards terms that occur in the query and document. Both signals are useful, but their raw scores do not necessarily share a common scale or interpretation.

Adding or averaging raw scores can make ranking sensitive to implementation details such as score normalization, query length, corpus composition, or the number of terms. Instead, treat each retrieval path as a ranked recommendation list and combine the order of its results.

  • Dense search is useful for semantic and paraphrase-style matches.
  • BM25 is useful for exact terms, identifiers, names, and uncommon vocabulary.
  • Rank positions are easier to combine than unrelated score distributions.

Fuse dense and sparse result lists with RRF

Run a dense query against the S3 Vectors-backed retrieval path and a sparse query against the Quickwit BM25-backed path. Request a sufficiently deep candidate list from each path, then deduplicate documents using a stable document identifier.

For every document appearing in either list, compute an RRF score by summing 1 divided by k plus the document's rank for each list where it appears. The constant k reduces the influence of small rank differences near the top of a list. Sort documents by the resulting total and return the leading results.

  • Use one stable ID across dense and sparse indexes.
  • Keep ranks one-based when implementing the formula.
  • Choose a fixed k and evaluate it against representative queries.
  • Fetch more candidates than the final number of results so fusion has useful overlap and alternatives.

Evaluate with query classes, not a single aggregate

Hybrid retrieval is most useful when evaluated across the kinds of queries users actually submit. Build a small judgment set that includes exact identifiers, product or domain terminology, natural-language questions, and queries whose wording differs from relevant documents.

Inspect both the final fused ranking and the source list that contributed each result. This makes failures diagnosable: an exact-match miss may point to sparse-query handling, while a paraphrase miss may point to embedding choice, document chunking, or candidate depth. RRF is a ranking layer, so it does not replace sound indexing and document preparation.

  • Track whether relevant documents appear in dense, sparse, or both candidate lists.
  • Review queries with no overlap between the two retrieval paths.
  • Test chunk boundaries and metadata filters alongside ranking changes.
  • Preserve retrieval provenance for debugging and relevance review.