Raw scores are not a shared ranking language

Dense retrieval and BM25 can both return a score, but those scores do not necessarily mean the same thing. A dense-search score depends on the embedding representation and similarity configuration. A BM25 score reflects term-level matches, document statistics, and query terms. Treating the two values as directly comparable can produce unstable weighting.

For example, adding a dense score to a BM25 score assumes that a one-point change has equivalent meaning in both systems. That assumption is usually difficult to justify and can change as documents, embeddings, analyzers, or index settings evolve.

  • Dense search is useful for semantic similarity and vocabulary mismatch.
  • BM25 is useful when exact terms, identifiers, and rare words matter.
  • Score ranges can shift independently as either retrieval path changes.
  • Rank position is often a safer common signal than a raw score.

Fuse candidate lists with Reciprocal Rank Fusion

RRF combines result lists using each document's position rather than its original retrieval score. For every candidate document, add a contribution from each list in which it appears. Documents found near the top of one or both lists rise in the final ranking.

The common formula is RRF(d) = sum of 1 divided by k plus rank(d). The constant k reduces the advantage of a single first-place result and makes fusion less sensitive to small rank differences near the top of a list.

  • Run a dense query against S3 Vectors and a BM25 query against Quickwit.
  • Request a bounded candidate set from each path, such as the top N results.
  • Assign ranks starting at 1 within each result list.
  • Deduplicate by a stable document or chunk identifier, then sort by the summed RRF score.

Make fusion observable and easy to tune

RRF works best when both retrieval paths refer to the same logical corpus and use stable IDs. If search operates on chunks, the dense and BM25 indexes should return the same chunk identifier or a reliable mapping to it. Otherwise, the fusion layer can accidentally treat one piece of content as multiple candidates.

Keep the retrieval traces for evaluation: the source list, original rank, fused score, and final rank for each returned item. This makes it possible to inspect whether dense retrieval, BM25, or their overlap is responsible for a result without relying on incomparable raw scores.

  • Start with the same candidate depth for dense and sparse retrieval.
  • Choose an RRF k value deliberately and evaluate it against representative queries.
  • Record whether each final result came from dense search, BM25, or both.
  • Test identifier-heavy, exact-phrase, semantic, and mixed-intent queries separately.