Why merge ranks instead of raw scores?

Dense retrieval and BM25 produce scores with different meanings and distributions. A dense similarity score reflects the relationship between embeddings, while BM25 is driven by term frequency, document frequency, and document length. Adding those values directly can make one retrieval method dominate for reasons unrelated to relevance.

RRF avoids score calibration by using each result's position in its own ranked list. A document that appears near the top of either list receives a useful contribution; a document that appears in both lists receives two contributions. This makes RRF a good baseline when the two engines are independently operated or their score ranges are not directly comparable.

  • Use dense retrieval for semantic similarity and paraphrases.
  • Use BM25 for exact terminology, codes, names, and rare tokens.
  • Treat each engine's output as an ordered list rather than a shared score scale.

Build two candidate lists with stable document IDs

For every indexed chunk, keep a stable identifier that is shared by the dense and sparse representations. The identifier can represent a chunk, passage, or another retrieval unit, but it should resolve to the same metadata and content regardless of which retriever found it.

At query time, create an embedding for the query and request a ranked dense candidate list from the S3 Vectors-backed path. In parallel, submit the query text to the Quickwit BM25-backed path. Request a bounded number of candidates from each list, then deduplicate by the shared identifier before presenting results.

  • Index the same retrieval unit in both dense and sparse systems.
  • Store metadata needed for filtering and result rendering with the retrieval unit.
  • Apply equivalent tenant, access-control, and content-status filters to both searches.
  • Keep the original query text for BM25 and use its embedding for dense retrieval.

Fuse candidates with a small, inspectable function

For a document d, RRF computes a combined score by summing 1 divided by k plus the rank from each list: RRF(d) = Σ 1 / (k + rank_i(d)). The constant k softens the difference between adjacent ranks and limits how much a single first-place result overwhelms the rest of the candidate set.

Start with the same candidate depth and equal contribution from dense and sparse lists. Then inspect queries where the result sets disagree: exact identifier queries, broad conceptual questions, short queries, and domain-specific wording. If needed, adjust candidate depths or add a weight per list, but retain logs of ranks and source lists so changes remain explainable.

  • Assign ranks starting at 1 and omit a contribution when a document is absent from a list.
  • Deduplicate before final ranking, while preserving each source rank for debugging.
  • Fetch final document fields only for the fused top results when possible.
  • Evaluate relevance separately for exact-match and semantic-query workloads.