Treat document identity as a retrieval contract

Dense and sparse indexes answer different questions. Dense search can retrieve semantically related passages even when wording differs, while BM25 is useful when exact terms, identifiers, product names, or uncommon vocabulary matter. For their result sets to be merged safely, each indexed unit needs the same canonical ID in both systems.

Choose an ID at the granularity users should retrieve: often a chunk or passage rather than an entire source file. Derive it deterministically from a source document identifier, a content version, and a chunk position or stable chunk key. Store that ID alongside the vector representation in S3 Vectors and as the indexed document ID in Quickwit.

  • Use one canonical retrieval ID across dense and sparse indexing.
  • Include a source or tenant namespace in the ID to prevent collisions.
  • Version content so that changed chunks do not silently overwrite older meanings.
  • Keep display metadata, such as title and source location, tied to the canonical ID.

Retrieve independently before attempting to rank jointly

At query time, send the same user query through two retrieval paths: embed it for dense search in S3 Vectors, and submit its textual form to Quickwit BM25. Request a candidate list from each path, then normalize each response into a small shared structure containing the canonical ID, rank, and any metadata needed for filtering or presentation.

Do not assume dense similarity scores and BM25 scores share a meaningful numeric scale. Their values may vary with embedding models, query composition, analyzer behavior, and index configuration. Combining raw scores directly can make one retriever dominate for reasons unrelated to relevance.

  • Apply authorization and tenant constraints consistently on both paths.
  • Preserve each candidate's one-based rank in its original result list.
  • Deduplicate repeated IDs within a list before fusion.
  • Fetch enough candidates from each retriever to give the fusion step useful overlap and diversity.

Fuse ranks with a simple, inspectable RRF rule

Reciprocal rank fusion assigns each candidate a contribution based on its position rather than its original score: RRF(d) = Σ 1 / (k + rank_i(d)). Sum that contribution for every retrieval list in which document d appears. The constant k reduces the difference between nearby top ranks and prevents a single first-place result from overwhelming all other evidence.

After sorting by fused score, retain provenance for debugging: whether a result came from dense retrieval, BM25, or both; its rank in each path; and the content version that produced it. This makes relevance investigations concrete. A result that appears only in BM25 may indicate valuable exact-match behavior, while a dense-only result may reveal a terminology mismatch worth understanding.

  • Use ranks, not uncalibrated dense and BM25 scores, as RRF inputs.
  • Choose and document one k value, then evaluate changes with representative queries.
  • Break fused-score ties deterministically, such as by the best individual rank and then canonical ID.
  • Log retrieval provenance so ranking changes can be traced to a specific path or index version.