Define the retrieval unit before defining the index

A document is rarely the best unit for retrieval. Long documents are commonly split into passages, sections, or other chunks so that both lexical and semantic search can return a focused piece of evidence. The key is to give every chunk a deterministic identity rather than letting each index create its own opaque identifier.

A practical ID can be derived from a canonical document identifier, a content version, and a chunk position or chunk hash. The exact format matters less than the contract: the same source chunk must produce the same ID for the dense representation and the BM25 record. Keep the document-level ID separately so results can later be grouped, diversified, or traced back to a source.

  • Use a canonical document ID that does not depend on the index being written.
  • Include a version marker when source content can change.
  • Store a chunk ordinal or byte-range reference for reconstruction and debugging.
  • Treat the chunk ID as an application-level identifier, not an index-specific implementation detail.

Write dense and sparse representations from one chunk record

Build one normalized chunk record in the ingestion pipeline, then derive both representations from it. The dense path sends the embedding associated with that chunk to regional S3 Vectors. The sparse path sends searchable text and the same chunk ID to Quickwit BM25. Metadata needed at retrieval time should be defined once and applied consistently to both paths where filtering or result interpretation requires it.

This avoids a subtle source of hybrid-retrieval errors: different chunk boundaries. If BM25 indexes whole sections while dense search indexes smaller passages, a fusion layer may combine results that overlap only partially. That can inflate one document's apparent relevance or make citations difficult to explain.

  • Create text normalization, chunking, and metadata extraction as shared ingestion steps.
  • Persist the source text or a durable source pointer alongside the chunk identity.
  • Record embedding-model and chunking-policy versions in ingestion metadata.
  • Reindex both retrieval paths when a chunking policy changes, rather than mixing incompatible units.

Fuse rankings only after identity and deduplication are clear

Once both result sets use the same chunk IDs, a service can merge them predictably. Reciprocal rank fusion is a straightforward option when scores from dense and BM25 retrieval are not directly comparable: each result receives credit based on its rank in each list, and matching IDs accumulate credit. Other fusion approaches are possible, but all benefit from stable identity.

Before returning results, decide whether the user should see chunks or documents. For passage-oriented answers, return the best chunks and preserve their document references. For research or browsing interfaces, collapse repeated chunks from the same document after fusion, while retaining the highest-ranked evidence. Logging each path's rank, the merged rank, and the chunk ID makes relevance investigations far more concrete.

  • Merge on chunk ID, not on display title or approximate text matching.
  • Deduplicate repeated IDs before fusion if a retrieval path can emit them.
  • Keep dense rank, BM25 rank, and final rank in diagnostic logs.
  • Apply document-level grouping after chunk-level relevance has been established.