Treat a chunk as the unit of retrieval

A document is often too large and too broad to be the right search result. Retrieval systems typically work better when they search smaller passages, sections, or records. The important design decision is to make that chunk definition identical for every retrieval path.

Create a canonical chunk record before generating embeddings or indexing text. That record should carry the chunk text, a stable identifier, source-document identity, ordering information, and any metadata needed for filtering or presentation. Both the dense and sparse indexes should be derived from this same record.

  • Use a stable chunk_id rather than a position-dependent database row ID.
  • Keep document_id and chunk_id separate: one document can produce many chunks.
  • Store chunk sequence or offsets so an application can restore surrounding context.
  • Version the chunking strategy when a change can alter chunk boundaries.

Index the same identity into dense and sparse paths

For dense retrieval, generate an embedding from the canonical chunk text and associate it with the chunk_id in the S3 Vectors-backed path. For sparse retrieval, index the searchable text in Quickwit BM25 with that exact same chunk_id and compatible metadata.

The text supplied to each system can differ deliberately, but the identity cannot. For example, BM25 may benefit from a title and body field, while an embedding input may include a compact title-prefixed passage. In both cases, the retrieved record must still resolve to the same canonical chunk.

  • Include chunk_id in every indexed representation.
  • Apply the same tenant, access-control, and lifecycle metadata to both paths.
  • Record an indexing version for embeddings and chunking separately.
  • Make index writes idempotent so retries do not create duplicate logical chunks.

Merge candidates by ID, then fetch the source of truth

Dense and sparse search can return overlapping candidate sets. When combining them, deduplicate on chunk_id before ranking or presenting results. This avoids showing the same passage twice and makes it clear when lexical and semantic retrieval independently selected the same content.

Do not rely on either search index as the only authoritative copy of display content. After candidate selection, fetch the canonical chunk record or a trusted content store using chunk_id. This makes updates easier to reason about and prevents stale indexed fields from silently becoming the user-facing answer.

  • Log which retrieval path contributed each candidate.
  • Measure overlap between dense and BM25 candidates during evaluation.
  • Reject or quarantine candidates whose IDs cannot resolve to canonical content.
  • Delete or tombstone the same chunk_id across both indexes when content is removed.