Define the retrieval unit before choosing ranking logic

A retrievable unit should represent a meaningful piece of source material: for example, a documentation section, policy clause, support article passage, or product record. The same unit should be sent to dense indexing and sparse indexing whenever the application expects results from those systems to be evaluated together.

If dense search indexes paragraph-sized chunks while BM25 indexes entire pages, the two result sets answer different questions. Dense retrieval may identify a precise passage, while sparse retrieval may return a broad document whose keyword match occurs far from the relevant text. That mismatch makes ranking behavior difficult to interpret.

  • Choose a source-aware boundary such as a heading, section, article, or record.
  • Set a target chunk size appropriate to the content, not a single universal size.
  • Preserve nearby context when splitting long sections, using overlap only when it serves a clear retrieval need.
  • Avoid mixing multiple unrelated topics in one chunk simply to reduce index records.

Make chunk IDs and metadata stable across both indexes

Every chunk needs a stable identifier that is shared by the dense and sparse representations. A practical ID can be derived from a canonical document identifier, a section identifier, and a version or content revision marker. The exact format matters less than its determinism: rebuilding an index should produce the same ID for unchanged content.

Metadata should also follow a common schema. Fields such as document ID, title, source URL or path, section position, content type, tenant, locale, and access scope help an application filter, display, and audit retrieved chunks. Store enough information to resolve a result without guessing which source passage it represents.

  • Use one canonical chunk ID in S3 Vectors and Quickwit BM25.
  • Keep filterable fields named and typed consistently across indexing pipelines.
  • Include a document revision or content hash to support update and deletion workflows.
  • Store display metadata separately from the retrieval text when the application needs citations or result labels.

Compare and combine results only after normalizing the candidate set

Dense search and BM25 produce scores with different meanings, so raw scores should not be treated as directly comparable. Instead, retrieve a candidate list from each system, deduplicate by the shared chunk ID, and apply an explicit application-level ranking policy. That policy may prioritize rank position, use a reciprocal-rank-style fusion method, or route certain query types to one retrieval path first.

A shared chunking contract makes debugging this process much simpler. For a query that performs poorly, inspect which chunk IDs appeared in each candidate list, which metadata filters were applied, and whether both representations contain the expected current revision. This turns retrieval tuning into a concrete data investigation rather than a guess about embeddings or keywords.

  • Treat dense and BM25 scores as system-specific signals rather than interchangeable values.
  • Deduplicate candidates using the canonical chunk ID before presenting or reranking results.
  • Log query text, filters, returned chunk IDs, ranks, and document revisions for evaluation.
  • Review failures by source content type, query intent, locale, and chunk boundary quality.