Treat a chunk as an independent retrieval unit

A chunk is often displayed after retrieval, but it is first evaluated as a candidate on its own. If its meaning depends entirely on a preceding heading, table label, or sentence outside the chunk boundary, dense retrieval has less context to represent and sparse retrieval has fewer useful terms to match.

The goal is not to duplicate an entire document around every passage. Instead, add the minimum local context that makes the passage identifiable: the subject, the action or rule being described, and any important named entity, identifier, or scope.

  • Include a nearby section title when it supplies essential meaning.
  • Keep product names, field names, error codes, and policy terms in their original form.
  • Preserve units, dates, versions, and qualifiers such as "only" or "except".
  • Avoid chunks made primarily of pronouns, references, or unlabeled list items.

Use a compact context prefix instead of aggressive rewriting

A practical pattern is to prepend a small, repeatable context prefix before the body of a chunk. For example, a passage from a deployment guide can begin with its document title and section path, followed by the original passage. This gives semantic retrieval a clearer topic signal while exposing lexical terms such as component names and configuration labels to BM25.

Keep the source text recognizable. Excessive summarization can remove exact language that users type into queries, including acronyms, commands, model numbers, and internal terminology. A context prefix should clarify the passage, not replace it.

  • Use a stable shape such as "Document: … | Section: … | Content: …".
  • Prefer the source heading hierarchy over generated topical labels when possible.
  • Retain code, commands, and identifiers in the chunk body when they are searchable content.
  • Apply the same prefixing rule consistently across a corpus.

Validate chunk quality with two kinds of queries

Test chunking with both intent-oriented queries and exact-term queries. An intent-oriented query asks for a concept in natural language, while an exact-term query uses a specific token, phrase, or identifier. The same chunk should have enough context to be a plausible semantic result and enough faithful source language to be a plausible BM25 result.

Review misses before changing retrieval settings. A result may fail because the relevant term was split from its explanation, a heading was separated from its content, or the chunk is so broad that its central topic is unclear. These are corpus-design problems that can often be corrected before experimenting with more complex retrieval logic.

  • Create a small evaluation set containing both natural-language and exact-match queries.
  • Inspect the retrieved chunk text, not only document-level relevance.
  • Check whether key terms and their definitions remain in the same retrieval unit.
  • Re-run the set after changes to chunk boundaries or context prefixes.