Treat meaning and exact terms as separate signals

Dense retrieval is valuable when a user’s wording differs from the wording in the relevant document. A query such as “how do I rotate access credentials?” can still be close in embedding space to content titled “API key lifecycle.”

Sparse BM25 retrieval contributes a different kind of evidence: exact or near-exact term overlap. That matters for ticket IDs, product names, error strings, API fields, filenames, acronyms, and other terms whose spelling carries meaning.

  • Use dense search to broaden recall across paraphrases and related concepts.
  • Use BM25 to retain precision for distinctive tokens and literal phrases.
  • Do not assume a semantically similar result contains the exact entity the user asked about.

Extract lexical anchors before issuing retrieval

A useful query-planning step is to identify lexical anchors: tokens that should remain prominent even when the rest of the query is interpreted semantically. In “resolve ERR_AUTH_401 after token refresh,” the error code is an anchor; in “compare retention for project Atlas,” the project name may be one.

Keep this step deliberately conservative. An ordinary word is not automatically an anchor, but quoted text, mixed-case identifiers, values containing punctuation, numbers, and uncommon terms are often strong candidates.

  • Preserve quoted phrases exactly when possible.
  • Flag structured-looking strings such as IDs, error codes, paths, and field names.
  • Keep a normalized query for embedding while retaining the original text for lexical retrieval.
  • Record which anchors were detected so unexpected results can be investigated.

Combine candidate sets with inspectable rules

Retrieve candidates from dense and sparse paths, then combine them in an application-level ranking policy that your team can explain and tune. The important engineering goal is not a universal formula; it is making sure an exact-match candidate is considered alongside semantically related candidates.

Evaluate the policy with a small, representative query set. Include paraphrases, short keyword queries, queries with identifiers, and queries containing ambiguous terms. Review the returned documents, not only whether a relevant document appears somewhere in the set.

  • Build test queries from real user language after removing sensitive content.
  • Include cases where one wrong character in an identifier changes the intended result.
  • Inspect dense-only, BM25-only, and combined candidate lists during debugging.
  • Adjust chunking and document text when both retrieval paths miss the same source.