Treat context as an indexed product
Start by defining the agent's allowed knowledge boundary. Create an index for the workload, choose the region and dimensions, and store metadata that represents tenant, user, visibility, source type, and freshness. An agent should not receive a workspace-wide token or unrestricted source dump when a read-only key scoped to one index is enough.
Use Serverless Processing for documents and connectors, and direct vector writes for application events or structured knowledge you already transform. Both paths produce retrieval records with provenance. This gives the agent evidence that can be inspected, replaced, or deleted rather than an untraceable prompt cache.
Retrieve before generation
For every agent action that depends on company knowledge, retrieve first. Generate a query embedding, retain the original user text as a sparse query, inject mandatory metadata filters, and ask for a bounded number of candidates. Include source names, pages, and record IDs with the context sent to the model.
Use hybrid retrieval for most business questions. Dense search covers paraphrases and conceptual requests; sparse search covers policy codes, people, account names, error IDs, and exact terminology. A relevance threshold lets the agent say it lacks evidence instead of turning a weak nearest neighbor into a confident answer.
Choose direct retrieval, Assistant RAG, or MCP
Call the Vector API directly when you need full control over prompts, models, reranking, and agent orchestration. Use Assistant RAG when you want a hosted grounded chat surface with instructions, source citations, SSE streaming, thread IDs, token accounting, and conversation traces. Use MCP when an MCP-capable client should discover and call retrieval tools through a standard remote tool interface.
AWS's agent and grounding guidance describes the same core pattern: retrieve curated evidence before generation, then retain traceability. The Assistant toolkit exposes `search_context` and optional `source_preview`. Keep the toolkit narrow and the API key read-only. Retrieval is a tool for evidence, not an unrestricted gateway into every database or connector your company uses.
Measure agent quality through evidence
Log the user request, index, filters, retrieved record IDs, sources, latency, token use, and final outcome. Review cases where the agent gave an unsupported answer, missed a known source, or retrieved data outside the expected boundary. Those traces tell you whether to improve the corpus, embedding, sparse text, filters, threshold, prompt, or application policy.
The practical goal is not to make an agent appear omniscient. It is to make it useful when evidence exists, transparent about where that evidence came from, and reliably able to stop when the index cannot support an answer.
Use AWS agent guidance to keep the tool boundary small
AWS's agent and serverless grounding guidance emphasizes a clear distinction between an agent's reasoning and the systems it is permitted to access. For a retrieval tool, this means the agent receives a narrow operation: search a specific index with controlled filters and return evidence. It does not receive broad database credentials or an invitation to decide which data it may access.
Treat each tool call as an observable event. Persist the thread ID, requested query, filter policy, returned IDs, selected citations, latency, and token counts. Set a relevance threshold and a maximum context budget. These controls make it possible to improve retrieval from observed failures without allowing a model to silently expand its data boundary.