Move beyond traditional agentic search with specialised retrieval sub-agents

Everything an agent sees becomes part of its context. The system prompt, conversation history, tool calls, tool outputs all compete for the same limited context window. For short interactions this is rarely a problem, but long-horizon agentic workloads, especially those involving repeated tool use and multi-hop reasoning, require context to be managed much more deliberately.

Common retrieval patterns

A common way to extend a model beyond its training data is a single-stage retrieval pipeline, often referred to as retrieval-augmented generation, or RAG. A query is sent to a retrieval system such as a vector database, relevant chunks are returned, and those chunks are added to the model's context before generation.

This works well when the information needed can be retrieved in one step but many real-world tasks are harder. Web research may begin with one query but require several follow-up searches as new concepts or sources are discovered.

Agentic search addresses this by giving the model search tools directly. The model can search, inspect results, refine its query, and search again. This flexibility is useful, but it introduces two costs. 

1. Every query and retrieved result can accumulate inside the main agent's context, including information that later turns out to be irrelevant. 

2. Multi-hop retrieval is often sequential, which adds latency as the model alternates between reasoning and tool calls.

Context retrieval sub-agents

A retrieval sub-agent keeps the flexibility of agentic search while moving most of the retrieval process outside the main agent's context. The main agent delegates a search task, the retrieval agent performs its own search loop, and only the final evidence or summary is returned.

This pattern appears in systems such as Chroma Context 1, SWE-grep, and Mixedbread Toast. Their implementations differ, but all treat retrieval as a bounded agentic workload rather than a sequence of search calls executed directly by the primary agent.

Advantages of the sub-agent pattern

• Preserves adaptive retrieval: The retrieval process remains agentic. The sub-agent can reformulate queries, follow references, explore different parts of the corpus, and stop once it has enough evidence.

• Protects the main agent's context: Search often requires many intermediate queries, tool outputs, and retrieved chunks before finding the useful few. Keeping that exploration inside the retrieval agent prevents noisy or irrelevant context from accumulating in the main agent loop.

• Creates a clear optimisation boundary: Retrieval can be evaluated independently using metrics such as precision, recall, latency, and context consumption, making the search harness easier to tune without changing the rest of the system.

Components of a retrieval sub-agent

A useful retrieval agent needs more than access to a search API. The surrounding harness determines how efficiently the model can explore a corpus and how much unnecessary context it accumulates.

Robust retrieval tools

Agentic retrieval does not remove the need for a strong retrieval layer. In practice, better retrieval often means fewer search turns and less context consumed by the agent. A model can compensate for weak search by issuing more queries, but this usually increases latency and context usage without guaranteeing better coverage.

Chunk state management

Long retrieval loops often surface the same chunks multiple times. Keeping track of previously retrieved chunks allows the harness to deduplicate results and avoid bloating the context. This state can also support later pruning, since the system knows which chunks are old, duplicated, or no longer relevant to the active search path.

Parallel tool calling

Many retrieval operations are independent and can run at the same time. A research agent might issue different formulations of the same query and compare the results. Parallel tool calling reduces the number of sequential round trips required for multi-hop retrieval.

Context window awareness

The agent should know roughly how much usable context remains. Without this signal, it can keep accumulating results until earlier evidence begins to crowd out more useful information. The harness can expose context usage every turn or notify when it approaches a threshold.

Context pruning

Some chunks are useful temporarily because they point the agent toward a better query, but they are no longer useful once that next step has been completed. Providing context-editing tools allows the model to remove stale or irrelevant information and retain only the evidence needed for the rest of the task. This becomes increasingly important as retrieval chains grow longer.

Training models specifically for retrieval

A separate retrieval agent also creates a useful training target because search is relatively verifiable. In many settings, it is possible to measure whether the correct document was found or not.

Chroma Context 1 and SWE-grep explore this direction using small models trained specifically for search-oriented agent loops. Their approaches use supervised fine-tuning as a starting point and reinforcement learning to improve behaviour over time.

The goal is not necessarily to make the model more knowledgeable rather improve behaviours such as query formulation, parallel search, stopping at the right point, and removing irrelevant context.

Corpus specific optimizations

There is no single retrieval setup that works best across every domain. The right search tools and the right model both depend heavily on the structure of the underlying data.

Choosing search tools

Coding agents often work well with lexical and structural tools such as grep, glob, and filename search. Code contains explicit identifiers, imports, paths, and references that give models strong search anchors.

Other domains may benefit more from semantic retrieval, while some work best with hybrid approaches that combine semantic and lexical search. A retrieval agent does not necessarily need access to every available search method. A smaller tool surface designed around the structure of the corpus can make retrieval more predictable and easier to optimise.

Choosing the model

Because retrieval is separated from the main agent, the search agent model can also be chosen independently. Simpler retrieval tasks may work well with smaller and cheaper models, while domains that require more interpretation or broader knowledge may benefit from stronger models.

The right choice depends on both the complexity of the retrieval path and the nature of the information being searched.

Conclusion

Agentic search gives models a flexible way to navigate large corpora, but keeping the entire retrieval loop inside the main agent becomes increasingly expensive as tasks get longer. Retrieval sub-agents address this by isolating the exploratory part of search while preserving the model's ability to adapt its retrieval strategy.

The result is a cleaner separation of responsibilities. The main agent focuses on the broader task, while the retrieval agent manages search, intermediate evidence, context usage, and pruning. For simple questions, a single retrieval step may still be sufficient. For workloads that require repeated exploration across large corpora, treating retrieval as its own agentic workload provides a more scalable way to manage context.