RAG or MCP? Build an AI Agent That Knows When to Search, When to Call a Tool, and When to Stop

The most expensive agent I've seen didn't fail. It answered correctly.

It took eleven retrieval iterations, three query rewrites, and a tool call to a system that had nothing to do with the question. The final answer was accurate. The bill was not. The latency was not. And the trajectory — the actual path the agent took — was indistinguishable from a much cheaper two-step run if you only looked at the output.

That's the trap in the RAG-versus-MCP debate. The question sounds like a choice between two architectures. It isn't. RAG is a retrieval pattern. MCP is a tool protocol. They operate at different layers, and the real engineering problem sits above both of them: the control policy that decides whether to retrieve, what to retrieve, whether the result is good enough, and when to stop.

Get that policy right and the stack choice becomes obvious. Get it wrong and you'll spend months tuning embeddings while the agent quietly loops.

📋 Table of Contents

• 1. RAG and MCP Are Not Alternatives — Stop Framing Them That Way
• 2. The Always-Retrieve Pipeline Breaks Before the Model Does
• 3. The Three Decisions Every Agent Makes (and Most Get Wrong)
• 4. Routing: Decide Whether to Search Before You Search
• 5. Multi-Hop Retrieval and the Stopping Problem
• 6. Tool Calls Are Retrieval With Side Effects
• 7. Wrap the Retriever as an MCP Tool
• 8. Budget the Loop: Iterations, Tokens, Repetition, and “Not Here”
• 9. Context Engineering: Tool Bloat Is a Retrieval Problem Too
• 10. Evaluate the Trajectory, Not Just the Answer
• The Decision Guide: What I’d Build and What I’d Skip

1. RAG and MCP Are Not Alternatives — Stop Framing Them That Way

A surprising amount of the “RAG vs. MCP” content treats them as competing answers to the same question. That framing produces bad architecture.

RAG is a pattern: embed a corpus, index it, retrieve relevant chunks at query time, and inject them into the prompt. It solves the knowledge problem. MCP is a protocol: an open, JSON-RPC-based standard for exposing tools, resources, and prompts to any compatible client. It solves the connection and action problem.

The confusion comes from the fact that both can deliver context. A retriever is, functionally, a tool the model can call. And an MCP resource can serve a document. But the similarity ends at the interface.

Dimension	RAG	MCP
What it is	An application pattern	An open protocol
Primary job	Ground generation in a known corpus	Standardize tool and resource access
Mechanism	Embed → index → retrieve → prepend	Server exposes tools; client invokes them
Data freshness	Depends on re-indexing cadence	Live at request time
Determinism	High — same query, same retrieval	Model decides when and whether to call
Side effects	Read-only	Can read or write
Best fit	Q&A over documents	Dynamic data, external systems, actions

The modern pattern is not either/or. It's RAG inside an agent, exposed through MCP. The retriever becomes one tool among several. The model decides on each turn whether retrieval is needed. Chat turns that don't need context skip the embedding and search round-trip entirely. The retriever can be swapped — BM25, dense, hybrid — without changing the model side of the system.

That's a meaningful architectural shift. But it doesn't answer the harder question: how does the model know when to call the retriever, when to call a different tool, and when to stop calling anything at all?

2. The Always-Retrieve Pipeline Breaks Before the Model Does

Scenario:
A user says, “Summarize what you just told me.” The pipeline embeds that sentence, searches the vector store, retrieves five chunks about return policies, and injects them into the prompt. The model summarizes the retrieved policy instead of the conversation.

Why it matters:
Naive RAG assumes every query needs exactly one retrieval pass. That assumption fails on the first turn where the answer is already in the conversation, already in the model's knowledge, or simply doesn't exist in the corpus. The retrieval isn't just wasted — it actively injects irrelevant context that can pull the answer off course.

Solution:
Stop treating retrieval as a pipeline step and start treating it as a decision. The agent should be able to skip retrieval entirely when it isn't needed, and the application should be able to enforce that skip for query classes it already understands.

A simple intent gate in Laravel, before the agent runs:

final class RetrievalGate
{
    private const SKIP_PATTERNS = [
        '/^(hi|hello|thanks|ok)\b/i',
        '/summari[sz]e (what|that|the above)/i',
        '/^(rewrite|rephrase|translate) /i',
    ];

    public function shouldRetrieve(string $prompt): bool
    {
        foreach (self::SKIP_PATTERNS as $pattern) {
            if (preg_match($pattern, trim($prompt))) {
                return false;
            }
        }

        return true;
    }
}

This isn't clever. It's a cheap first filter that catches the obvious cases before they cost an embedding call and a vector search. For everything else, let the model decide through a retrieval tool.

Why this works:
The gate removes the most predictable unnecessary retrievals without spending a model call to classify them. The remaining queries go through the agent's normal tool-selection loop, where the model can choose the retriever, another tool, or no tool.

💡 Practical note: Log every skip decision with the matched pattern. If a pattern starts suppressing legitimate retrieval, you'll see it in the logs before users complain about vague answers. Gates are cheap to add and cheaper to remove when they're observable.

3. The Three Decisions Every Agent Makes (and Most Get Wrong)

An agentic retrieval loop is not one decision. It's three, and they fail differently.

Decision one: whether to retrieve at all. A single-pass pipeline retrieves unconditionally. An agent evaluates the turn. “What's 2+2?” doesn't need a corpus pull. “Why did my card get declined again?” probably does, but the phrasing won't match the documentation, so the agent needs to rewrite it.

Decision two: what to query and where. Users don't phrase questions the way documents are written. The agent can rewrite the query into the vocabulary of the corpus, split a compound question into separate searches, or route to the right source when there are multiple indexes. Routing is a decision, and getting it wrong looks exactly like a retrieval failure.

Decision three: whether the result is good enough. This is the step that separates agentic RAG from a loop with a retry. The agent inspects what came back and judges sufficiency before generating. That judgment is itself an evaluation problem, and it's the one models are worst at. Models tend to say retrieved context is sufficient even when it isn't.

The failure mode of the third decision is the most expensive. A model that can't recognize thin evidence will reformulate and search again, and again, until something else stops it. That's why the stopping rule has to be external to the model.

🧠 The important part: The agent decides whether and what to retrieve. The application decides how many times and at what cost. A loop with no hard budget is not an agent — it's an unbounded search with a language model attached.

4. Routing: Decide Whether to Search Before You Search

Scenario:
Your support agent has two knowledge bases: billing and technical. A user asks about a failed charge. The agent searches the technical index because the query contains “failed” and “error,” retrieves irrelevant incident reports, and answers with a plausible but wrong explanation.

Why it matters:
Retrieval isn't free. Every unnecessary search adds latency, cost, and the risk of pulling in irrelevant context. When multiple knowledge bases exist, searching the wrong one is worse than not searching at all, because the retrieved chunks look relevant enough to suppress the agent's uncertainty.

Solution:
Add a routing step before retrieval. The router classifies the query into a source, and the agent only sees the retriever for that source. In LangGraph terms, this is a conditional edge: a router node evaluates state and sends execution down one of several paths.

A minimal router in PHP, before the agent gets a retriever tool:

final class QueryRouter
{
    public function route(string $prompt): string
    {
        $classification = (new BillingOrTechnicalClassifier)->classify($prompt);

        return match ($classification) {
            'billing' => 'billing_knowledge_base',
            'technical' => 'technical_knowledge_base',
            default => 'general_knowledge',
        };
    }
}

The agent's tool set is then scoped to the routed source:

public function tools(): iterable
{
    return match ($this->route) {
        'billing_knowledge_base' => [new SearchBillingDocs],
        'technical_knowledge_base' => [new SearchTechnicalDocs],
        default => [new SearchGeneralDocs],
    };
}

In LangGraph, the same idea is a conditional edge from a router node to one of several retrieval nodes. The router function inspects state and returns a string that maps to the next node. This is the primitive that turns a fixed pipeline into a runtime decision.

Why this works:
Scoping the tool set does two things at once. It prevents the model from searching the wrong index, and it reduces the number of tool definitions competing for attention in the context window. Tool selection accuracy degrades as the option space grows, so narrowing the set is both a retrieval-quality fix and a context-efficiency fix.

⚠️ Gotcha: A router that's wrong is worse than no router, because it confidently sends the agent to the wrong source. Measure router accuracy separately from end-to-end answer quality. If the router is below 90% on your evaluation set, fix the classifier before adding more sources.

5. Multi-Hop Retrieval and the Stopping Problem

Scenario:
A user asks, “Who founded the company with the highest revenue last quarter?” One retrieval finds the company. A second retrieval finds the founder. The agent answers correctly in two hops. A different user asks a question whose answer isn't in the corpus at all. The agent reformulates and searches eleven times before the request times out.

Why it matters:
Multi-hop questions can't be answered by a single retrieval. If you embed the whole question, no single document contains both facts, so retrieval either fails or returns an incomplete mix. Breaking the question into sub-questions and chaining lookups is the right pattern. But the same loop that makes multi-hop possible makes runaway search possible. Each iteration is at least one model call to decide, one search, and one call to assess. The cost multiplies silently.

Solution:
Make the loop explicit and put a budget around it. The loop should terminate on four conditions, not one:

1. The model returns no retrieval call. It has enough evidence to answer.
2. The iteration cap is reached. A hard maximum, enforced by the runtime.
3. The same query repeats past a threshold. A stuck-repetition signal.
4. The “not here” condition fires. The evidence is consistently absent, and the agent should say so instead of searching again.

A stopping policy in Laravel:

final class RetrievalBudget
{
    public function __construct(
        private int $maxIterations = 4,
        private int $maxRepeatedQueries = 2,
        private int $maxTokens = 12_000,
    ) {}

    public function exhausted(
        int $iterations,
        array $queryHistory,
        int $tokensUsed,
    ): bool {
        if ($iterations >= $this->maxIterations) {
            return true;
        }

        $repeats = collect($queryHistory)
            ->countBy()
            ->max();

        if ($repeats > $this->maxRepeatedQueries) {
            return true;
        }

        return $tokensUsed >= $this->maxTokens;
    }
}

The loop checks the budget before each retrieval, not after:

while (! $budget->exhausted($iterations, $queries, $tokens)) {
    $response = $agent->prompt($messages, tools: [$retriever]);

    if (! $response->hasToolCalls()) {
        return $response->text;
    }

    foreach ($response->toolCalls as $call) {
        $queries[] = $call->arguments['query'];
        $results = $retriever->handle($call->arguments);
        $tokens += $response->usage->totalTokens;
        $messages[] = toolResult($call->id, $results);
    }

    $iterations++;
}

return $agent->prompt(
    $messages,
    instructions: 'The retrieval budget is exhausted. Answer with what you have, or state that the information is not available.',
);

Why this works:
The budget is outside the model's control. No prompt can override a while condition. The final prompt on exhaustion gives the model a graceful exit: answer from what it has, or admit the gap. That's better than a timeout and better than a hallucinated answer.

🚨 Production warning: Do not set the iteration cap by intuition. Instrument the distribution of iteration counts in production and set the cap at the 99th percentile of legitimate runs. The average is misleading because the runaway cases live in the tail. A cap of four is a reasonable starting point for multi-hop support queries, but it should be derived from your data, not copied from a blog post.

6. Tool Calls Are Retrieval With Side Effects

Scenario:
The agent answers a policy question from the knowledge base. Then the user asks it to issue a refund. The agent calls a tool, the refund executes, and no one notices that the tool was given a stale order ID from a retrieved document rather than the current database record.

Why it matters:
RAG and tool calling look similar from the model's perspective. Both are structured actions that return text. But tools can write. A retrieval mistake produces a wrong answer; a tool mistake produces a wrong side effect. The blast radius is different, and the evaluation criteria should be different.

This is where MCP earns its place. MCP standardizes tool discovery, schema, invocation, and authentication. A tool exposed through MCP can be read-only or mutating, and the protocol's 2026-07-28 revision made the core stateless: every request carries its protocol version and client capabilities in _meta, the initialize handshake is gone, and servers expose server/discover for version negotiation.

The practical consequence for agents is important: tool state cannot live in the MCP session. There is no session. If a tool needs state across calls, the server mints an explicit handle and passes it as an ordinary argument. That forces the approval and audit logic into your application, where it belongs.

A tool policy that distinguishes read from write is not optional:

final class ToolPolicy
{
    public function requiresApproval(string $toolName): bool
    {
        return in_array($toolName, [
            'issue_refund',
            'cancel_subscription',
            'delete_account',
        ], true);
    }

    public function isReadOnly(string $toolName): bool
    {
        return str_starts_with($toolName, 'search_')
            || str_starts_with($toolName, 'look_up_');
    }
}

Why this works:
The policy is local to the application. The model can propose any tool call, but the runtime decides whether it executes automatically, pauses for approval, or is rejected. Read tools run inline. Write tools go through a gate the model cannot bypass.

🔍 Why this matters: The same query can be answered by retrieval or by a tool call, and the model will often choose the tool because it returns fresher data. That's usually correct — but it means retrieval quality and tool quality are coupled. A bad retrieval that sends the agent to a write tool is not a retrieval failure. It's a trajectory failure.

7. Wrap the Retriever as an MCP Tool

Scenario:
You have a working RAG pipeline. You add a second agent, and now you're duplicating the retriever logic. You add a third, and the two copies have drifted. One uses a different embedding model. The other filters by tenant differently.

Why it matters:
Hardcoded retrieval couples the agent to the retriever's implementation. It also hides the retriever from the model: the model can only call a tool it knows about. Wrapping the retriever as an MCP tool gives every agent one contract, one schema, and one place to enforce tenant isolation and access control.

Solution:
Expose the retrieval pipeline as an MCP tool. The tool takes a query and optional filters, returns ranked chunks, and handles the embedding and search internally. The agent doesn't know whether the retriever uses pgvector, BM25, or a hybrid pipeline.

namespace App\Mcp\Tools;

use App\Models\KnowledgeChunk;
use Laravel\Mcp\Tool;
use Laravel\Mcp\ToolInput;
use Laravel\Mcp\ToolOutput;

class SearchKnowledgeBase extends Tool
{
    public function description(): string
    {
        return 'Search the internal knowledge base for policies, procedures, and product documentation. Use this before answering any question about policies or account rules. If the search returns nothing relevant, say so instead of guessing.';
    }

    public function schema(): array
    {
        return [
            'type' => 'object',
            'properties' => [
                'query' => ['type' => 'string'],
                'source' => [
                    'type' => 'string',
                    'enum' => ['billing', 'technical', 'general'],
                ],
                'limit' => ['type' => 'integer', 'default' => 5],
            ],
            'required' => ['query'],
        ];
    }

    public function handle(ToolInput $input): ToolOutput
    {
        $chunks = KnowledgeChunk::query()
            ->where('tenant_id', $input->tenantId)
            ->when($input->source, fn ($q) => $q->where('source', $input->source))
            ->whereVectorSimilarTo('embedding', $input->query, minSimilarity: 0.35)
            ->limit($input->limit ?? 5)
            ->get();

        if ($chunks->isEmpty()) {
            return ToolOutput::text('No relevant documents found.');
        }

        return ToolOutput::text(
            $chunks->map(fn ($c) => "## {$c->title}\n{$c->content}")->join("\n\n")
        );
    }
}

Why this works:
The tool description tells the model when to call it and what to do when the result is empty. The schema constrains the query and the source. The handler enforces tenant isolation in code, not in the prompt. And because the retriever is now a tool, the agent can decide to skip it on turns that don't need it.

💡 Practical note: A retrieval tool that returns nothing should say so explicitly, not return an empty string. An empty result is ambiguous — the model may interpret it as “keep searching” or “answer from memory.” A clear “no relevant documents found” message gives the model a stopping signal.

8. Budget the Loop: Iterations, Tokens, Repetition, and “Not Here”

Most agent tutorials show an iteration cap and stop there. That's one budget, not four.

Iteration cap. The hard maximum on retrieval or tool calls. It prevents infinite loops. Vercel's ToolLoopAgent, for example, defaults to 20 steps with isStepCount(20) as the stop condition, and lets you override it with stopWhen. Twenty is generous for a general agent and far too high for a support retrieval loop. Set it based on your workload.

Token budget. A cumulative limit on input plus output tokens per run. Iteration caps don't catch the case where each iteration is expensive because the conversation history keeps growing. A token budget does.

Repetition limit. Stop when the same query or the same tool call repeats past a threshold. This catches the model that reformulates slightly but searches for the same thing. OpenRouter's guidance is explicit: stop when a call repeats too many times.

“Not here” rule. The hardest one. Models are reluctant to conclude that the answer isn't in the corpus, and they'll keep reformulating to avoid saying it. A “not here” condition should fire when retrieval consistently returns low-similarity or empty results, and the runtime should force a graceful failure rather than another search.

A four-signal budget class:

final class RunBudget
{
    private int $iterations = 0;
    private int $tokens = 0;
    private array $queries = [];
    private int $emptyResults = 0;

    public function recordIteration(string $query, int $tokens, bool $empty): void
    {
        $this->iterations++;
        $this->tokens += $tokens;
        $this->queries[] = $query;

        if ($empty) {
            $this->emptyResults++;
        }
    }

    public function shouldStop(): bool
    {
        if ($this->iterations >= 4) {
            return true;
        }

        if ($this->tokens >= 12_000) {
            return true;
        }

        if (collect($this->queries)->countBy()->max() >= 3) {
            return true;
        }

        return $this->emptyResults >= 2;
    }
}

Why this works:
Each signal catches a different failure. The iteration cap catches generic looping. The token budget catches expensive looping. The repetition limit catches stuck reformulation. The empty-result limit catches the “answer isn't here” case that the model won't admit. You need all four because they fail independently.

⚠️ Gotcha: Check the budget before executing the next tool call, not after. If this is the last permitted iteration, stop before running the tool — its result could never reach the model, and you've paid for a call you can't use.

9. Context Engineering: Tool Bloat Is a Retrieval Problem Too

Scenario:
Your agent has twelve MCP tools across four servers. The tool definitions consume a significant fraction of the context window before the user says anything. Tool selection accuracy drops, and the agent starts calling the wrong tool for simple requests.

Why it matters:
Tool definitions are not free. A typical agent carries 20–100 tools at 100–400 tokens per JSON schema, which is 2,000–40,000 tokens of pure boilerplate on every turn. GitHub's official MCP server has been cited at roughly 42,000 tokens of tool definitions — over a fifth of a 200K window. More tools mean more options, more schemas, and more descriptions competing for attention. Tool selection accuracy degrades as the option space grows.

This is the same problem as retrieval, just with a different corpus. The model is searching its context for the right tool, and the signal-to-noise ratio is dropping.

Solution:
Treat tool exposure as a scoped decision, not a global one. Scope tools per agent. Load only the tools relevant to the current intent. And use progressive disclosure: give the model a tool-search tool instead of every definition upfront.

A tool registry scoped by agent role:

final class ToolRegistry
{
    public function forRole(string $role): array
    {
        return match ($role) {
            'support' => [
                new SearchKnowledgeBase,
                new LookUpOrder,
            ],
            'deployment' => [
                new GitHubIssues,
                new DeploymentStatus,
            ],
            'finance' => [
                new LookUpInvoice,
                new IssueRefund,
            ],
            default => [],
        };
    }
}

For larger tool inventories, a tool-search pattern is more scalable. The agent gets one tool that searches tool definitions and returns the relevant schemas:

class FindTools extends Tool
{
    public function description(): string
    {
        return 'Search available tools by capability. Call this before using a tool you have not used in this session.';
    }

    public function handle(ToolInput $input): ToolOutput
    {
        $matches = ToolIndex::search($input->capability, limit: 5);

        return ToolOutput::text(
            $matches->map(fn ($t) => $t->schema)->toJson()
        );
    }
}

Why this works:
Progressive disclosure reduces the upfront token cost and keeps the model focused. Claude Code's tool-search documentation describes exactly this trade-off: instead of loading all definitions upfront, load only the tools the model needs, which improves both context efficiency and selection accuracy. LangChain's Deep Agents v0.7 reported a 65% reduction in base input tokens from a leaner harness at comparable performance.

🧠 The important part: The “more tools equals more capable” assumption is the most common agent design mistake. Capability is not the sum of available tools. It's the model's ability to choose the right tool at the right time, and that ability degrades as the option space grows.

10. Evaluate the Trajectory, Not Just the Answer

Scenario:
Two runs produce the same correct answer. One retrieved once, used a relevant chunk, and stopped. The other retrieved six times, rewrote the query into something adjacent to the original question, and got lucky on the last search. Your evaluation suite scores them identically.

Why it matters:
Final-answer scoring can't distinguish a clean path from a wasteful one. The same answer can come from a two-step retrieval or from nine flailing searches. If you only measure the output, you'll optimize for the wrong thing and never see the runaway cases until the bill arrives.

Solution:
Evaluate the trajectory. Record every step as a span: the decision to retrieve, the query, the retrieved chunks, the sufficiency assessment, and the stopping decision. Then score the path, not just the destination.

The metrics that matter:

Metric	Question it answers
Retrieval precision	Were the retrieved chunks relevant to the query?
Query quality	Did the agent rewrite the query sensibly, or drift off target?
Stopping correctness	Did the agent stop when it had enough, or too early / too late?
Iteration count	How many loops did it take, and was each one productive?
Groundedness	Is the final answer supported by the retrieved evidence?
Cost per successful run	Tokens and tool calls divided by correct answers

A simple trajectory recorder in Laravel:

final class Trajectory
{
    private array $steps = [];

    public function record(string $type, array $payload): void
    {
        $this->steps[] = [
            'type' => $type,
            'payload' => $payload,
            'at' => now()->toIso8601String(),
        ];
    }

    public function toArray(): array
    {
        return $this->steps;
    }
}

Why this works:
The trajectory is the evidence you need to debug a bad run. Without it, you can't tell whether a wrong answer came from a bad retrieval, a bad model decision, or a bad stopping rule. With it, you can replay the run, compare it to a known-good trajectory, and fix the specific step that failed.

🔍 Why this matters: Agentic RAG requires trajectory-level evaluation because a correct answer can come out of a wasteful or lucky path. If your evaluation only looks at the final text, you're not measuring the thing that determines cost, latency, and reliability.

The Decision Guide: What I’d Build and What I’d Skip

The RAG-or-MCP question dissolves once you separate the layers. RAG is how you ground answers in a corpus. MCP is how you expose tools and resources to the agent. The control policy is what decides when either is used.

Here's the architecture I'd build for a production support agent in 2026:

Use naive RAG when: the workload is a well-scoped FAQ over one clean corpus, queries are simple and single-fact, and latency matters more than flexibility. A single retrieval pass is cheaper, faster, and easier to debug. Don't add a control loop because it sounds more advanced. Add it because a measured failure mode demands it.

Use query routing when: you have multiple knowledge bases, mixed query types, or a meaningful fraction of turns that don't need retrieval at all. The router is one extra classification step, and it prevents the most expensive failure: searching the wrong source and answering confidently from irrelevant context.

Use multi-hop retrieval when: questions require chaining facts across documents, and a single retrieval demonstrably fails. Cap the hops and log the intermediate queries. Multi-hop is where cost and latency start to climb.

Use MCP when: tools need to be shared across agents, clients, or teams; when you need a standard contract for tool discovery and auth; or when the tool ecosystem already speaks MCP. Don't build an MCP server for a tool only one agent uses — function calling is simpler and faster.

Wrap the retriever as an MCP tool when: more than one agent needs retrieval, or when you want the model to decide whether to search on each turn. This is the pattern that turns RAG from a pipeline into a capability.

Don't use an agentic loop at all when: a single retrieval pass solves the problem. The loop is not free. Each iteration is a model call, a search, and an assessment. If the task is lookup, not research, the pipeline is the correct architecture.

The production checklist, in the order I'd implement it:

1. Fix retrieval before adding agency. Chunking, embeddings, hybrid search, and reranking determine whether the right passage is reachable. An agentic loop on top of a broken retriever just searches for the wrong thing more times.
2. Add a routing gate for the obvious skips. Cheap patterns first, then a classifier if needed.
3. Wrap the retriever as a tool. One schema, tenant isolation enforced in code, explicit empty results.
4. Enforce four budgets. Iterations, tokens, repetition, and empty results.
5. Scope tools per agent. Don't give the support agent the deployment tools.
6. Record trajectories. Every decision, query, result, and stop condition.
7. Evaluate the path. Precision, query quality, stopping correctness, groundedness, cost per success.

The one thing I wouldn't do is choose between RAG and MCP as if they were competitors. The agent that knows when to search, when to call a tool, and when to stop is the one that has both — and a control policy that treats “stop” as a first-class decision, not an afterthought.