Two tools return different answers, and the agent has to decide which one to believe. In most codebases, that decision is made by the model, silently, inside a prompt. The agent reads both results, picks the one that "looks more consistent," and writes an answer. Nothing in the logs records that a conflict existed. Nothing records which source lost.
That is a strange way to run a system. If two backend services returned different prices for the same SKU, you would not ask a language model to average them and move on. You would look at which service owns the price, how old each value is, and whether the disagreement indicates a bug. An agent's tools deserve the same treatment.
The interesting part is that tool disagreement is not primarily a model problem. It is a data-provenance problem that surfaces through the model. Once you accept that, the design work moves out of the prompt and into the tool layer, where you can test it.
Disagreement is a signal, not noise to smooth over
Here is a common trajectory. An agent is asked whether an invoice has been paid. It calls a billing tool and gets status: "paid". It calls a reporting tool and gets status: "overdue". The model decides the billing tool sounds more authoritative and answers "paid."
Sometimes that is right. Sometimes the billing tool read a stale replica, and the reporting tool is correct because it queries the ledger directly. The model had no way to know. It substituted plausibility for provenance.
A disagreement between two tools is evidence about your system: one source is stale, one is wrong, two sources measure different things under the same name, or the write path has not propagated. Suppressing the conflict destroys that evidence. A good agent does not just pick a winner; it produces a decision that a human can reconstruct later.
🔍 Why this matters: the cost of a wrong pick is often invisible. The agent returns a fluent answer, the user acts on it, and the conflict never appears in a metric. You find out weeks later when a customer disputes a charge.
Four disagreements that look the same and aren't
Engineers tend to say "the tools disagree" as if there were one failure mode. There are at least four, and they need different handling.
Value conflict. Two sources return different values for the same field at the same point in time. The invoice example above. Resolution depends on ownership and freshness.
Existence conflict. One tool says the record exists; another says it does not. This is common when one service has a soft-delete and another has a hard filter, or when a search index lags behind the primary database. It is almost never safe to resolve by preferring the positive answer.
Schema or semantic conflict. Both tools return a number, but one is in cents and the other in dollars; one reports total including tax, the other excluding it. The values are not contradictory, they are incomparable. This is a tool-contract bug, and the right response is usually to stop and fix the contract rather than teach the agent a conversion.
Temporal conflict. Both values are correct, at different times. A cached tool returns yesterday's balance; a live tool returns today's. Without timestamps, the agent cannot tell the difference between a stale value and a wrong one.
A useful exercise: for every tool your agent can call, write down which of these four it can participate in. Most teams discover that their tools return bare values with no timestamp, no source identifier, and no unit metadata. At that point, arbitration is guesswork by construction.
The tempting wrong fix: let the model reconcile
The first instinct is to add a line to the system prompt: "If tools disagree, use the most authoritative source and explain your reasoning." This fails for three reasons.
First, authority is not visible in the tool output. The model sees two JSON objects. Unless you put an authority rank in the payload, it is inferring from field names and tool descriptions, which is exactly the surface most likely to mislead.
Second, the reasoning is not replayed. The model's explanation is generated after the fact. It is a narrative, not a decision record, and it will sound confident either way.
Third, prompt-level arbitration is untestable in the way you need. You can write an eval case where tools conflict, but you cannot assert on the internal choice with any stability across model versions. You can only assert on the final text, which is the least reliable part of the system.
There is a place for the model in conflict handling: interpreting ambiguous evidence, summarizing why a conflict matters, and drafting a message to a human. There is no good place for it in deciding which database owns a fact.
Give every claim a provenance envelope
The fix starts before the model is involved. Make each tool result carry enough metadata to reason about it. A minimal envelope has five fields:
- the value, in a normalized unit
- the field or predicate it describes
- the source identifier, including the replica or region if relevant
- the observation time, in UTC
- an authority class, assigned by you, not by the model
Authority classes are deliberately coarse. Three levels cover most systems: derived (computed caches, search indexes, aggregations), cached (materialized views, CDN responses, snapshot exports), and system of record (the service that owns writes for that field). If you find yourself needing ten levels, you probably have an ownership problem, not a ranking problem.
A tool result can then be treated as a claim, not an answer:
from dataclasses import dataclass, field
from datetime import datetime, timedelta, timezone
from enum import IntEnum
class Authority(IntEnum):
DERIVED = 10
CACHED = 20
SYSTEM_OF_RECORD = 30
@dataclass(frozen=True)
class Claim:
field: str
value: object
source: str
authority: Authority
observed_at: datetime
confidence: float | None = None
@dataclass
class Resolution:
field: str
value: object | None
status: str # "resolved" | "conflict" | "insufficient"
reason: str
evidence: list[Claim] = field(default_factory=list)
Assume Python 3.10+ for the union syntax. The confidence field is optional on purpose; most tools do not produce a calibrated score, and inventing one is worse than leaving it empty.
A precedence policy you can explain to an auditor
With envelopes in place, resolution becomes a small deterministic function. The policy I would default to:
- Drop claims older than the field's freshness budget (for example, 60 seconds for a balance, 24 hours for a shipping address).
- Prefer the highest authority class among the remaining claims.
- If the highest class has exactly one distinct value, resolve to it.
- If the highest class has more than one distinct value, treat it as a conflict. Do not fall through to a lower class, because that would silently prefer a weaker source over an unresolved strong one.
- If no claim survives the freshness filter, return
insufficientrather than the least-stale guess.
Step 4 is the one teams get wrong. It is tempting to keep looking until something breaks the tie. But two systems of record disagreeing is a serious signal — a replication split, a failed write, a double-entry bug. Falling through to a cache to produce a clean answer hides it.
Values need normalization before comparison, or you will generate false conflicts. Compare money in minor units with an explicit currency, timestamps in UTC, and strings after a documented case and whitespace policy. For floats, use a field-specific tolerance rather than exact equality; 0.1 + 0.2 and 0.3 should not trigger an incident. Put normalization in one module and test it, because every downstream arbitration depends on it.
def resolve(
claims: list[Claim],
freshness: dict[str, timedelta],
now: datetime | None = None,
) -> Resolution:
now = now or datetime.now(timezone.utc)
if not claims:
return Resolution("", None, "insufficient", "no claims supplied")
field_name = claims[0].field
budget = freshness.get(field_name, timedelta(minutes=5))
fresh = [c for c in claims if now - c.observed_at <= budget]
if not fresh:
return Resolution(
field_name, None, "insufficient",
f"all {len(claims)} claims exceeded freshness budget",
claims,
)
top = max(c.authority for c in fresh)
strongest = [c for c in fresh if c.authority == top]
distinct = {normalize(c.value) for c in strongest}
if len(distinct) == 1:
winner = max(strongest, key=lambda c: c.observed_at)
return Resolution(
field_name, winner.value, "resolved",
f"authority={top.name}, {len(strongest)} corroborating claim(s)",
fresh,
)
return Resolution(
field_name, None, "conflict",
f"{len(distinct)} distinct values at authority={top.name}",
fresh,
)
normalize is intentionally not shown; it belongs in your domain code and should be unit-tested per field type. The important property is that the function is pure, has no model call, and returns the evidence it used.
Correlated errors: when two tools agree and are both wrong
Quorum feels safe until you notice that two tools can share an upstream. A search index and a product page may both read the same denormalized table. A CRM sync and a support tool may both consume the same nightly export. Agreement between them tells you almost nothing.
Before you treat agreement as corroboration, ask whether the sources are independent. If they share a write path, a cache, or a vendor, they are one source with two names. The dangerous version of this is a conflict that gets "resolved" by majority vote among three tools where two are downstream of the same stale replica.
The practical fix is to record a lineage identifier alongside authority. Two claims with the same lineage should be collapsed into one before voting. This is cheap to add and prevents a whole class of false confidence.
Self-consistency techniques make the same mistake in a different place. Sampling the same model multiple times and taking the majority reduces variance on some reasoning tasks, but it does not create independent evidence. If the model is wrong because the retrieved context is wrong, five samples will be wrong together. Majority voting over model samples is a variance-reduction trick, not a truth-finding mechanism, and it should not be used to overrule a tool that owns the data.
Freshness belongs to the field, not the tool
A tool is not globally fresh or stale. The same endpoint might return an account balance from a live ledger and a lifetime-value metric from a nightly job. Freshness budgets have to be attached to fields.
This matters because "most recent timestamp wins" is a bad universal rule. A user-edited shipping address saved thirty seconds ago is more current than a warehouse system's copy from last week. A recomputed risk score from a batch job five minutes ago may be less trustworthy than a score from this morning if the batch is known to be flaky. Recency is evidence; it is not authority.
A table helps make the policy explicit:
| Field type | Typical owner | Freshness budget | On conflict |
|---|---|---|---|
| Account balance | Ledger service | Seconds | Block and flag |
| Shipping address | User profile service | Until edited | Prefer most recent user write |
| Order status | Order service | Seconds | Block and flag |
| Search document | Index | Minutes | Prefer primary, log lag |
| Risk score | Batch scorer | Hours | Present with timestamp |
| Product price | Pricing service | Minutes | Block; never average |
The right-hand column is the part people skip. Deciding what to do with an unresolved conflict is a product decision, and it should be made per field, not globally.
Resolution is not always the goal
Sometimes the correct behavior is to stop. An agent that cannot resolve a conflict should be able to say so, and the system around it should make that safe.
Three outcomes are usually enough:
Resolve and continue. The policy produced a single value with adequate authority and freshness. The agent proceeds, and the decision record notes which claim won and why.
Degrade to a constrained answer. The agent can still help, but not with the disputed fact. "I can see the order shipped on the 14th, but the payment status is inconsistent between two systems, so I can't confirm whether it's been charged." This is often more useful than a refusal and more honest than a guess.
Escalate. For irreversible actions, a conflict should block the action, not merely annotate it. If the agent is about to issue a refund, a disagreement about whether the order was already refunded is a stop condition. There is no prompt wording that makes proceeding acceptable.
The failure mode to avoid is a fourth outcome that appears in real systems: resolve silently, proceed, and log nothing. That is how a conflict becomes an incident with no timeline.
Record the conflict, not just the winner
Your audit record should answer four questions without re-running anything: what did each source claim, when was each claim observed, which policy version was applied, and what came out.
A compact structure works better than free-text reasoning:
{
"decision_id": "dec_01J8Z3K4",
"field": "order.payment_status",
"policy_version": "arbitration-2026-09",
"outcome": "conflict",
"claims": [
{"source": "billing-api", "authority": "system_of_record",
"value": "paid", "observed_at": "2026-09-14T09:12:03Z"},
{"source": "reporting-replica", "authority": "derived",
"value": "overdue", "observed_at": "2026-09-14T09:11:58Z"}
],
"reason": "1 distinct value at authority=CACHED vs SYSTEM_OF_RECORD; precedence applied",
"action_taken": "degraded_response"
}
Note the discipline: the outcome is a status, not a paragraph. You can count conflicts per field, alert on rate changes, and show the evidence to a support engineer without asking a model to rephrase it. If you want a human-readable explanation, generate it from this record at render time. Do not store the generated prose as the source of truth.
This record also gives you a feedback loop. Conflicts that resolve cleanly are boring. Conflicts that cluster on one field are telling you something: a lagging replica, a broken sync, a naming collision. That is a bug report your agent produced for free.
Tool contracts that make arbitration possible
Most of the work happens in tool design, before any agent logic exists. A few conventions pay for themselves quickly.
Return structured metadata, not prose. If a tool returns a sentence like "The customer's plan is Pro, as of this morning," the agent has to parse it. Return {"plan": "pro", "observed_at": "...", "source": "..."} and let the model talk about it.
Never return a bare boolean for existence. exists: false is ambiguous: not found, not authorized, or filtered out. Distinguish not_found from forbidden and unavailable. An agent that treats forbidden as not_found will confidently tell a user their record does not exist.
Use an explicit error channel. MCP tool results carry an isError flag, and the convention exists for a reason: a tool that failed must not look like a tool that returned an empty result. If your adapter swallows a 500 and returns [], you have created a disagreement that is actually an outage.
Version the semantics. If total changes from tax-exclusive to tax-inclusive, bump the field name or the contract version. Silent semantic changes are the hardest conflicts to debug because both tools are "working."
State units in the payload. {"amount": 1250, "currency": "USD", "minor_units": true} is unambiguous. {"amount": 12.50} invites a hundred-fold error that no arbitration policy can catch.
The part the model is good at
Once arbitration is deterministic, the model's job becomes narrower and more useful. It can:
- explain the conflict to a user in plain language, using the decision record as input;
- decide whether a conflict is material to the user's actual question, when the field list is long;
- draft a message to the owning team with the evidence attached;
- propose which field-level policy should apply, for a human to approve.
That last one is interesting. Models are decent at reading a conflict and suggesting that "these two values probably mean different things." But the suggestion should become a reviewed policy change, not a runtime decision. You want the model improving the rules between incidents, not improvising during them.
When the user is one of the sources
A user's message is a claim too, and it often collides with a tool. "I already paid this invoice" arrives while the billing tool says unpaid. The agent now has two sources and a social pressure to side with the human.
Treat user statements as a distinct authority class — call it asserted — that sits below any system of record but above nothing else. A user assertion is valuable evidence about intent and about events the system may not have recorded yet, such as a bank transfer that has not reconciled. It is not proof of state. The right behavior is to verify, then explain what verification found.
This matters for tone as much as correctness. "You're right, that's paid" when the ledger disagrees is a bad answer that will be quoted back to support later. "The ledger still shows this as unpaid; bank transfers sometimes take a day to appear, so here's the reference number and here's what I can see" is honest and useful. The agent can hold the user's claim and the system's claim at the same time without pretending they agree.
The same logic applies to documents the user uploads. A PDF invoice is a claim from an external source. It may be forged, outdated, or simply a different revision. If the agent is going to act on it, the action needs a check against a system that owns the state, not just a plausible-looking document.
Propagating a conflict through a plan
Single-fact conflicts are the easy case. The harder one is a multi-step plan that silently depends on a disputed value. An agent checks the order status, decides it is delivered, and prepares a replacement shipment. If the status was in conflict and the arbitration defaulted to the wrong claim, the error is now embedded three steps downstream, where it looks like a decision rather than a data problem.
The fix is to keep the plan's dependencies explicit. When a tool result comes back with status: "conflict", any step that consumes that field should be marked blocked, not merely annotated. In practice that means the planner needs a small amount of structure: each step declares which facts it reads, and a blocked fact invalidates every dependent step.
You do not need a full planning engine for this. A dictionary from fact name to resolution status is enough:
blocked = {r.field for r in resolutions if r.status != "resolved"}
def step_is_runnable(step) -> bool:
return not (set(step.reads) & blocked)
The value of this is not elegance. It is that the agent cannot "reason around" a blocked fact by rewording the plan. If the refund step reads order.payment_status and that field is in conflict, the step does not run, regardless of how confident the model sounds about the alternative.
There is a second-order benefit: when the conflict clears on a retry, only the dependent steps need to be reconsidered. Everything already completed stays completed. Without dependency tracking, a late-resolving conflict tends to trigger a full restart, which is both expensive and likely to duplicate side effects.
Testing disagreement paths
Conflict handling is easy to write and easy to leave untested, because the happy path never exercises it. A small fixture suite catches most regressions:
- Single claim, fresh → resolved.
- Single claim, stale → insufficient, not resolved.
- Two claims, different authority, both fresh → higher authority wins.
- Two claims, same authority, different values → conflict.
- Two claims, same authority, same value, different sources → resolved with corroboration.
- Same-lineage claims that agree → counted once.
- Missing timestamp → treated as stale, or rejected at the contract boundary.
- Unit mismatch → rejected by normalization, not arbitrated.
- Tool error represented as empty result → caught by schema, not by arbitration.
Run these as plain unit tests against resolve, not through the model. You are testing the policy, and the policy should not need a model to be correct. Then add two or three end-to-end cases to confirm the agent degrades or escalates as intended when resolve returns conflict.
One more test is worth writing: a conflict where the correct behavior is to ask the user. Agents are often eager to resolve; a test that asserts "the agent did not proceed" is the only thing that keeps that behavior honest.
Where this design costs you
Per-tenant or per-field authority metadata is real work. It requires someone to decide, for each field, which service owns it. In a system with a fuzzy ownership history, that conversation is uncomfortable, and it is also the conversation you needed to have anyway.
There is also a latency cost if freshness checks require a live read. You may need to accept a slightly slower answer for high-stakes fields, or precompute freshness at write time. And there is a product cost: users do not enjoy "the systems disagree." The mitigation is to make the degraded answer specific and actionable, not to hide the conflict.
For low-stakes fields, full arbitration is overkill. A summary of a support ticket does not need an authority class. The investment should follow consequence: money, access, legal state, and anything irreversible get envelopes and policy; a draft email does not.
A decision guide for the next conflict
When you find your agent picking between two tool results, work through this order:
- Is this actually a conflict? Normalize units, currency, time zones, and case before you treat two values as different.
- Do both claims have timestamps and sources? If not, fix the tool contract first. Arbitration without provenance is theater.
- Is one source the owner of the field? If yes, encode that as authority and stop there.
- Are the sources independent? If they share lineage, collapse them before comparing.
- Is the field consequential? If yes, an unresolved conflict blocks the action. If not, degrade and continue.
- Can the user see the disagreement? For anything the user will act on, show the timestamp and the uncertainty rather than a false single value.
- Did you log the losing claim? If not, you have thrown away the most useful debugging artifact in the system.
The broader point is that an agent is a consumer of facts it did not produce. Treating those facts as claims with provenance, rather than as answers, is what makes the agent's behavior explainable when the facts collide. The model can write the explanation. It should not be the explanation.