The assistant said: "I've cancelled your subscription and you won't be charged again." The customer believed it, closed the tab, and was charged eleven days later. The assistant had no tool for cancellations. It had generated a sentence that was plausible, helpful, and false.
This is the moment most teams start thinking about giving the assistant an API. It's the right instinct, and it's usually where the next set of mistakes begins — because "give it an API" is not a design. A model with access to POST /subscriptions/{id}/cancel is not a workflow. It's a chatbot with a gun.
The real transition from chatbot to workflow is a set of decisions about which requests deserve an action, what kind of action, who authorizes it, and what the user sees while it happens. Get those right and the assistant becomes genuinely useful. Get them wrong and you've automated the production of irreversible mistakes.
Describing an action and taking one are different products
A chatbot's job is to produce text that helps a human decide what to do next. A workflow's job is to change state in a system of record. Those are different products with different failure costs, different latency expectations, and different definitions of success.
The failure cost difference is the one that matters. A wrong sentence wastes a few seconds and damages trust slightly. A wrong refund moves money, a wrong cancellation churns a customer, a wrong address ships a package to the wrong place. When the output of a language model becomes an input to a state change, the tolerance for "probably right" collapses.
That's why the interesting question isn't "should the assistant have tools?" Almost every useful assistant eventually needs read access to something. The interesting question is: for this specific request, should the system act, propose, or answer? Those are three distinct modes, and most products need all three.
Classify the request before you design the tool
Requests that arrive at an assistant fall into a few categories, and each has a natural mode.
| Request type | Example | Natural mode | Why |
|---|---|---|---|
| Informational | "What's your return window?" | Answer | No state change, cheap to be wrong |
| Diagnostic | "Why was I charged twice?" | Read + answer | Needs data, not action |
| Navigational | "Where do I update my card?" | Answer with deep link | The UI is better than the action |
| Transactional, low risk | "Resend my receipt" | Execute with confirmation | Reversible, idempotent |
| Transactional, high risk | "Cancel my account" | Propose, then execute | Irreversible, needs intent |
| Ambiguous | "Fix my billing" | Clarify | The action is underspecified |
| Out of scope | "Dispute this charge with my bank" | Hand off | Needs a human process |
The classification is not something the model should do unsupervised, at least not at first. It's something you should do from your own logs: take a thousand real conversations, label them, and look at the distribution. You will usually find that 60–80% of requests are informational or diagnostic, which means the highest-value first tool is a read tool, not a write tool. That's a much cheaper and safer place to start.
Four tests before you wire a write API
Before an assistant gets the ability to change something, run the request through four questions. If any answer is uncomfortable, the mode is "propose" rather than "execute."
1. Is it reversible? Sending a receipt is reversible in effect (you can send a correction). Cancelling an account is reversible only through a retention flow that may not exist. Reversibility determines how much confirmation you need and whether the action should require an approval step.
2. Is the intent unambiguous? "Cancel my subscription" is clear about the object and the action, but not about timing, refund expectations, or whether the customer means the add-on. An ambiguous intent executed confidently is worse than an ambiguous intent answered with a question.
3. Is the target deterministic? If the assistant has to choose between two subscriptions, three shipping addresses, or four open orders, the model is making a selection that the user should make. Read tools can help disambiguate; write tools should not guess.
4. Is the actor accountable? Someone has to be able to say who authorized the action. If the answer is "the model decided," you don't have accountability, you have a plausible deniability generator. Either the user confirmed, an approver approved, or a policy with a named owner authorized it.
A useful shorthand: execute when the action is reversible, unambiguous, deterministic, and user-confirmed. Propose when any of those is shaky. Answer when the request doesn't need state to change at all.
Read tools first, and read tools properly
Read tools are the best return on investment in assistant design. They turn "I can't see your account" into "your last invoice was on March 3 and it's still pending," which resolves a large share of support contacts without any risk of a bad write.
They're not risk-free, though. A read tool still needs authorization (as discussed in the permissions piece), and it still needs to return data the model can use. Three design rules make read tools effective:
Return facts, not prose. A tool that returns a formatted paragraph invites the model to paraphrase it loosely. A tool that returns structured fields gives the model something to reason over and gives your logs something to compare.
Include the identifiers the model will need for follow-up calls. If get_order returns an order but not its order_id, the model will invent one or ask the user. Include stable IDs, and include display names separately so the model doesn't confuse them.
Cap the result size. A tool that returns 400 line items will consume the context window and degrade the answer. Paginate, summarize server-side, or return a count plus the top N with an explicit truncated: true.
The point of read tools isn't only to answer questions. They're how the assistant earns the right to write. A model that can look up the order, the customer, and the policy is far more likely to construct a correct write than one that has to infer all three from the conversation.
Write tools need a different contract
A write API designed for a human-driven frontend is usually wrong for an agent, and not because of authentication. It's wrong because the frontend supplies context that the model doesn't have: which user is acting, which record is selected, whether the action was already attempted, and what the user saw before clicking.
Four contract changes close most of the gap.
Idempotency keys derived from the run. The tool should accept or derive a key that identifies the business intent, so a retry or a duplicated model call returns the original result instead of performing the action twice. Deriving it from (run_id, action, resource) is usually right; deriving it from the model's arguments alone invites duplicates when the model rephrases.
An explicit dry-run. A dry_run: true mode that performs all validation and returns what would happen, including the amount, the target, and any policy constraints. This is the single most useful tool for building confirmation UI: the model proposes, the system validates, the user sees a precise summary.
Structured errors with recovery hints. {"code": "insufficient_funds", "retryable": false, "hint": "Offer the customer a payment link instead"} lets the model respond usefully. A bare 422 produces an apology loop.
Narrow semantics. cancel_subscription(subscription_id, reason_code, effective: "immediately" | "end_of_period") is a better tool than update_subscription(fields: object). The narrow version has a small state space, an auditable meaning, and no way to accidentally change the plan while cancelling.
@tool
def cancel_subscription(
subscription_id: str,
reason_code: Literal["too_expensive", "missing_feature", "switching", "other"],
effective: Literal["immediately", "end_of_period"],
dry_run: bool = False,
) -> ToolResult:
"""Cancel a subscription. Use dry_run first to show the customer what will change."""
The Literal types are doing quiet work: they constrain the model's output space, they make the tool's behavior testable, and they give you a taxonomy for analytics later. Free-form reason: str gives you a column of prose you'll never aggregate.
The confirmation step is an interface, not a formality
"Are you sure? Yes / No" is the worst possible confirmation. It asks the user to confirm something they can't see, based on a model's summary they may not have read carefully. The confirmation should show the exact change: which object, which values, which effective date, and what happens next.
Cancel "Pro plan" for acme.example?
Effective: end of current period (April 30, 2026)
You will keep access until then.
You will not be charged on May 1.
[Cancel subscription] [Keep plan]
Three properties make this work.
It's generated from validated data, not from the model's text. The model proposes the action; the system renders the confirmation from the tool's dry-run result. If the model's summary and the dry run disagree, the dry run wins and the discrepancy is logged — that's a model bug you want to see.
It names the object in human terms. "Pro plan for acme.example" is checkable. "Your subscription" is not.
It states the consequence. "You will not be charged on May 1" is a commitment the system can keep. If it can't be made, the confirmation is lying, which is worse than no confirmation.
The confirmation also gives you a clean audit event: the user approved a specific action with specific arguments at a specific time. That event is what makes the write defensible later.
Where the model doesn't belong
The most valuable workflow design decision is often to take the model out of the execution path entirely.
If the assistant determines the user wants to cancel, and cancellation has a five-step retention flow with offers and confirmations, the right architecture is: the assistant classifies intent, then hands off to a deterministic workflow that owns the steps. The model's job ends at routing. It doesn't narrate each step, it doesn't decide whether to skip the retention offer, and it doesn't get to improvise a discount.
This pattern — model for understanding, code for doing — is how you get the benefits of natural language input without accepting natural language as a control flow. It's also easier to test: the routing is a classification problem, and the workflow is ordinary software with ordinary tests.
The same logic applies to multi-step operations. "Change my plan and apply the retention discount and update my billing date" should be a named workflow with a fixed sequence, not five tool calls the model sequences on the fly. Tool sequences chosen by a model are hard to audit, hard to test, and prone to partial completion.
Latency is part of the contract
A chatbot can stream an answer and feel fast. A workflow that calls three APIs and then writes can take six seconds, and users notice.
The design lever is to separate the perceived action from the actual action:
- The model responds immediately: "I can cancel that at the end of your billing period. Here's what will change."
- The confirmation renders from a dry run, which is fast because it doesn't mutate anything.
- The commit happens on confirmation, with an optimistic UI state and a clear "working" indicator.
- The result is reported with a stable reference: "Cancelled. Reference: sub_cancel_8f2c."
What you must not do is let the model narrate progress it can't observe ("cancelling now... almost done..."). That's the original sin of this article, dressed as UX.
If an action is slow enough that the user might leave, the right pattern is a job: accept the request, return a reference, and notify when it completes. That's a normal API design, and it applies to agents too.
Who owns the state while the action is in flight
The hardest part of the transition isn't the API call. It's deciding where the workflow's state lives, because an assistant's conversation and a backend workflow have different lifetimes.
A conversation can last an hour with gaps. A workflow that cancels a subscription and issues a proration credit has a defined set of steps and a much shorter window. If you keep that state in the model's context, three things go wrong: the model forgets steps after summarization, it re-derives progress from prose, and a resumed conversation has no reliable way to know whether step three completed.
Keep workflow state in your system, keyed by a workflow ID that the assistant holds as an opaque string.
@dataclass
class WorkflowRef:
id: str
kind: str # "cancel_subscription"
state: str # "proposed" | "approved" | "committed" | "compensated"
version: int
result: dict | None
The assistant's tools become transitions on that state rather than free-standing actions:
@tool
def begin_cancellation(subscription_id: str) -> WorkflowRef:
"""Start a cancellation workflow. Returns a reference; nothing has changed yet."""
@tool
def confirm_cancellation(workflow_id: str, expected_version: int) -> WorkflowRef:
"""Commit the cancellation. Fails if the workflow changed since you read it."""
Two properties fall out of this. The model can't accidentally skip a step, because the transition function rejects invalid states. And your audit log gets a workflow ID that ties every event together, which makes the reconstruction in the previous article trivial.
It also solves the resumed-conversation problem. When a user comes back the next day and says "did that go through?", the assistant reads the workflow by ID rather than trying to remember. The answer comes from the system of record, not from context.
Making an existing API agent-ready without forking it
Most teams already have the endpoints. They're built for a frontend that knows the user, has the record selected, and will render a form. The temptation is to expose them directly as tools and let the model figure out the parameters. That usually produces a tool that works in demos and fails on the first ambiguous case.
An adapter layer is cheaper than a rewrite and much safer than direct exposure. It does five jobs:
Maps a small intent surface onto the larger API. cancel_subscription calls PATCH /subscriptions/{id} with a body the frontend would normally construct. The model never sees the PATCH; it sees one verb with three validated fields.
Resolves identifiers server-side. If the user says "my Pro plan," the adapter resolves that to a subscription ID using the authenticated principal, and returns an error if the resolution is ambiguous. The model doesn't guess between two subscriptions; it asks.
Translates errors into recovery hints. The billing API returns 402 with a provider-specific code. The adapter maps it to {"code": "payment_method_declined", "retryable": false, "hint": "Ask the customer to update their card; do not retry the cancellation."} The model can act on that. It cannot act on a raw 402.
Enforces limits the frontend enforced implicitly. Maximum refund amount, one cancellation per subscription per day, no address changes after fulfillment. These belong in the adapter, where they apply to every caller, not in the prompt, where they apply until someone edits them.
Adds the idempotency and dry-run semantics the original API lacks. If the underlying endpoint isn't idempotent, the adapter keeps a request ledger keyed by the business intent and short-circuits duplicates. This is often ten lines of code and it prevents the most expensive failure mode in the whole system.
The adapter is also where you keep the model's context small. A raw API response with forty fields and three nested objects is mostly noise to a language model. Return the eight fields that matter, plus IDs for follow-up, plus an explicit truncated flag when you've dropped something. Smaller, shaped responses improve both accuracy and cost.
⚠️ Gotcha: if the adapter and the frontend construct the same request differently, you now have two definitions of "cancel." Put the adapter's request construction in shared code, or accept that the two paths will drift and write a contract test that compares them.
What the model should never see in a tool result
The tool result is part of the prompt. Anything you return becomes context the model can quote, summarize, or leak into an answer. That has consequences beyond token cost.
Don't return internal risk scores, fraud flags, margin data, other customers' identifiers, internal notes, or full document bodies when a summary suffices. Not because the model will maliciously disclose them, but because it will helpfully mention them when a user asks a related question. "Why was my refund declined?" is a very natural prompt for a model that has a risk score in context.
Don't return secrets or tokens, even scoped ones, unless the tool needs to hand them to another tool — and if it does, keep them out of the model's visible result by passing them through a server-side reference.
Do return the fields the model needs to reason and the IDs it needs to act. The test is simple: for each field, ask whether a support agent would put it in a customer-facing email. If not, it probably shouldn't be in the tool result either.
Four failure modes worth designing against
Duplicate action. The model calls the write twice because the first result was slow, or the run was retried. Fix: idempotency keys, plus a check in the tool that returns the existing result with deduplicated: true.
Wrong entity. The user says "cancel the second one," the model picks the first. Fix: require explicit IDs in write calls, resolve them through a read tool, and show the resolved object in the confirmation.
Partial action. A multi-step workflow cancels the subscription but fails to stop the invoice. Fix: named workflows with compensation, not model-sequenced calls. If the model must sequence, give it a single tool that performs the whole operation transactionally.
Stale state. The model reads the order, the user changes it in another tab, the model writes based on the old version. Fix: optimistic concurrency (expected_version or updated_at in the write), and return a conflict the model can explain rather than silently overwriting.
None of these are model problems. They're the ordinary problems of building a client that writes to an API, and they get worse when the client is non-deterministic and has no memory of its last attempt.
Measuring whether the transition worked
The metrics that matter change when you move from answering to acting. Answer quality is still relevant, but it's no longer sufficient.
| Metric | What it tells you | Warning sign |
|---|---|---|
| Containment | Share of conversations resolved without a human | Rising while CSAT falls — you're containing, not resolving |
| Action success rate | Share of proposed actions that complete | Below ~95% suggests tool or validation problems |
| Action reversal rate | Share of completed actions undone | High rates mean the confirmation isn't working |
| Clarification rate | Share of requests where the assistant asked | Very low may mean it's guessing |
| Handoff rate by intent | Which intents still need humans | Your roadmap, ranked |
| Time to resolution | End-to-end, not first response | Actions that add latency without reducing contacts |
Action reversal rate is the one teams forget to instrument, and it's the most diagnostic. If 12% of cancellations are reinstated within a week, the assistant is executing an intent the user didn't have, and no amount of model quality fixes an interface problem.
The migration path, in order
Most successful transitions follow the same sequence, and skipping steps is how teams end up shipping an autonomous write tool to a support queue that isn't ready for it.
Stage 1: answer with citations. No tools. The assistant answers from documentation, links to sources, and says "I don't know" when it doesn't. Measure answer quality and the share of questions it can't handle.
Stage 2: read-only copilot. Add tools that look things up: orders, invoices, tickets, account status. The assistant answers questions about the user's actual data. This is where most of the value shows up, and it's where you learn what people actually ask.
Stage 3: propose actions. The assistant can call write tools in dry-run mode and produce a structured proposal, but a human clicks the button. Measure how often proposals are correct, and how often they're accepted without edits. This is your training data for stage 4.
Stage 4: assisted execution. The assistant executes low-risk, reversible actions on explicit confirmation. High-risk actions still route to a human. Instrument reversal rates obsessively.
Stage 5: bounded autonomy. The assistant executes within policy limits without per-action confirmation — refunds under a threshold, address changes before shipment, resends. Each limit has an owner and a review date.
Each stage has a clear entry criterion: the previous stage's error rate has to be low enough that the next stage's failure cost is acceptable. That's a business decision, not an engineering one, but engineering should provide the number.
When staying a chatbot is the right answer
There are products where the assistant should never write, and saying so is a legitimate design position rather than a lack of ambition.
- Regulated decisions with individual assessment requirements. If a human must assess the specifics of a case, an assistant that decides is a compliance problem, not an efficiency gain.
- Actions with no undo and high emotional weight. Account deletion, data erasure, cancellation of a bereavement-related service. The confirmation burden outweighs the convenience.
- Requests that need negotiation. Pricing exceptions, contract terms, escalations. These are conversations with another human, and automating the first half makes the second half worse.
- Systems where the API doesn't exist. If the write path is a manual back-office process, an agent calling a fragile internal endpoint is a new failure mode, not an integration.
In those cases, the assistant still earns its keep by preparing the action: gathering the facts, drafting the message, routing to the right queue with a summary. That's a workflow too, just one where a human holds the write permission.
The decision, compressed
For each request type in your logs, ask:
- Does answering require data the user can't see? → add a read tool.
- Is the state change reversible and unambiguous? → candidate for execute.
- Is it irreversible, expensive, or underspecified? → propose, confirm, or route to a human workflow.
- Does the operation have more than one step? → build it as a named workflow, not a model-chosen sequence.
- Can you state, in one sentence, who authorized the action? → if not, don't ship the write.
The transition from chatbot to workflow isn't a feature flag. It's the point where your product starts making commitments on the user's behalf, and every commitment needs a way to be kept, explained, and undone. The teams that do it well don't have better models. They have narrower tools, better confirmations, and a clear line between what the model understands and what the system is allowed to do.