What Happens When an LLM Returns a Half-Finished Tool Call?

byHossein Hezami •
0

You are streaming an agent response. The model says it will call create_invoice.

The tool name arrives. Then the arguments start arriving:

{
  "customer_id": "cus_9821",
  "currency": "USD",
  "line_items": [
    {
      "sku": "PRO-SEAT",
      "quantity": 3,
      "price_cents": 49

Then the stream stops.

Do you execute the tool? Do you retry? Do you ask the model to try again? Do you show an error to the user? Do you send a tool result back saying the call failed?

This is the half-finished tool call problem. It sounds niche until your agent system has real tools, real side effects, and real users. Suddenly you are dealing with truncated JSON, missing required fields, interrupted streams, schema violations, and the uncomfortable question of whether the model meant to do something that it failed to finish expressing.

The wrong answer is to sprinkle try/catch around JSON.parse and hope.

The right answer is to treat tool calls as untrusted, incomplete proposals that must pass through a strict pipeline before they are allowed to touch your system.

TL;DR

  • A half-finished tool call is not one failure mode. It can be transport truncation, token truncation, malformed JSON, schema failure, or semantic incompleteness.
  • Never execute a tool call while streaming. Execute only after the response is finalized and validated.
  • Treat tool arguments like untrusted input: parse, validate with a schema, then apply business rules.
  • If a call is invalid, you can often ask the model to repair it, but use a bounded retry loop.
  • Do not let an invalid tool call create partial side effects. Use idempotency keys and transactional execution.
  • Log the raw tool call attempt, finish reason, validation errors, and repair attempts.
  • Design tools so the model has less to generate: IDs instead of giant objects, pagination instead of huge arrays, enums instead of free text.

📋 Table of Contents

The failure you see is rarely the failure you have

A naive agent loop looks something like this:

const response = await llm.chat(messages);

for (const toolCall of response.toolCalls) {
  const args = JSON.parse(toolCall.arguments);
  const result = await executeTool(toolCall.name, args);
  messages.push(toolResult(toolCall.id, result));
}

This works beautifully in demos.

Then production happens.

The model returns:

{"customer_id":"cus_9821","line_items":[{"sku":"PRO-SEAT","quantity":3,"price_cents":49

JSON.parse throws. Your agent loop crashes. Or worse, you catch the error and silently continue. Or even worse, you try to “helpfully” complete the JSON yourself and execute something the model did not actually request.

The deeper problem is that “half-finished tool call” is an umbrella term. It can mean:

  • The network stream ended early.
  • The model reached its output token limit.
  • The tool arguments are syntactically invalid JSON.
  • The JSON is valid but fails schema validation.
  • The JSON and schema are valid, but the business intent is incomplete.
  • One tool call in a batch is complete while another is not.
  • The model emitted a tool call but no final stop reason.
  • The provider stopped the response due to a content filter.
  • The client disconnected before the server

What Happens When an LLM Returns a Half-Finished Tool Call?

You are streaming an agent response. The model says it will call create_invoice.

The tool name arrives. Then the arguments start arriving:

{
  "customer_id": "cus_9821",
  "currency": "USD",
  "line_items": [
    {
      "sku": "PRO-SEAT",
      "quantity": 3,
      "price_cents": 49

Then the stream stops.

Do you execute the tool? Do you retry? Do you ask the model to try again? Do you show an error to the user? Do you send a tool result back saying the call failed?

This is the half-finished tool call problem. It sounds niche until your agent system has real tools, real side effects, and real users. Suddenly you are dealing with truncated JSON, missing required fields, interrupted streams, schema violations, and the uncomfortable question of whether the model meant to do something that it failed to finish expressing.

The wrong answer is to sprinkle try/catch around JSON.parse and hope.

The right answer is to treat tool calls as untrusted, incomplete proposals that must pass through a strict pipeline before they are allowed to touch your system.

TL;DR

  • A half-finished tool call is not one failure mode. It can be transport truncation, token truncation, malformed JSON, schema failure, or semantic incompleteness.
  • Never execute a tool call while streaming. Execute only after the response is finalized and validated.
  • Treat tool arguments like untrusted input: parse, validate with a schema, then apply business rules.
  • If a call is invalid, you can often ask the model to repair it, but use a bounded retry loop.
  • Do not let an invalid tool call create partial side effects. Use idempotency keys and transactional execution.
  • Log the raw tool call attempt, finish reason, validation errors, and repair attempts.
  • Design tools so the model has less to generate: IDs instead of giant objects, pagination instead of huge arrays, enums instead of free text.

📋 Table of Contents

The failure you see is rarely the failure you have

A naive agent loop looks something like this:

const response = await llm.chat(messages);

for (const toolCall of response.toolCalls) {
  const args = JSON.parse(toolCall.arguments);
  const result = await executeTool(toolCall.name, args);
  messages.push(toolResult(toolCall.id, result));
}

This works beautifully in demos.

Then production happens.

The model returns:

{"customer_id":"cus_9821","line_items":[{"sku":"PRO-SEAT","quantity":3,"price_cents":49

JSON.parse throws. Your agent loop crashes. Or worse, you catch the error and silently continue. Or even worse, you try to “helpfully” complete the JSON yourself and execute something the model did not actually request.

The deeper problem is that “half-finished tool call” is an umbrella term. It can mean:

  • The network stream ended early.
  • The model reached its output token limit.
  • The tool arguments are syntactically invalid JSON.
  • The JSON is valid but fails schema validation.
  • The JSON and schema are valid, but the business intent is incomplete.
  • One tool call in a batch is complete while another is not.
  • The model emitted a tool call but no final stop reason.
  • The provider stopped the response due to a content filter.
  • The client disconnected before the server finished accumulating the response.

Each of these needs a different response.

If you collapse them all into catch (error), you will build an agent that feels randomly broken.

1. A tool call is a proposal, not a command

The most important mental model is this:

An LLM tool call is not an executable command. It is a proposed command.

It may be malformed. It may be incomplete. It may be dangerous. It may be based on stale context. It may have been cut off mid-generation.

Your system should treat it the same way a backend treats an incoming API request from an untrusted client.

A safe pipeline looks more like this:

LLM response
  ↓
Accumulate raw tool call data
  ↓
Check completion reason
  ↓
Parse arguments
  ↓
Validate against schema
  ↓
Apply business validation
  ↓
Apply policy and permission checks
  ↓
Execute tool idempotently
  ↓
Return structured tool result

In code, that might be represented as a state machine:

type ToolCallStatus =
  | "received"
  | "transport_incomplete"
  | "invalid_json"
  | "schema_invalid"
  | "business_invalid"
  | "policy_rejected"
  | "approved"
  | "executed"
  | "failed";

type ToolCallRecord = {
  callId: string;
  toolName: string;
  rawArguments: string;
  finishReason?: string;
  status: ToolCallStatus;
  parsedArguments?: unknown;
  validationErrors?: string[];
  repairAttempts: number;
  createdAt: string;
};

This gives you a place to reason about the call before executing it.

The key shift is that execution is not the first thing that happens after parsing. Execution is the last thing that happens after several gates.

🔍 Why this matters: Once tool calls can write data, send messages, charge customers, or call external APIs, the cost of a bad parse is no longer a stack trace. It can be a partial refund, a wrong email, or a corrupted record.

2. Five different kinds of “half-finished”

Before you can handle a half-finished tool call, you need to classify it.

I think of five main categories.

Failure typeSymptomDetectionBest response
Transport incompleteStream ended before final responseMissing stop/finish event, socket errorRetry LLM request or resume if supported
Token truncationJSON stops mid-string or mid-objectfinish_reason / stop_reason indicates length limitReduce payload, split tool, repair, increase budget if possible
Malformed JSONJSON.parse failsParse errorAsk model to repair or fail safely
Schema-invalidJSON parses but violates tool schemaSchema validation failureReturn validation errors for repair
Semantically incompleteValid schema but missing business intentBusiness rules failAsk clarifying question or reject

This classification matters because the recovery strategy changes.

If the stream disconnected, retrying the model request may be enough.

If the model ran out of tokens, asking it to “try again” without changing the task may just produce the same truncation.

If the JSON is invalid, a repair loop may work.

If the JSON is valid but the intent is incomplete, the model needs clarification, not a JSON repair prompt.

A useful normalization layer can hide provider differences:

type NormalizedToolCall = {
  callId: string;
  name: string;
  rawArguments: string;
  finishReason?: string;
  provider: string;
};

Some providers return tool arguments as a JSON string. Others return structured input blocks. Some stream argument deltas. Some return the final tool call object only after completion. Your job is to normalize all of that into one envelope before you reason about it.

3. Token truncation: the model ran out of room before the JSON closed

This is the classic half-finished tool call.

The model is generating a tool call. The arguments are long. It reaches the output token budget. The response stops.

You may get something like:

{
  "report_id": "rpt_2231",
  "filters": {
    "start_date": "2026-01-01",
    "end_date": "2026-03-31",
    "segments": [
      "enterprise",
      "mid-market",
      "self-serve"
    ],
    "metrics": ["mrr", "churn", "expansion"

The JSON never closes.

The first clue is often the completion reason. Different providers use different names, but the concept is common:

  • OpenAI-style APIs may expose a finish_reason.
  • Anthropic-style APIs may expose a stop_reason.
  • Local inference servers may expose a similar completion reason.

A generic handler might look like this:

type CompletionReason =
  | "stop"
  | "tool_calls"
  | "tool_use"
  | "length"
  | "content_filter"
  | "error"
  | "unknown";

function isPossiblyTruncated(
  reason: CompletionReason | undefined,
  toolCall: NormalizedToolCall
): boolean {
  if (reason === "length") {
    return true;
  }

  // If the provider says the response ended normally, still sanity-check JSON.
  try {
    JSON.parse(toolCall.rawArguments);
    return false;
  } catch {
    return true;
  }
}

Do not treat length as an automatic hard failure if the tool call still parses and validates. Sometimes the truncation happened after the tool call was complete, or the provider’s reason is coarse. But treat it as a strong warning.

The better long-term fix is to reduce the chance that tool arguments become huge.

For example, instead of asking the model to emit a full report with thousands of rows, use a tool like:

{
  "name": "create_report_job",
  "arguments": {
    "report_type": "revenue_summary",
    "start_date": "2026-01-01",
    "end_date": "2026-03-31",
    "segment_filter": ["enterprise", "mid-market"]
  }
}

Then let your backend generate the large artifact asynchronously.

In general, tool arguments should contain intent and references, not enormous payloads.

Bad:

{
  "name": "send_email",
  "arguments": {
    "recipients": ["... 5,000 emails ..."],
    "body": "... 20 KB of generated HTML ..."
  }
}

Better:

{
  "name": "send_email_campaign",
  "arguments": {
    "campaign_id": "camp_8842",
    "audience_id": "aud_enterprise_trial",
    "template_id": "tpl_onboarding_v3"
  }
}

💡 Practical note: Increasing the model’s max output tokens can help, but it is not a universal fix. Providers have model-level output caps, and long tool arguments still increase latency, cost, and truncation risk. If your tool calls regularly need large payloads, your tool design is probably wrong.

4. Streaming tool calls are drafts until the stream commits

Streaming makes the problem more confusing because the tool call appears to exist before it is finished.

You may receive deltas like:

tool name: create_invoice
arguments chunk: {"customer_id":"cus_98
arguments chunk: 21","currency":"USD",
arguments chunk: "line_items":[{"sku":"PRO

At this point, you have a partial tool call. It is not a tool call yet. It is a draft.

The biggest mistake is executing from the draft.

A safer accumulator looks like this:

type ToolCallBuffer = {
  callId?: string;
  name?: string;
  argumentChunks: string[];
  finalized: boolean;
  finishReason?: string;
};

class StreamInterruptedError extends Error {}

function accumulateToolCallDelta(
  buffer: ToolCallBuffer,
  delta: {
    callId?: string;
    name?: string;
    argumentChunk?: string;
  }
) {
  if (delta.callId) buffer.callId = delta.callId;
  if (delta.name) buffer.name = delta.name;
  if (delta.argumentChunk) buffer.argumentChunks.push(delta.argumentChunk);
}

function finalizeToolCall(buffer: ToolCallBuffer): NormalizedToolCall {
  if (!buffer.finalized) {
    throw new StreamInterruptedError("Tool call stream ended without finalization");
  }

  if (!buffer.callId || !buffer.name) {
    throw new StreamInterruptedError("Tool call stream missing id or name");
  }

  return {
    callId: buffer.callId,
    name: buffer.name,
    rawArguments: buffer.argumentChunks.join(""),
    finishReason: buffer.finishReason,
    provider: "normalized",
  };
}

The important detail is finalized.

A tool call should only be considered finalized when the provider has indicated the response is complete. Depending on the API, that may be a final response object, a done event, a message stop event, or a completion reason.

If you are building a user-facing chat UI, it is fine to show a status like:

Preparing tool call: create_invoice

But do not execute until the tool call is final.

There is also an architectural point here: if possible, do the authoritative stream accumulation on the server, not in the browser. Browsers disconnect. Tabs sleep. Networks change. If your agent’s tool execution depends on a client-side stream staying alive, you are building on sand.

A common production shape is:

Client → Backend API → LLM provider stream
                ↓
          Backend accumulates tool calls
                ↓
          Backend validates and executes
                ↓
          Backend streams safe UI events to client

The client can display progress, but the backend owns truth.

⚠️ Gotcha: If you retry after a stream interruption, be careful if the original request may have already caused side effects. For pure LLM generation, retrying is usually safe. For tools that may have already executed, you need idempotency and reconciliation.

5. Parse and validate tool arguments like untrusted user input

Once you have a finalized tool call, the next mistake is assuming the arguments are meaningful just because they are JSON.

They are not.

Tool arguments need the same treatment as form input:

  1. Parse.
  2. Validate against a schema.
  3. Apply business rules.
  4. Apply permissions.

A good library for schema validation in TypeScript is Zod. The exact library is less important than the pattern.

import { z } from "zod";

const CreateInvoiceSchema = z.object({
  customer_id: z.string().min(1),
  currency: z.enum(["USD", "EUR", "GBP"]),
  line_items: z
    .array(
      z.object({
        sku: z.string().min(1),
        quantity: z.number().int().positive(),
        price_cents: z.number().int().nonnegative(),
      })
    )
    .min(1),
  idempotency_key: z.string().optional(),
});

type CreateInvoiceArgs = z.infer<typeof CreateInvoiceSchema>;

Then classify the tool call:

type ToolCallParseResult<T> =
  | {
      status: "ok";
      callId: string;
      name: string;
      args: T;
      warnings: string[];
    }
  | {
      status: "invalid_json";
      callId: string;
      name: string;
      rawArguments: string;
      parseError: string;
    }
  | {
      status: "invalid_schema";
      callId: string;
      name: string;
      rawArguments: string;
      issues: string[];
    };

export function parseToolCall<T>(
  call: NormalizedToolCall,
  schema: z.ZodType<T>
): ToolCallParseResult<T> {
  const warnings: string[] = [];

  if (call.finishReason === "length") {
    warnings.push("Response ended due to length limit");
  }

  let parsedJson: unknown;

  try {
    parsedJson = JSON.parse(call.rawArguments);
  } catch (error) {
    return {
      status: "invalid_json",
      callId: call.callId,
      name: call.name,
      rawArguments: call.rawArguments,
      parseError: error instanceof Error ? error.message : String(error),
    };
  }

  const result = schema.safeParse(parsedJson);

  if (!result.success) {
    return {
      status: "invalid_schema",
      callId: call.callId,
      name: call.name,
      rawArguments: call.rawArguments,
      issues: result.error.issues.map((issue) => {
        const path = issue.path.join(".") || "(root)";
        return `${path}: ${issue.message}`;
      }),
    };
  }

  return {
    status: "ok",
    callId: call.callId,
    name: call.name,
    args: result.data,
    warnings,
  };
}

This gives you a clean branching point.

If the result is invalid_json, the model failed to produce syntactically valid JSON.

If the result is invalid_schema, the model produced JSON but did not match the contract.

If the result is ok, you still are not done. You still need business validation and policy checks.

But at least now you are not guessing.

One subtle point: if the provider already returns parsed tool input instead of a JSON string, still validate it. Provider parsing does not mean your schema is satisfied.

6. Repair loops: ask the model to fix the call, but put a hard cap on it

When a tool call is malformed, one of the best recovery strategies is to tell the model what went wrong and ask it to try again.

This works surprisingly well for schema violations and mildly broken JSON.

But it needs boundaries.

A repair loop should:

  • Retry only a small number of times.
  • Include the tool name and validation errors.
  • Avoid sending huge raw payloads back if they are too large.
  • Stop if the model repeatedly fails.
  • Fall back to a user-facing clarification or safe failure.

A simplified version:

type RepairContext = {
  toolName: string;
  rawArguments: string;
  errors: string[];
  repairAttempt: number;
};

function buildRepairMessages(context: RepairContext): ChatMessage[] {
  const truncatedRaw =
    context.rawArguments.length > 2000
      ? `${context.rawArguments.slice(0, 2000)}...`
      : context.rawArguments;

  return [
    {
      role: "system",
      content: [
        "Your previous tool call was invalid.",
        "Return a corrected tool call that matches the required schema exactly.",
        "Do not invent missing information. If required information is unavailable, say what is missing.",
      ].join("\n"),
    },
    {
      role: "user",
      content: [
        `Tool name: ${context.toolName}`,
        `Previous raw arguments: ${truncatedRaw}`,
        `Validation errors:`,
        ...context.errors.map((error) => `- ${error}`),
      ].join("\n"),
    },
  ];
}

Then the loop:

async function executeToolWithRepair(
  initialCall: NormalizedToolCall,
  schema: z.ZodType<any>,
  executor: ToolExecutor,
  llm: LlmClient,
  options: { maxRepairs?: number } = {}
) {
  const maxRepairs = options.maxRepairs ?? 1;
  let currentCall = initialCall;
  let repairAttempt = 0;

  while (true) {
    const parsed = parseToolCall(currentCall, schema);

    if (parsed.status === "ok") {
      return executor.execute(parsed.name, parsed.args);
    }

    if (repairAttempt >= maxRepairs) {
      return {
        status: "failed",
        reason: parsed.status,
        errors:
          parsed.status === "invalid_schema"
            ? parsed.issues
            : [parsed.parseError],
      };
    }

    repairAttempt += 1;

    const repairMessages = buildRepairMessages({
      toolName: currentCall.name,
      rawArguments: currentCall.rawArguments,
      errors:
        parsed.status === "invalid_schema"
          ? parsed.issues
          : [parsed.parseError],
      repairAttempt,
    });

    const repairResponse = await llm.chat(repairMessages);

    const repairedCall = repairResponse.primaryToolCall;

    if (!repairedCall) {
      return {
        status: "failed",
        reason: "repair_failed",
        errors: ["Model did not return a repaired tool call"],
      };
    }

    currentCall = repairedCall;
  }
}

This is intentionally conservative. maxRepairs defaults to one. In many systems, one repair attempt is enough. More than that and you risk burning tokens while the model repeatedly produces the same broken shape.

There is also a protocol question: should you send the error back as a tool result?

In many tool-calling APIs, yes. If the model emitted a tool call with an ID, you can respond with a tool result marked as failed or containing validation errors. In Anthropic-style APIs, tool result blocks can indicate an error. In OpenAI-style APIs, tool messages can carry the error content for the corresponding tool call ID.

The exact mechanism varies, but the principle is the same: tell the model the tool call failed validation before execution, not after.

If the tool call is so malformed that there is no usable call ID, do not force a tool result. Retry the higher-level generation step or surface a user-facing error.

🧠 The important part: Repair loops are not magic. They are a controlled way to convert a recoverable formatting failure into a valid call. They should never become an open-ended negotiation with the model.

7. Valid JSON is not the same as a complete plan

This is where many agent systems still fail even after parsing and schema validation.

The tool call may be syntactically perfect and schema-valid, but still not represent a complete business action.

Example:

{
  "customer_id": "cus_9821",
  "currency": "USD",
  "line_items": []
}

If your schema accidentally allows an empty array, this passes.

But an invoice with no line items is not a complete invoice.

Another example:

{
  "date": "soon",
  "action": "cancel_subscription",
  "customer_id": "cus_9821"
}

If date is typed as string, the schema may accept it. The business cannot.

You need a second validation layer.

function validateCreateInvoiceBusinessRules(
  args: CreateInvoiceArgs
): string[] {
  const problems: string[] = [];

  if (args.line_items.length === 0) {
    problems.push("At least one line item is required.");
  }

  const total = args.line_items.reduce(
    (sum, item) => sum + item.quantity * item.price_cents,
    0
  );

  if (total <= 0) {
    problems.push("Invoice total must be greater than zero.");
  }

  if (args.currency === "USD" && total > 1_000_000_00) {
    problems.push("Invoices over $1,000,000 require manual review.");
  }

  return problems;
}

Then integrate it:

const parsed = parseToolCall(call, CreateInvoiceSchema);

if (parsed.status === "ok") {
  const businessErrors = validateCreateInvoiceBusinessRules(parsed.args);

  if (businessErrors.length > 0) {
    return {
      status: "business_invalid",
      callId: parsed.callId,
      errors: businessErrors,
    };
  }
}

This layer is also where you catch placeholders and vague values.

Watch for:

{
  "email": "TODO",
  "date": "ASAP",
  "reason": "user said they want the thing",
  "amount": null
}

Schema validation can reject null if you configure it, but it cannot know that "TODO" is a placeholder unless you add rules.

For high-risk tools, you may also want a policy layer:

function canExecuteTool(
  toolName: string,
  args: unknown,
  context: AgentContext
): { allowed: boolean; reason?: string } {
  if (toolName === "delete_customer_account" && !context.user.isAdmin) {
    return { allowed: false, reason: "Requires admin approval" };
  }

  return { allowed: true };
}

The sequence should be:

Schema validation
  ↓
Business validation
  ↓
Policy validation
  ↓
Execution

If any layer fails, the tool should not run.

8. Keep half-finished calls from becoming half-executed side effects

The truly dangerous case is not a crashed agent loop. It is a partially executed action.

This can happen in several ways.

Case 1: Streaming execution

You start executing tool arguments as they stream. The stream cuts off halfway. Now you have executed part of an action based on incomplete intent.

Do not do this.

Case 2: Multiple tool calls

The model returns several tool calls. One is valid. Another is truncated.

What do you do?

If the tools are independent and safe, you could execute the valid one and report the invalid one. But in many agent workflows, the calls are part of one plan. Executing only part of the plan can create inconsistent state.

A conservative default is:

If a batch of tool calls is meant to be atomic, validate all of them before executing any of them.

If they are independent, you can execute valid calls and return errors for invalid ones. But make that decision explicitly.

Case 3: Retry after timeout

The tool call was valid. You started executing it. The external API timed out. You do not know whether the action succeeded.

Now you need idempotency.

A good idempotency key can come from the tool call ID, the tool name, and a hash of the validated arguments:

import { createHash } from "node:crypto";

function buildIdempotencyKey(
  call: NormalizedToolCall,
  args: unknown
): string {
  const argsHash = createHash("sha256")
    .update(JSON.stringify(args), "utf8")
    .digest("hex");

  return `${call.callId}:${call.name}:${argsHash}`;
}

Then check before execution:

async function executeIdempotently(
  key: string,
  run: () => Promise<unknown>,
  store: IdempotencyStore
) {
  const existing = await store.get(key);

  if (existing) {
    return existing.result;
  }

  const result = await run();

  await store.put(key, {
    result,
    completedAt: new Date().toISOString(),
  });

  return result;
}

This does not solve every distributed systems problem. External APIs still need their own idempotency support. But it prevents your own retry logic from blindly repeating a tool execution.

For database mutations, wrap the tool execution in a transaction where possible.

For external side effects, prefer dry-run or preview modes if the external system supports them.

For dangerous operations, require human approval before execution.

A useful rule:

The more irreversible the tool, the more validation and approval it needs.

Reading a record can fail fast. Sending money needs a much stronger gate.

9. Store the corpse: observability for malformed tool calls

When a tool call fails, you need more than an error message. You need the evidence.

At minimum, log:

  • Provider name.
  • Model ID.
  • Request ID or trace ID.
  • Tool call ID.
  • Tool name.
  • Finish or stop reason.
  • Raw arguments, or a hash/redacted version.
  • Parse error.
  • Schema validation errors.
  • Business validation errors.
  • Repair attempts.
  • Final outcome.

A compact audit event might look like this:

type ToolCallFailureEvent = {
  eventType: "tool_call_failed";
  traceId: string;
  provider: string;
  modelId: string;
  callId: string;
  toolName: string;
  finishReason?: string;
  rawArgumentsHash: string;
  rawArgumentsRedacted?: string;
  failureType:
    | "transport_incomplete"
    | "invalid_json"
    | "invalid_schema"
    | "business_invalid"
    | "policy_rejected"
    | "execution_failed";
  errors: string[];
  repairAttempts: number;
  occurredAt: string;
};

Be careful with raw arguments. Tool calls can contain sensitive data: personal information, tokens, internal IDs, or customer content. If you store them, apply redaction, encryption, or retention limits.

A practical compromise is to store full raw arguments only for a short debugging window, and store hashes plus redacted summaries for long-term audit.

You should also track metrics:

llm_tool_call_total
llm_tool_call_invalid_json_total
llm_tool_call_schema_invalid_total
llm_tool_call_truncated_total
llm_tool_call_repair_attempts_total
llm_tool_call_execution_failed_total

If invalid tool calls suddenly spike, you may have:

  • Changed the tool schema.
  • Changed the system prompt.
  • Switched models.
  • Hit a provider-side regression.
  • Started passing too much context.
  • Introduced a tool with an ambiguous description.

Without metrics, you will discover these problems through user complaints.

🚨 Production warning: If you cannot reproduce a malformed tool call because you did not store the raw attempt, you are debugging an agent system blindfolded.

10. Design tools so the model has less room to fail

A lot of half-finished tool calls are not model failures. They are tool design failures.

If your tool expects a 20,000-character JSON object, the model will eventually truncate it.

If your tool expects ten nested arrays with ambiguous optional fields, the model will eventually get confused.

If your tool accepts free-text dates like "next Tuesday" instead of ISO dates, validation will become painful.

Good tool design reduces failure.

Prefer references over payloads

Bad:

{
  "name": "create_invoice",
  "arguments": {
    "customer": {
      "id": "cus_9821",
      "address": "...",
      "tax_info": "...",
      "billing_history": ["... lots of rows ..."]
    },
    "line_items": ["... lots of items ..."]
  }
}

Better:

{
  "name": "create_invoice",
  "arguments": {
    "customer_id": "cus_9821",
    "line_item_set_id": "lis_4471"
  }
}

Let the backend fetch the full objects.

Use enums where possible

Bad:

{
  "priority": "kind of high"
}

Better:

{
  "priority": "high"
}

With schema:

priority: z.enum(["low", "medium", "high", "critical"])

Split large operations

Instead of one tool that does everything:

{
  "name": "migrate_customer_workspace",
  "arguments": {
    "customer_id": "cus_9821",
    "migrate_data": true,
    "migrate_permissions": true,
    "notify_users": true,
    "update_billing": true
  }
}

Consider separate tools:

plan_customer_workspace_migration
start_customer_data_migration
migrate_customer_permissions
schedule_customer_notification
update_customer_billing

Smaller tools are easier to validate, easier to repair, and easier to audit.

Use constrained generation when available

Some providers and frameworks support schema-constrained generation, structured outputs, or JSON mode. Use them when available.

But do not remove validation.

Constrained generation reduces malformed output. It does not eliminate:

  • Token truncation.
  • Stream interruptions.
  • Business-rule violations.
  • Permission problems.
  • Provider bugs.
  • Timeout failures.

Validation remains mandatory.

Write better tool descriptions

This sounds soft, but it matters.

A tool description should say:

  • What the tool does.
  • When to use it.
  • When not to use it.
  • Which arguments are required.
  • What format dates and IDs use.
  • What the tool returns.
  • What errors mean.

For example:

create_invoice:
  Creates a draft invoice for an existing customer.
  Use this only after line items have been confirmed.
  customer_id must be an existing customer ID.
  currency must be one of USD, EUR, or GBP.
  line_items must contain at least one item.
  Returns invoice_id and preview_url.

The more precise the contract, the fewer weird tool calls you get.

What I would build

If I were designing this system today, I would not try to make the model perfect. I would assume tool calls will sometimes arrive broken and build the system around that assumption.

The core pipeline would look like this:

1. Normalize provider response into tool call envelopes.
2. Track stream finalization and finish reasons.
3. Reject transport-incomplete calls.
4. Parse arguments.
5. Validate with schema.
6. Run business rules.
7. Run policy checks.
8. Attempt bounded repair for parse/schema failures.
9. Ask clarification questions for semantic incompleteness.
10. Execute only validated calls.
11. Use idempotency keys for all side-effecting tools.
12. Log every failure with enough detail to reproduce it.

For recovery behavior, I would use this decision table:

SituationResponse
Stream disconnected before final responseRetry generation if safe; do not execute
Finish reason indicates length limitReduce payload, split tool, or repair with smaller schema
Invalid JSONOne bounded repair attempt, then fail safely
Schema violationReturn validation errors to model for repair
Valid schema but missing business intentAsk user or upstream agent for clarification
Dangerous toolRequire approval or dry-run before execution
Tool already partially executedUse idempotency and reconciliation, not blind retry
Repeated repair failuresStop and surface a deterministic error

The final architectural point is this: an LLM tool call is not the end of reasoning. It is the beginning of validation.

Half-finished tool calls are not a rare provider bug. They are a normal class of output from a probabilistic system operating under token limits, network instability, and imperfect schema adherence.

Your agent system becomes reliable not when the model stops producing broken tool calls, but when broken tool calls stop being able to hurt you.

Tags:

Post a Comment

Previous Post Next Post