Free Ai API

Stateless MCP Servers in Production: Replacing Session State with Durable Handles

byHossein Hezami •
0

Your MCP server passes every local test. Then you put two replicas behind a load balancer.

start_report lands on replica A, which stashes the job in a Map. get_report lands on replica B, which has never heard of that job. The model receives Error: unknown job id, retries three times, and eventually tells the user the reporting service is broken.

A rolling deploy is worse. The pod holding half-finished work disappears mid-conversation. Every handle the client is holding becomes a dangling reference.

This isn't an MCP problem. It's a statefulness problem wearing an MCP costume. And the fix isn't sticky sessions or a bigger Redis cluster. It's replacing implicit, process-local session state with explicit, durable handles that any replica can resolve.

TL;DR: MCP permits genuinely stateless HTTP servers. Build them by creating a fresh protocol server per request, persisting application state in Postgres or object storage, and returning opaque, signed, tenant-bound handles from tools. Model long work as start / get / cancel. Make every mutating tool idempotent. Keep exactly one component stateful if you need push notifications—and make that component disposable.

📋 Table of Contents

1. Two Kinds of State, One Very Confusing Failure

There are two separate things people mean when they say an MCP server "has a session," and conflating them causes most of the pain.

Protocol session state. With Streamable HTTP—the transport introduced in the 2025-03-26 protocol revision—a server may return an Mcp-Session-Id header during initialize. If it does, the client must echo that header on subsequent requests. The server uses it to correlate requests and, often, to locate an in-memory connection object.

Crucially, the spec does not require this. If the server omits Mcp-Session-Id during initialization, the client must treat the server as stateless and must not attempt to establish a session. Stateless operation is a first-class mode, not a hack.

Application state. Your report job. The draft document the agent is editing. The pagination cursor. The uploaded file waiting to be indexed. The half-completed multi-step workflow.

Here's the trap: you can faithfully implement the stateless protocol mode and still build a completely stateful application. Omitting Mcp-Session-Id while keeping a module-level Map<jobId, Job> buys you nothing. The load balancer still routes get_report to a replica that doesn't have the map entry.

Conversely, sticky sessions don't save you from a deploy. They only delay the failure until the next rollout.

The mental model I'd use:

MCP client
    │  POST /mcp  (any replica can serve this)
    ▼
┌──────────────────────────────┐
│  Stateless HTTP edge         │  ← no Maps, no per-client objects
│  authenticate → validate     │
│  → resolve handle → respond  │
└──────────┬───────────────────┘
           │
     ┌─────┴──────┐
     ▼            ▼
  Postgres     Object storage
  (jobs,       (artifacts,
   results,     large payloads)
   outbox)
     ▲
     │
  Workers (also stateless; lease jobs from the DB)

Statelessness means no affinity between a request and a process. It does not mean the system forgets things. The state moves out of the request handler and into storage that outlives it.

2. Stateless Transport Means Per-Request Servers

Even after you externalize your application state, there's a subtler statefulness problem: the MCP server object itself is stateful.

The official TypeScript SDK's McpServer tracks negotiation state, registered capabilities, and the transport it's connected to. Share one instance across concurrent HTTP requests and you've built a cross-client state leak disguised as a performance optimization.

The correct stateless pattern constructs both the server and the transport per request:

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";

function buildServer() {
  const server = new McpServer({ name: "reports", version: "1.4.0" });
  registerReportTools(server); // handlers read Postgres, never a module-level Map
  return server;
}

app.post("/mcp", async (req, res) => {
  const server = buildServer();

  const transport = new StreamableHTTPServerTransport({
    sessionIdGenerator: undefined, // do not issue Mcp-Session-Id
    enableJsonResponse: true,      // reply with application/json, not an SSE stream
  });

  res.on("close", () => {
    void transport.close();
    void server.close();
  });

  await server.connect(transport);
  await transport.handleRequest(req, res, req.body);
});

Two options do the heavy lifting. sessionIdGenerator: undefined tells the transport not to mint session identifiers. enableJsonResponse: true makes a POST return a single JSON response rather than opening an event stream that must stay attached to this process.

That second option is the one people miss. You can omit the session ID and still be stateful if every request opens an SSE stream pinned to one replica.

If you don't implement server-initiated streaming, implement the GET endpoint as an explicit 405 Method Not Allowed. Don't return an empty 200. Clients use that response to learn that push isn't available, and a silent 200 tends to produce hangs that look like network problems.

⚠️ Gotcha: Per-request construction feels wasteful, and people "optimize" it by pooling server instances. Don't. The construction cost is microseconds compared to the LLM call that triggered the tool. Correctness here is cheap; debugging cross-session contamination is not.

3. The Handle Is a Capability, Not a UUID

A handle is the token the client carries between calls. Its design determines your security model, your revocation story, and your operational burden.

Three designs are common:

A raw UUID (job_7f3a…) is easy to issue and easy to look up, but it's an identifier, not an authorization artifact. Every lookup must independently verify ownership, and guessing becomes a meaningful attack if your IDs are sequential.

A self-contained signed token encodes tenant, resource, scope, and expiry. Verification needs no database round trip. But you cannot revoke it before it expires, and it grows awkward as you add fields.

A hybrid—a signed envelope wrapping a reference, with a small database row for status and revocation—is what I'd ship. Signature verification rejects garbage cheaply; the row lookup enforces current authorization and lifecycle state.

Here's a compact implementation using Node's built-in crypto:

import { createHmac, timingSafeEqual } from "node:crypto";

const KEY = Buffer.from(process.env.HANDLE_KEY!, "hex"); // 32 random bytes

const sign = (data: string) =>
  createHmac("sha256", KEY).update(data).digest("base64url");

export function issueHandle(tenantId: string, jobId: string, ttlMs: number) {
  const payload = Buffer.from(JSON.stringify({
    v: 1,
    tid: tenantId,
    jid: jobId,
    exp: Math.floor((Date.now() + ttlMs) / 1000),
  })).toString("base64url");

  return `h1.${payload}.${sign(payload)}`;
}

export function readHandle(token: string, callerTenant: string) {
  const [version, payload, signature] = token.split(".");
  if (version !== "h1" || !payload || !signature) throw new Error("invalid_handle");

  const a = Buffer.from(signature);
  const b = Buffer.from(sign(payload));
  if (a.length !== b.length || !timingSafeEqual(a, b)) {
    throw new Error("invalid_handle");
  }

  const claims = JSON.parse(Buffer.from(payload, "base64url").toString());
  if (claims.exp < Math.floor(Date.now() / 1000)) throw new Error("handle_expired");
  if (claims.tid !== callerTenant) throw new Error("forbidden");

  return { jobId: claims.jid };
}

The non-obvious part is the last authorization check. A valid signature proves the server issued the handle. It does not prove the current caller may use it.

This matters more in MCP than in a normal API. Handles appear in tool results, which land in the model's context. A model can copy a handle from one conversation into another tool call. A document your agent ingests might contain someone else's handle. If you validate only the signature, you've built a confused-deputy vulnerability.

Always derive callerTenant from the authenticated transport request—never from a tool argument, never from the handle alone, and definitely never from something the model typed.

4. Turn Long-Running Tools Into Start, Get, and Cancel

The natural MCP tool shape is request/response: call generate_report, wait, get a report. That works until the report takes 90 seconds, an intermediary times out at 30, and the client retries into a second expensive job.

Replace the blocking tool with three small ones:

ToolSemanticsCost
start_reportEnqueue work, return a handleOne DB insert
get_reportResolve handle, return status or resultOne DB read
cancel_reportMark a live job cancelledOne conditional update

Every one of these is fast enough to serve from a stateless edge.

Persist the job before you acknowledge anything:

CREATE TABLE mcp_job (
  id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  tenant_id   text NOT NULL,
  tool_name   text NOT NULL,
  idem_key    text,
  input_hash  text NOT NULL,
  status      text NOT NULL DEFAULT 'queued'
              CHECK (status IN ('queued','running','succeeded','failed','cancelled')),
  attempt     int  NOT NULL DEFAULT 0,
  lease_until timestamptz,
  result_ref  text,
  error_code  text,
  created_at  timestamptz NOT NULL DEFAULT now(),
  finished_at timestamptz,
  expires_at  timestamptz NOT NULL DEFAULT now() + interval '24 hours',
  UNIQUE (tenant_id, tool_name, idem_key)
);

Workers claim work with FOR UPDATE SKIP LOCKED, set a lease, and execute. If a worker dies, the lease expires and another worker picks the job up. The MCP edge never needs to know which worker owns anything.

The tool registration is where a lot of implementations go wrong:

server.registerTool("start_report", {
  description: "Queue a report. Poll get_report with the returned handle.",
  inputSchema: {
    customer_id: z.string().uuid(),
    idempotency_key: z.string().min(8),
  },
  outputSchema: {
    handle: z.string(),
    status: z.literal("queued"),
    poll_after_ms: z.number(),
  },
  annotations: {
    readOnlyHint: false,
    destructiveHint: false,
    idempotentHint: true,
    openWorldHint: false,
  },
}, async (args) => {
  const job = await enqueue(callerTenant, args);
  return {
    content: [{ type: "text", text: "Report queued. Use get_report to check progress." }],
    structuredContent: {
      handle: issueHandle(callerTenant, job.id, 24 * 60 * 60 * 1000),
      status: "queued",
      poll_after_ms: 2000,
    },
  };
});

Notice outputSchema and structuredContent. These arrived with the 2025-06-18 revision's structured tool output, and they're the difference between a reliable handle and a fragile one.

Put the handle in structured content, not buried in a prose sentence. Models copy opaque 64-character strings out of natural language imperfectly. A structured field is something the client can store and pass back verbatim.

The poll_after_ms hint is equally important. Without it, an eager agent will poll every second and turn your durable queue into a self-inflicted DDoS.

5. Idempotency Keys Belong in the Tool Contract

MCP clients retry. Networks drop. Models re-issue tool calls when a response looks suspicious. If start_report isn't idempotent, a single interrupted request becomes two reports, two invoices, or two emails to a customer.

Make the idempotency key an explicit part of the input schema, then enforce it with a unique constraint:

const inputHash = createHash("sha256")
  .update(stableStringify(args))
  .digest("hex");

const [inserted] = await sql`
  INSERT INTO mcp_job (tenant_id, tool_name, idem_key, input_hash)
  VALUES (${tenantId}, ${toolName}, ${args.idempotency_key ?? null}, ${inputHash})
  ON CONFLICT (tenant_id, tool_name, idem_key) DO NOTHING
  RETURNING id, input_hash
`;

if (inserted) return inserted;

const [prior] = await sql`
  SELECT id, input_hash FROM mcp_job
  WHERE tenant_id = ${tenantId}
    AND tool_name = ${toolName}
    AND idem_key = ${args.idempotency_key}
`;

if (!prior) throw new Error("concurrent_retry"); // deleted or racing; surface it
if (prior.input_hash !== inputHash) {
  throw new Error("idempotency_key_reused_with_different_arguments");
}
return prior;

Two details carry the weight here.

First, the uniqueness scope includes the tenant. A global idempotency namespace would let one customer's key collide with another's—either leaking existence or silently returning the wrong job.

Second, the input hash check. Reusing a key with different arguments is a client bug, and returning the old result silently is how you get a report for the wrong customer. Fail loudly with a distinguishable error instead.

💡 Practical note: In Postgres, NULL values don't conflict in a unique index. Leaving idem_key nullable therefore lets requests without a key through normally. That's convenient during development and dangerous in production—make the key required for any tool that mutates anything.

Also resist the urge to generate the idempotency key inside your handler from a timestamp. That produces a different key on every retry, which is exactly the behavior you were trying to prevent. The key must originate from the caller's logical operation and survive the retry.

6. Pagination Cursors That Survive a Redeploy

list_invoices returning 20 rows and a next_cursor looks stateless. Until your cursor is "page_7" backed by a server-side cache entry.

The durable version is a keyset cursor: an opaque, signed encoding of the last sort key and last primary key, bound to the query that produced it.

// Encode: signed payload containing { tid, q, ts, id }
// q = fingerprint of the normalized filter set
// ts/id = last row's sort tuple

const rows = await sql`
  SELECT id, created_at, subject
  FROM invoice
  WHERE tenant_id = ${tenantId}
    AND (created_at, id) < (${cursor.ts}::timestamptz, ${cursor.id}::uuid)
  ORDER BY created_at DESC, id DESC
  LIMIT ${pageSize}
`;

Keyset pagination beats offset pagination here for a reason that has nothing to do with state: inserts and deletes don't shift the window. An agent paging through a live invoice table while new invoices arrive will neither skip nor duplicate rows.

The q fingerprint is the security-relevant part. Without it, a cursor issued for list_invoices(status=paid) can be replayed against list_invoices(status=all), or worse, against a different tool entirely. Binding the cursor to the normalized query makes it a capability for one specific traversal.

Pair it with a composite index:

CREATE INDEX invoice_tenant_created_id
  ON invoice (tenant_id, created_at DESC, id DESC);

When the cursor is expired or malformed, return a structured error that tells the model what to do: cursor_expired, restart_from_beginning. Don't silently fall back to page one. An agent that silently restarts pagination will process the same records twice, and if those records trigger side effects, you've built a duplicate-action bug into your retrieval path.

7. Versioned Writes Fix the Multi-Step Edit Problem

Read-only tools are easy to make stateless. Editing is where people give up and reach for sessions.

Consider an agent that reads a draft, changes one paragraph, and writes it back. Between the read and the write, a human edits the same draft. Without a concurrency check, the agent silently overwrites the human's work.

The session-based "solution" keeps the draft open on a replica. The durable solution is a versioned handle:

const [updated] = await sql`
  UPDATE draft
  SET body = ${args.body}, version = version + 1
  WHERE id = ${handle.draftId}
    AND tenant_id = ${tenantId}
    AND version = ${args.base_version}
  RETURNING id, version
`;

if (!updated) {
  return {
    isError: true,
    content: [{
      type: "text",
      text: "The draft changed since you read it. Read it again before editing.",
    }],
    structuredContent: { code: "version_conflict" },
  };
}

The handle returned by read_draft carries the version. The write tool requires base_version. The conditional update is atomic, so it doesn't matter which replica executes it or how many replicas are involved.

This is the same contract as HTTP ETags, and it deserves the same treatment: a conflict is an expected outcome, not a server fault.

That distinction maps onto MCP's error model. Tool-level, recoverable conditions—version conflict, job still running, cursor expired—belong in a successful JSON-RPC result with isError: true and machine-readable structured content. The model can read that and adapt. Transport and authentication failures belong in HTTP status codes and JSON-RPC errors.

Returning a JSON-RPC protocol error for a benign conflict tends to make clients abort the whole turn instead of letting the agent retry thoughtfully.

8. The State You Cannot Delete: Notifications and Elicitation

Here's the honest limit of the stateless design.

Streamable HTTP lets a client open a long-lived GET stream to receive server-initiated messages. Progress notifications, resource updates, elicitation requests, sampling requests—all of these need a live channel from server to client.

A POST-only server that returns application/json has no such channel. It cannot push.

You have three options, and they're genuinely different trade-offs rather than varying quality implementations.

Poll. The client calls get_report on an interval. This is the simplest design, it survives any number of restarts, and for jobs measured in seconds or minutes it's perfectly adequate. Return poll_after_ms and respect it.

Keep a stateful notification gateway. Workers remain stateless and write progress events to an outbox table. A small gateway component owns the SSE connections and drains the outbox. This is the only stateful component in the system, and it holds only connection state—no jobs, no results, no authority. If it crashes, clients reconnect and resume from a durable event ID.

Model the interaction explicitly. Elicitation is the interesting case. The 2025-06-18 revision lets a server ask the client for additional input mid-operation. That fundamentally needs a live channel. In a stateless design, you can instead return a structured needs_input result carrying a handle and a description of what's missing. The agent asks the user, then calls resume_report with the answer and the handle.

That third option is more than a workaround. It turns an implicit, connection-bound interaction into an explicit, durable state transition—which is exactly what you want if the user might take ten minutes to answer.

🚨 Production warning: Don't try to make elicitation work by stuffing interaction state into a Redis key keyed by session ID and hoping the next request hits the same replica. You've recreated the original bug with extra steps and a network dependency.

9. Expiry, Revocation, and the Garbage You Leave Behind

Durable handles that never expire are a slow-moving security incident. Durable results that never get deleted are a slow-moving bill.

Give each handle class its own lifetime:

Handle typeTypical TTLRationale
Pagination cursorMinutesCheap to recreate; bound to a snapshot
Job handleJob completion + retentionClient may poll long after starting
Artifact handleMatches object lifecycleMust not outlive the object it references
Cancellation / resume handleUntil terminal job stateUseless once the job is done

Two rules prevent most of the operational pain.

Never let a handle expire before the data it references. A cursor that outlives its result set produces confusing "not found" errors that look like authorization bugs.

Never let a handle expire inside the client's retry window. If your job handle lives 60 seconds and the client's backoff can stretch past that, a perfectly healthy retry turns into an unrecoverable error.

Revocation is where the hybrid design earns its keep. A purely self-contained token cannot be revoked. With a reference row, revocation is a single update:

UPDATE mcp_job SET revoked_at = now() WHERE id = ${jobId} AND tenant_id = ${tenantId};

readHandle still validates the signature cheaply, then the row lookup checks revoked_at. You get fast rejection of garbage and immediate revocation of legitimate-but-no-longer-authorized handles.

Run a reaper that deletes expired jobs and their artifacts on a schedule. Keep a small tombstone window after deletion so a late retry returns a clear handle_expired rather than an ambiguous not_found. That distinction is the difference between "the model should stop" and "the model should retry forever."

10. Chaos Tests That Prove Your Server Is Stateless

"Stateless" is a claim about behavior under failure, not about whether you used a Map. Test the claim.

Build a small integration suite that runs against two real replicas behind a real load balancer, driven by a real MCP client—not curl. The protocol handshake has enough edge cases that hand-rolled HTTP tests miss the interesting failures.

InjectionExpected behavior
Route start and get to different replicasSame handle resolves; same result
Kill the replica that issued a handleA different replica resolves it
Redeploy while a job is runningLease expires; another worker completes the job
Replay an identical tools/call with the same idempotency keyOne job, one handle
Same idempotency key, different argumentsExplicit idempotency_key_reused error
Present tenant A's handle with tenant B's credentialsForbidden, regardless of signature validity
Replay a cursor against a different queryRejected by the query fingerprint
Client reconnects with no prior initialize on this replicaHandles still resolve
Worker crashes after writing results but before marking completeJob is retried or failed—never silently lost

That last one is the test most teams skip and most teams regret. The boundary between "work completed" and "state recorded as completed" is where distributed systems earn their reputation. Write the result and the terminal status in the same transaction, or make the result write idempotent so a retry is harmless.

Also watch your correlation strategy. In a stateful server, the session ID is a convenient log correlation key. In a stateless server, it doesn't exist. Use the handle as your correlation spine: MCP request → job row → worker attempt → result. Log the handle reference and tenant, not the full signed token.

Choosing the Right Architecture

Four approaches are commonly proposed for this problem. They solve different problems.

ApproachSurvives deployScales horizontallyOperational costBest fit
In-process session state❌❌Near zeroLocal dev, single-user CLI tools
Sticky sessions❌PartiallyLowDemo environments; never a production boundary
Shared session store (Redis)✅✅MediumProtocol-level session metadata, short-lived interaction state
Durable handles + external state✅✅Medium–highProduction MCP servers, async work, multi-tenant

The shared-session-store option deserves a fair reading. If your state is genuinely small, short-lived, and reconstructible—negotiated capabilities, a transient request map—a Redis-backed session store is a reasonable engineering choice. It's not automatically wrong.

Where it breaks down is when the session store becomes the only copy of something valuable. A report job that exists only as a Redis hash with a 30-minute TTL is not a durable job. It's a cache pretending to be a queue.

For a new production MCP server, I'd start with durable handles and a Postgres-backed job table. Add Redis only when profiling shows a real hot path that needs it. Add a notification gateway only when polling proves inadequate for your latency requirements.

And if you're building a genuinely interactive experience—live collaborative editing, real-time log tailing, a streaming terminal—don't contort it into a POST-only stateless shape. Keep a stateful connection layer. Just make sure the layer holds connections and nothing else, so losing it costs a reconnect rather than a conversation.

The architectural insight worth keeping is this: the handle is your session. Once you design it deliberately—as a signed, tenant-bound, versioned, expiring capability with a defined lifecycle—you discover that most of what you thought required a session was really just a reference that needed to outlive a request.

Tags:

Post a Comment

Previous Post Next Post