Open a trace from last Tuesday's slow support assistant request and you'll find the whole conversation: the customer's message, the four help-center articles the retriever pulled, the account summary injected as context, and the model's reply. If the customer pasted a medical note into the chat, that's in there too. So is their email address, their order number, and whatever the model said about their refund.
That trace was almost certainly created to answer an engineering question: why did this take nine seconds? It now answers a different question — what do we hold about this person, where does it live, and who can read it — and most teams have no good answer for the second one.
This is the specific problem of AI observability: the payload is the sensitive data. An HTTP access log records that a request happened. A database query log records a statement, usually with parameters elided. An LLM trace records the content itself, because the content is what you need in order to understand quality. The thing that makes the trace useful is the thing that makes it dangerous.
Why AI traces behave like a second copy of your customer database
Four properties separate AI telemetry from conventional application telemetry.
The payload is unbounded. A request body in a well-designed API has a schema and usually a size limit. A prompt can contain an entire document, a pasted email thread, a CSV of transactions, or a screenshot transcribed by an OCR step. There is no field-level contract that tells you what's inside.
The payload mixes trust levels. A single trace contains your system instructions, retrieved private documents, user input, and tool results from third-party systems. Each has a different owner, a different retention obligation, and a different blast radius if leaked. In a normal log line, you can classify the whole record. Here you can't.
It's high-cardinality and high-volume. Traces are per-request and often per-step. A ten-step agent run with retrieval and three tool calls can produce thirty spans, each with content. Retaining all of it for a year is a storage decision that quietly becomes a compliance decision.
It's replicated by default. Trace backends ship to a vendor, replicate across regions, and feed secondary indexes for search. Deleting a record is not a single DELETE. If your trace store has no per-tenant key management, you cannot honestly promise erasure.
🚨 Production warning: if your AI traces are in a separate SaaS observability tool from your application database, that tool is a new subprocessor holding personal data. It needs to be in your DPA, your records of processing, and your retention schedule. Most teams add it during a sprint and document it never.
Take inventory before you configure anything
You can't redact what you haven't enumerated. The inventory is short and specific: for each span type your instrumentation emits, list the fields that can contain content.
| Span or event | Fields that carry content | Typical sensitivity |
|---|---|---|
| Inference | system instructions, input messages, output messages | Mixed: policy + personal data |
| Retrieval | query text, returned document text, document IDs | Personal data, sometimes regulated |
| Tool execution | arguments, results, error strings | Depends entirely on the tool |
| Memory operations | records written or read | Long-lived personal data |
| Embeddings | input text | Personal data, often overlooked |
| Client-side events | rendered text, user feedback, screenshots | Highest risk, lowest governance |
That last row is the one people forget. A session-replay tool that captures the DOM will record every AI response on screen, plus whatever the user typed. If your replay tool has different retention and access rules than your trace store, you've created a bypass.
Multimodal inputs deserve their own line in the inventory, because they break the assumptions the rest of the pipeline makes. A base64 image or an audio clip in a trace is content you cannot regex, cannot tokenize, and cannot classify without running another model over it. It's also much larger, so it dominates storage and query cost. The defensible default is to store a reference — object key, hash, duration, dimensions — and keep the bytes in the same governed store as other user uploads, subject to the same retention. If you must retain the bytes in telemetry, treat the trace backend as a file store with all the obligations that implies, including malware scanning and legal hold. Most teams are better off not doing that.
The OpenTelemetry GenAI semantic conventions are helpful here because they make the distinction explicit. Content-bearing attributes — gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, gen_ai.retrieval.query.text, and the memory record attributes — are marked opt-in, and the spec says instrumentations "SHOULD NOT capture them by default." Several SDKs gate this behind an environment variable such as OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT. That default is the correct one, and teams turn it on permanently in production because debugging got easier the week they did.
The conventions themselves are still at development status, so pin your instrumentation versions and expect attribute names to move. Don't build a compliance control that depends on an experimental attribute name surviving a minor upgrade.
Metadata by default, content by exception
The single most effective policy is a split: metadata is always on, content is captured only under conditions you can articulate.
Always-on metadata answers most operational questions without touching personal data:
- model, provider, and snapshot
- prompt version hash and template ID
- token counts, including cached and reasoning tokens
- latency, split into queue, retrieval, time-to-first-token, and generation
- tool name, duration, status code, and a redacted argument digest
- retrieval: index name, document IDs, scores, count
- outcome: accepted, rejected, escalated, errored
- tenant, workspace, and a pseudonymous user reference
Conditional content covers the cases metadata can't: a specific complaint, a quality regression, a safety flag. Capture it for a bounded window, with a reason attached, and let it expire automatically.
That structure gives you a defensible answer to "why do you have this data?" — you have it because a named process needed it for a defined period. Compare that to "because the SDK default was on," which is what most teams would have to say today.
💡 Practical note: store the prompt template and its hash, not the rendered prompt, as your default artifact. Templates have no personal data in them. When you need to reconstruct a specific prompt, you need the inputs, and those should live under tighter rules than the template.
Redact before the data leaves the process
There are three places to put a redaction step: in the process that creates the span, in the collector that receives it, and in the storage layer that indexes it. They have different failure modes.
| Location | Advantage | Failure mode |
|---|---|---|
| In-process span processor | Data never leaves the app; per-language control | Must be implemented in every SDK; easy to forget in a new service |
| Collector pipeline | One policy for all languages; easy to change | Raw content crosses the network and exists in collector memory |
| Storage/index pipeline | Can be updated without redeploying apps | Content is already persisted, and often replicated |
The strongest posture is in-process redaction for the highest-risk fields, with a collector policy as a second layer that catches new services and misconfigured SDKs. Defense in depth here isn't paranoia; it's the recognition that instrumentation is copied from service to service by people in a hurry.
A redacting span processor in Python, using the OpenTelemetry SDK's SpanProcessor interface:
import hashlib
import hmac
import re
from opentelemetry.sdk.trace import SpanProcessor
SENSITIVE = re.compile(
r"(?i)\b([\w.+-]+@[\w-]+\.[\w.]+)\b" # email
r"|\b(?:\d[ -]*?){13,19}\b" # card-like digit runs
r"|\b\d{3}-\d{2}-\d{4}\b" # national ID shaped
)
CONTENT_ATTRS = {
"gen_ai.input.messages",
"gen_ai.output.messages",
"gen_ai.system_instructions",
"gen_ai.retrieval.query.text",
}
class RedactingSpanProcessor(SpanProcessor):
def __init__(self, hmac_key: bytes, capture_content: bool = False):
self._key = hmac_key
self._capture = capture_content
def _token(self, value: str) -> str:
digest = hmac.new(self._key, value.encode("utf-8"), hashlib.sha256)
return f"tok_{digest.hexdigest()[:16]}"
def on_end(self, span):
attrs = dict(span.attributes or {})
for name, value in attrs.items():
if name in CONTENT_ATTRS and not self._capture:
span.set_attribute(name, "[not captured]")
continue
if isinstance(value, str) and SENSITIVE.search(value):
span.set_attribute(name, SENSITIVE.sub(lambda m: self._token(m.group(0)), value))
def on_start(self, span, parent_context=None):
pass
def shutdown(self):
pass
def force_flush(self, timeout_millis=None):
return True
A few things about that snippet are deliberate.
It runs in on_end, not on_start. Start-time attributes are used for sampling decisions, and content shouldn't influence sampling anyway. Doing the work at end means you see the final attribute set, including anything the instrumentation added after the call returned.
It replaces rather than deletes. A missing attribute and a redacted attribute mean different things during an incident. [not captured] tells you the pipeline worked as designed; an absent field tells you nothing.
It uses HMAC, not a bare hash. For correlation — "is this the same email across two traces?" — you want a deterministic token. For that token to be safe, the key must be secret, because a plain SHA-256 of an email address is trivially reversible with a wordlist. This is the mistake almost everyone makes: hashing a low-entropy identifier and calling it pseudonymized when it's actually just encoded.
It's pattern-based, which is the weakest form of detection. Regexes miss names, addresses, free-text medical details, and anything in a non-Latin script. They also produce false positives that mangle legitimate content. Treat them as a floor, not a ceiling, and layer a proper DLP classifier or a dedicated PII detection library in the collector where you can afford the compute.
Pseudonymize with referential integrity, or don't bother
Redaction that breaks joins is redaction engineers route around. If the trace says [redacted] and the database says customer_id 88123, an on-call engineer can't connect the complaint to the trace, and they'll ask for content capture to be turned back on.
The workable pattern is a tokenization service: the application replaces the sensitive value with a stable token before it goes anywhere near telemetry, and a separate, small, tightly controlled mapping table resolves tokens back to values. Same token for the same value within a tenant, different token across tenants, so cross-tenant correlation is impossible by construction.
trace attribute: user.ref = "u_9f2c41ab7e"
resolve (break-glass, audited):
SELECT subject_id FROM token_map
WHERE token = 'u_9f2c41ab7e' AND tenant_id = $1;
Note what the mapping table gives you that a hash doesn't: revocation. Delete the row and the token becomes permanently unresolvable, even though the trace still exists. That's how you handle an erasure request in a system whose trace backend has no delete API.
Be careful with encryption as a substitute for minimization
Field-level encryption of trace content sounds like the answer. Encrypt everything at the collector, decrypt only for authorized readers. In practice it creates two problems.
First, encrypted content can't be searched, filtered, or evaluated. Many of the reasons you keep traces are analytical: find all requests where the model mentioned a competitor, sample conversations with a specific complaint. If the content is opaque to the pipeline, you need a separate plaintext index, which defeats the purpose.
Second, key management multiplies. Per-tenant keys are the only version that supports clean deletion (crypto-shredding: destroy the key, and every ciphertext becomes noise), but now you have a key store with its own availability requirements in the critical path of telemetry. During an incident, a KMS outage shouldn't take down your ability to observe the incident.
Where encryption earns its place is the narrow case: a small set of high-value content you must retain for a defined reason, accessed rarely, with per-tenant keys and an audit log on every decrypt. That's a different product from "the trace store."
Sampling is a privacy control, not just a cost control
Everyone thinks of sampling as a billing lever. It's also a data-minimization lever, and thinking about it that way changes the design.
Head sampling decides at trace start, based on a ratio and a few stable attributes. It's cheap and consistent, and it gives you a uniform random sample — which is exactly what you want for latency percentiles. It also means you keep 100% of the traces you'd most like to delete, because a random 5% includes someone's medical details.
Tail sampling decides after the trace completes, based on outcome: keep errors, keep slow traces, keep flagged ones. Operationally this is far more useful, because the interesting traces are rare. From a privacy standpoint it's more dangerous, because it preferentially retains the traces involving failures, and failures are where users paste the most context.
The compromise I'd recommend: head-sample at a low rate for baseline metrics, tail-sample for errors with content replaced rather than captured, and reserve full content capture for an explicit, time-boxed debug session scoped to a single request or user, with a record of who turned it on.
That last mechanism needs a name in your audit log. "Who enabled content capture, for which trace, for how long, and why" is the question a privacy reviewer will ask.
Retention needs to be per-purpose, not per-system
A single retention number for a trace store is almost always wrong, because the store holds different things.
| Data | Purpose | Reasonable default |
|---|---|---|
| Span metadata (tokens, latency, model) | Performance, cost, capacity | 30–90 days hot, aggregates longer |
| Redacted content | Quality debugging | 7–30 days |
| Full content (captured) | Specific investigation | Hours to days, auto-expiring |
| Safety flags and blocked outputs | Abuse review, regulatory | Per policy, often 1 year+ |
| Evaluation samples | Product quality | De-identified, longer |
| Token mapping table | Support lookup | As short as operationally possible |
Two operational details make retention real instead of aspirational.
Expire at the storage layer. A policy document that says thirty days while the backend keeps data for a year is a finding waiting to happen. Configure TTLs in the index, verify them by querying the oldest record, and put that check on a schedule.
Separate the stores. If full content and metadata live in the same index, you can't expire one without the other, and you can't restrict access to one without restricting both. Two datasets, two retention policies, two access roles.
Access control and the break-glass path
The trace UI is usually readable by every engineer in the company. That's fine for latency data and not fine for conversation content.
A workable arrangement: metadata in the general observability tool with normal SSO access; content in a separate project reachable only by a named group; a break-glass flow that grants temporary access with a reason, notifies the data owner, and writes an audit record. The friction is the point — not to stop debugging, but to make reading someone's conversation a deliberate act rather than a scroll.
One more access question people skip: what does your vendor's support staff see? If a trace contains content and the vendor's on-call can open it to debug an ingestion issue, that's a disclosure path you should know about and disclose. Ask for the vendor's own access controls, not just their SOC 2 report.
Deletion, erasure, and the honest limits
When a user exercises a right to erasure, your trace store is one of the places where their data lives, and it's usually the hardest to clean. Three patterns, in increasing order of effort:
- Tokenized references only. Trace content never contains direct identifiers, so erasure of the trace is not required — the mapping row is deleted and the trace becomes unlinkable. This is the only approach that scales.
- Per-tenant keys. Destroy the key for the tenant; all ciphertext becomes unrecoverable. Works for tenant-level deletion, not individual-level.
- Targeted deletion. Query by token, delete matching documents, and verify. Feasible for a few requests, not for a year of history across replicas and backups.
If you're starting today, design for option one. It costs a tokenization step and saves a category of impossible requests later. And be honest in your privacy documentation about backups: most trace backends cannot selectively delete from immutable storage, so state a bounded window after which backups roll off rather than promising instantaneous erasure you can't deliver.
Your evaluation dataset is the leak you keep
Trace stores expire. Evaluation datasets don't. This is the most persistent sensitive-data problem in AI engineering, and it's almost always created by a well-intentioned person trying to improve quality.
The sequence is predictable. Someone notices the assistant handles refunds poorly. They pull fifty real conversations from the trace store, paste them into a JSON file, add expected answers, and commit it to the repo. The eval runs in CI. It runs against a hosted model. The file gets shared in a Slack thread, copied into a notebook, and eventually uploaded to a prompt-engineering tool. Six months later, nobody remembers which of those copies exists or who can read them.
Now consider what's in those fifty conversations: real names, real order numbers, a few account balances, one person describing a medical situation.
Three practices keep evaluation useful without turning it into a shadow archive.
Synthesize from structure, not from text. If your decision records are structured, you can generate realistic test cases by sampling the structure and replacing the values. A case becomes {orderAgeDays: 94, hasReceipt: false, category: "final_sale"}, not a transcript. This covers most classification and policy behavior, which is where most regressions live.
Redact at extraction, not at review. Build a single script that pulls conversations and applies the same tokenization used in telemetry, then writes the eval file. If extraction is a manual copy-paste step, redaction will be skipped under deadline pressure.
Keep goldens in the same governance boundary as production data. If an eval file contains anything derived from real users, it needs an owner, a retention period, and access control. A JSON file in a repository has none of those, and git log makes deletion nearly impossible. Prefer a private object store with versioned access over a commit.
🔍 Why this matters: your trace store has a TTL and a privacy review. Your eval fixtures probably don't. The second one is where data quietly outlives its purpose.
Provider-side retention belongs on your data map
Everything above concerns data you store. There's a parallel flow you don't control: what the model provider retains.
Providers differ in whether they retain prompts and completions, for how long, whether humans review them, and whether the data is used for training. Enterprise tiers usually offer stronger commitments — zero data retention arrangements, regional processing, or contractual no-training terms — but the specifics vary by product surface, not just by vendor. An API endpoint may have different terms than a consumer chat product; a batch endpoint may differ from a real-time one; a fine-tuning upload is retained by definition.
Three consequences for a team building on top of an API:
Your minimization work still matters. A no-training clause doesn't stop a provider from logging a request for abuse detection, and it doesn't reduce the impact of a breach on their side. Sending less is still better.
Data residency has a per-request dimension. If you route requests to a region based on tenant, the routing decision has to be enforced in code and verified in traces, not just promised in a contract. Records with a residency commitment almost always require evidence of where each request went.
Your DPA has to cover subprocessors you didn't pick. If your observability vendor embeds a model provider for summarization features, that's a fourth party. Ask, in writing, and keep the answer current.
Make the safe path the easy path
Every control described here competes with a developer's desire to see what happened. If the compliant path requires three extra steps, engineers will find a shortcut, and the shortcut will be a console.log(prompt) in a container whose stdout goes somewhere ungoverned.
The countermeasure isn't policy, it's ergonomics.
Give engineers a redacted trace that's genuinely useful. Show the prompt template version, the token breakdown, the retrieval scores, the tool latencies, and the outcome. If that view answers eighty percent of debugging questions, content capture becomes the exception rather than the default.
Make temporary capture a first-class feature. A button that says "capture content for this request for one hour" with an audit entry is better than a config flag someone flips globally and forgets. It's also easier to defend to a reviewer, because the intent and the duration are recorded.
Fail closed on new services. When a service starts emitting content attributes without a redaction processor registered, the collector should drop the content and increment a counter rather than accept it. New services are where policies leak, because the person scaffolding them copies an example from a blog post.
Publish the numbers. A monthly line in the engineering review — how many traces were captured with content, by whom, why, and how many expired — turns a policy into an operational fact. Teams behave differently when the count is visible.
What to actually build first
If I were tightening an existing AI feature this week, the order would be:
- Turn off default content capture in production. Set the SDK's capture flag to off everywhere, then re-enable deliberately per environment.
- Enumerate content-bearing attributes, including client-side replay and any log line that prints the prompt.
- Add the in-process redactor for the fields you can't avoid capturing, with HMAC tokens and a secret key.
- Split storage into metadata and content, with different TTLs and different roles.
- Add the break-glass path with audit, before someone builds a shadow path with a spreadsheet export.
- Instrument the capture switch itself so you can answer who enabled content capture and when it expires.
- Add a retention verification job that fails loudly when the oldest record is older than policy.
The framing that makes this stick with a team: traces are not logs, they're records. Logs describe events; these contain the event's substance. Once you accept that, most of the controls follow naturally — you classify the record, minimize it, tokenize the identifiers, bound the retention, and audit the reads. The alternative is what most teams have right now: excellent debugging, and a data inventory nobody can complete.