A product team ships a "smart suggestions" panel. It watches which fields people edit, learns their preferred formats, and reorders options over time. Six months later, someone in legal asks a simple question: which users agreed to have their behavior used for model training? The answer, it turns out, is "everyone who accepted the terms," and the terms mention "service improvement" in a paragraph nobody has read since 2023.
That is not a consent design. It is a consent-shaped artifact created after the fact. The engineering problem is that by the time anyone asks the question, the behavior data has already flowed through a feature store, a labeling pipeline, an evaluation set, and possibly a fine-tuning job. Retrofitting consent at that point is expensive and, in some cases, impossible.
Consent has to be designed as a runtime property of the system, not a document that sits next to it. That means treating it as state with a schema, an enforcement point, and an audit trail — the same way you treat authorization.
Three kinds of "learning" that get conflated
The first design mistake is using one word for three different activities. They have different data flows, different risk, and often different legal bases.
Per-user adaptation at inference. The system changes its output for a specific user based on that user's history, within a session or across sessions. A prompt gets prefixed with the user's last five corrections. A ranking model reweights options for this account. Nothing is shared with other users.
Global model training or fine-tuning. Behavior from many users is aggregated into a training set, and the resulting weights are shared by everyone. This is the highest-risk activity because one user's data influences outputs for people who never interacted with them.
Evaluation and analytics. Behavior is used to measure quality, build test sets, or analyze error patterns. Even if no model is trained, humans may read the data, and evaluation sets have a habit of being reused for years.
A consent design that covers only the first is common and insufficient. A checkbox that says "personalize my experience" does not authorize using the same events to fine-tune a model, and an "improve our services" clause is too vague to be specific consent in most privacy regimes. Each activity needs its own purpose identifier, and the system needs to know which purpose a given data flow serves.
What consent actually has to satisfy
The legal details vary by jurisdiction, and none of this is legal advice. But the engineering shape is stable across the major regimes. Under the GDPR, consent must be freely given, specific, informed, and unambiguous, expressed by a clear affirmative act (Article 4(11)). It must be as easy to withdraw as to give, and the controller must be able to demonstrate that consent was obtained (Article 7). The UK ICO's guidance makes the same points in operational terms: name the controller, name each purpose, avoid bundling, and keep records (ICO consent guidance).
Translate that into requirements your system can enforce:
- Purpose granularity. One record per purpose, not one boolean per user.
- Informed. The user saw a specific description of what will be done, and you can prove which version they saw.
- Affirmative. The default is off, and the action that turns it on is a distinct act, not a side effect of clicking "Continue."
- Withdrawable. Revocation takes effect within a defined window, and the UI path is as short as the opt-in path.
- Demonstrable. You store who consented, to what, when, from which surface, and against which text.
Consent is not the only lawful basis for processing, and legitimate interests can cover some analytics. But if your product strategy is "ask people and respect the answer," the system still needs the same machinery. The machinery is also what lets you answer a subject access request without a week of manual archaeology.
Model consent as a scoped, versioned record
A single consent_given column will not survive contact with reality. Consent is a set of grants with dimensions:
| Dimension | Example | Why it matters |
|---|---|---|
| Subject | user:8412 or workspace:99 |
Users and tenants are different subjects |
| Purpose | personalization, model_training, analytics |
Different activities, different grants |
| Scope | edits, messages, uploads |
Users may allow some data, not all |
| Policy version | privacy-2026-04-v3 |
The text they actually saw |
| Granted at | 2026-04-12T18:03:11Z |
Ordering and expiry |
| Source | settings/personalization |
Where the grant came from |
| Withdrawn at | null or timestamp |
Revocation is a state, not a delete |
Storing withdrawal as a timestamp rather than deleting the row is deliberate. You need to prove that consent existed at the time data was collected, and you need to know that it was later withdrawn. Deleting the record destroys both facts.
A minimal schema:
CREATE TABLE consent_grants (
id bigserial PRIMARY KEY,
subject_type text NOT NULL, -- 'user' | 'workspace'
subject_id text NOT NULL,
purpose text NOT NULL,
scope text NOT NULL,
policy_version text NOT NULL,
granted_at timestamptz NOT NULL,
withdrawn_at timestamptz,
source text NOT NULL,
UNIQUE (subject_type, subject_id, purpose, scope, granted_at)
);
CREATE INDEX ON consent_grants (subject_type, subject_id, purpose)
WHERE withdrawn_at IS NULL;
The partial index is doing real work: the hot query is "is there an active grant for this subject and purpose," and that is exactly the subset you want indexed.
Note what is not in this table: the policy text itself. Store that separately and reference it by version, with a hash of the rendered text. If you edit the text in place, the version becomes a lie, and you lose the ability to show what a user actually agreed to.
Enforce consent at every boundary that moves data
A consent check in the settings UI is decoration. Enforcement has to sit where data crosses a boundary:
- Collection. If the event is only collected for a purpose the user has not granted, do not collect it. This is stronger than collecting and filtering later, because it removes the data from backups, logs, and vendor pipelines you may not control.
- Storage. Tag rows with the purpose and scope they were collected under. Purpose is a property of the record, not a property of the table.
- Training-set assembly. Filter by active grants at the moment the set is built, and record the filter. A training set is a derived artifact; it needs its own provenance.
- Inference. Per-user adaptation should check the grant at read time, so a withdrawal stops affecting behavior on the next request rather than at the next deploy.
- Export and vendor transfer. Any downstream processor must receive only data covered by the grant, and the contract with that processor must match.
- Deletion. Revocation should trigger deletion or de-identification of raw data, and a documented policy for derived artifacts.
The training-set boundary is where most teams get caught. A common pattern is an immutable snapshot taken monthly, with consent applied when the snapshot is created. That is workable, but only if you record which grants were active at snapshot time and can rebuild the set. If your pipeline joins consent at query time and the consent table changes, yesterday's reproducible training run becomes unreproducible.
The cleanest pattern I have seen is to materialize a training_examples table with a consent_policy_version and consent_checked_at column, and to treat the table as immutable for a given dataset version. Rebuilding creates a new version. That gives you reproducibility and an audit trail without freezing the live consent table.
Revocation is the part that decides whether the design is real
Withdrawal is easy to implement as a UI event and hard to implement as a system property. The hard questions are all about derived data.
Does the model need retraining? Strictly, removing one user's examples from a trained model is not the same as retraining without them, and exact unlearning is an active research area with no universal drop-in solution. In practice, most teams handle this with a combination of: exclude the data from the next training run, age out models trained on the old set within a defined period, and document the limitation. What you should not do is claim deletion you cannot perform.
What about aggregates? A count of edits per week is not personal data in the same sense as a row of raw events, but an aggregate over three users can be. Define a minimum cohort size for retained aggregates, and treat small cohorts as re-identifiable.
What about backups? You cannot rewrite immutable backups. The honest approach is a documented retention window and a restore procedure that re-applies deletions after restore. Test that procedure; an untested restore path is where deleted data quietly comes back.
What about evaluation sets? These are the most overlooked. A 500-example golden set built from real user sessions may contain data from users who later withdrew. Either rebuild the set or replace the examples with synthetic or consented ones. Keep the mapping from example to subject so this is possible at all.
What about the feature store? Per-user features computed from withdrawn data should be invalidated. If your feature store has no way to delete by subject, you have a gap that will surface in an audit.
A useful internal metric is revocation latency: the time from a withdrawal event to the last system that still uses the data. Measure it per store, not as a single number. "Raw events: 4 minutes, feature store: 26 hours, training set: next monthly build" is a real answer. "Handled" is not.
Granularity users can actually understand
Privacy engineers sometimes respond to granularity requirements by shipping fourteen toggles. That is not informed consent; it is a puzzle. Users cannot meaningfully agree to a distinction they do not understand, and a wall of switches trains people to click "accept all."
The useful granularity is the granularity of consequences. Three or four purposes, each described by what the user will notice:
- "Use my edits to improve suggestions for me." Affects only this account, visible immediately.
- "Use my anonymized activity to improve the product for everyone." Affects model training.
- "Send me product updates." Marketing, separate basis, separate toggle.
- "Allow analysis of my support conversations." Often a separate, more sensitive category.
Each toggle should carry a one-sentence example of what happens if it is on and what changes if it is off. And the off state should be real: if personalization is off, the feature should behave like a non-personalized version, not like a degraded nag screen.
Do not bundle training consent into the terms of service for an existing feature. "Accept or lose access" is generally not freely given when the processing is not necessary for the contract. If a feature genuinely cannot work without a data use, say so and offer a non-AI path rather than a consent screen that is really a paywall.
Defaults, dark patterns, and the asymmetry of the ask
The default state matters more than the copy. Off by default is the only default that makes the affirmative act meaningful. If the toggle is pre-checked and the user clicks "Save," you have an argument about whether that click was consent, and you will probably lose it in front of a regulator.
Watch for the patterns that quietly undermine the design:
- Confusing double negatives. "Don't not use my data" is not a choice.
- Asymmetric friction. One click to enable, a support ticket to disable.
- Visual weight. The accept button in brand colors, the decline link in 11px gray.
- Consent by continued use. "By continuing to use this page, you agree."
- Withdrawal that only stops future collection but leaves the current session personalized. Users expect the change to apply now.
The European Data Protection Board has written specifically about deceptive design patterns in consent interfaces, and the underlying principle is simple: the interface should make the privacy-protective choice as easy as the permissive one. If you would be embarrassed to show the screen to a journalist, it is probably not compliant either.
Consent inside a power imbalance
Two contexts deserve special handling because "freely given" is hard to establish: employees and enterprise tenants.
For employees, monitoring-adjacent AI features — productivity suggestions, message drafting analytics, behavior-based scoring — are usually better governed by works council agreements, transparency notices, and strict purpose limitation than by a checkbox. An employee clicking "I agree" under managerial pressure is weak consent at best. If the feature is optional, make the opt-out genuinely consequence-free, and log that the opt-out was offered.
For enterprise customers, the tenant is often the decision-maker and the individual user is a data subject. The admin may be able to enable a feature for the workspace, but that does not automatically authorize training on individual employees' content. Distinguish tenant-level configuration (a contract and DPA question) from user-level consent (a data subject question). They are different gates, and both need to be checked.
This is also where a "customer-facing transparency" feature earns its keep: a page where an admin can see which purposes are enabled, which data scopes are included, and when the setting last changed. Support teams will use it more than lawyers will.
Show your work: receipts and policy versions
A consent receipt is a record the user can retrieve: what they agreed to, when, and the exact text. You do not need a standard format to get most of the value, though structured receipts are emerging as a practice. What you do need is immutability.
Version your policy text and hash it. When a user grants consent, store the version and hash. When the text changes materially, re-ask rather than assuming the old grant covers the new purpose. "Materially" is the hard word; a useful internal rule is that any change to the list of data types, the list of purposes, or the retention period is material, and copy edits are not.
Keep the rendering path reproducible. If the consent screen is generated from a CMS and someone edits a paragraph, the hash changes and the old grants no longer match any current text. That is not a bug in the audit trail; it is the audit trail telling you the truth.
Code: a gate that fails closed for training
The enforcement function should be boring, explicit, and hard to bypass. Something like this, used at the point where a batch of examples is selected:
from dataclasses import dataclass
from datetime import datetime, timezone
from enum import Enum
class Purpose(str, Enum):
PERSONALIZATION = "personalization"
MODEL_TRAINING = "model_training"
ANALYTICS = "analytics"
@dataclass(frozen=True)
class Grant:
subject_id: str
purpose: Purpose
scope: str
policy_version: str
granted_at: datetime
withdrawn_at: datetime | None
def has_active_grant(
grants: list[Grant],
subject_id: str,
purpose: Purpose,
scope: str,
required_version: str,
at: datetime | None = None,
) -> bool:
at = at or datetime.now(timezone.utc)
for g in grants:
if (
g.subject_id == subject_id
and g.purpose is purpose
and g.scope == scope
and g.policy_version == required_version
and g.granted_at <= at
and (g.withdrawn_at is None or g.withdrawn_at > at)
):
return True
return False
Three deliberate choices. The check requires an exact policy version, so a policy change invalidates old grants instead of silently extending them. It evaluates the grant at a supplied timestamp, so backfills can ask "was this permitted when the data was collected" separately from "is it permitted now." And it defaults to False — no grant, no data. For training, failing closed is the only defensible default.
Then use it in the query that assembles the dataset, not in the code that reads the dataset later:
def build_training_examples(events, grants_by_subject, policy_version):
selected, rejected = [], 0
for event in events:
grants = grants_by_subject.get(event.subject_id, [])
if has_active_grant(
grants, event.subject_id, Purpose.MODEL_TRAINING,
scope=event.data_scope, required_version=policy_version,
at=event.occurred_at,
):
selected.append(event)
else:
rejected += 1
return selected, rejected
Log rejected per scope. A sudden drop in eligible examples is a signal that a policy version changed or a UI regression turned a toggle off. That number is worth a dashboard, because it is the difference between "we respect consent" and "we think we do."
Handling the ambiguous middle
Real systems have states that don't map cleanly to granted or denied: consent expired, policy version changed, subject merged across accounts, tenant disabled a feature after users opted in.
Write down the tie-breaking rules in advance:
- Expired grant → treated as no grant for new data; existing data follows the retention policy.
- Policy version mismatch → no grant for new collection; re-prompt on next visit, do not silently grandfather.
- Account merge → union of grants is wrong; take the intersection for purposes that expand scope, and keep per-subject history for audit.
- Tenant disables feature → effective permission is the intersection of tenant policy and user grant. Neither alone is sufficient.
- Unknown or missing record → deny for training, allow for core functionality that does not depend on the learning behavior.
That last distinction is important. Consent gates learning, not the product's basic operation. A user who declines personalization should still be able to use the editor. If your architecture couples the two, you have turned a privacy choice into a functional hostage situation.
The prompt is a data flow too
Teams that carefully gate the training pipeline often forget that per-user personalization puts behavior into the prompt on every request. If the assistant is "reminded" of the user's last twenty edits, those edits are being transferred to a model provider, logged in an observability tool, and possibly retained under the provider's terms.
That is a different processing operation from training, and it may have a different basis. It also has different exposure. A training set is internal; a prompt may cross a vendor boundary and land in a trace dashboard that half the company can read. Consent for "improve suggestions for me" does not automatically cover "send my editing history to a third-party inference provider in every request."
Two engineering practices reduce the gap. First, keep a manifest of what goes into the prompt for each feature — not the text, but the categories of data and their purposes. Review it when the prompt changes, the same way you review a schema migration. Second, make redaction and minimization a function of purpose: if the purpose is format preference, inject a compact style profile, not raw document text. The most privacy-preserving prompt is the one that contains the smallest sufficient representation of the user.
This is also where retention rules for traces matter. If your observability platform stores full prompts for ninety days, that platform is a copy of the data, and a withdrawal needs to reach it. Many teams discover this only when they try to honor a deletion request and find the payload sitting in three vendor systems.
Consent debt after an acquisition or a pivot
The other failure mode is historical. A product collects behavior under one privacy notice, gets acquired, and the new owner wants to use the data for a different purpose. The data is already there, the users are gone, and the original consent covered something narrower.
There is no clean engineering fix for this, which is why it should be handled as a product constraint rather than a data-science opportunity. The defensible options are to re-contact users for the new purpose, to restrict the new use to data collected after the change, or to rely on a different lawful basis if one genuinely applies. Re-labeling old data with a new purpose because it is convenient is the option that creates liability.
Practically, this means your schema should make the collection purpose immutable. A row of behavior should carry the purpose it was collected under, and any new use should check whether that purpose covers it. If you only store a foreign key to the current privacy policy, you have made the past editable, and you will not be able to reconstruct what was permitted.
What to measure after launch
Consent infrastructure is invisible when it works, so instrument it like any other critical path:
- Grant rate by purpose and surface, to catch confusing UI.
- Revocation rate and revocation latency per downstream store.
- Training-set rejection rate by scope, to catch policy/version drift.
- Share of examples with a policy version that no longer matches current text.
- Count of subjects present in evaluation sets without an active grant.
- Restore drills that verify deletions are re-applied.
The fifth metric is the one that tends to be nonzero for a long time. Evaluation sets are built by engineers who grab "a sample of real conversations," and those samples outlive the consent state that produced them. A quarterly job that reconciles eval sets against current grants is cheap insurance.
A design checklist
Before shipping a feature that learns from behavior, confirm:
- Each learning activity has its own purpose identifier, and the data flow is tagged with it.
- Consent is stored per subject, purpose, scope, and policy version, with withdrawal as a timestamp.
- The policy text is versioned and hashed; old grants reference the text the user actually saw.
- Training-set assembly filters on active grants at collection time and records the filter.
- Revocation propagates to the feature store, caches, and eval sets, with measured latency.
- The UI default is off, the decline path is as short as the accept path, and the off state does not degrade the core product.
- Tenant-level policy and user-level consent are checked separately, and the effective permission is their intersection.
- Backups and restores have a tested procedure for re-applying deletions.
- The team can answer "which users' data trained this model?" from records, not from memory.
- Someone owns the re-consent process for material policy changes.
The underlying shift is from consent as a document to consent as a control plane. Documents get written once and drift. A control plane has a schema, an enforcement point, tests, and metrics, which is what it takes to keep a promise that a user made in a settings panel six months ago and has since forgotten — but which your system has not.