How to Verify RAG Citations and Catch Unsupported AI Claims

Your RAG pipeline returns a confident answer: "According to Section 4.2 of the security policy, all API keys must be rotated every 30 days [Source: security-policy.pdf, p.12]." You check page 12. Section 4.2 discusses password complexity requirements. API key rotation is mentioned in Section 7.1, and it says 90 days, not 30.

The citation looks right. The formatting is correct. The document exists. But the claim is fabricated and anchored to a plausible-looking source. This is the most dangerous class of RAG failure because it passes every superficial check.

If you're shipping RAG systems in production—legal research tools, medical literature assistants, internal knowledge bases—citation verification isn't a nice-to-have. It's the difference between a useful tool and a liability engine.

TL;DR

RAG citation failures aren't just "wrong documents." They include partial misattributions, temporal conflation, and synthesis claims that don't exist in any single source. Verify citations by decomposing answers into atomic claims, running entailment checks against source passages, and layering programmatic rules on top of LLM-based judgment. No single technique catches everything. You need a pipeline.

📋 Table of Contents

• The Five Ways Citations Actually Break
• Decompose Before You Verify: Atomic Claim Extraction
• NLI Entailment: The Workhorse of Citation Checking
• LLM-as-Judge With Structured Rubrics
• Embedding Similarity as a Fast First-Pass Filter
• RAGAS Faithfulness Scoring in Practice
• Programmatic Verification for Structured and Numerical Claims
• Building a Layered Verification Pipeline
• The Citation Drift Problem Nobody Talks About
• Choosing Your Verification Stack

---

1. The Five Ways Citations Actually Break

Most teams assume citation failure means "the retriever grabbed the wrong document." That's the easiest case to catch and the least common one in well-tuned systems. The real failures are subtler:

Type 1: Correct document, wrong passage. The model retrieves the right PDF but attributes a claim to Section 3 when the information is in Section 9. The citation is "real" but misleading.

Type 2: Synthesis hallucination. The answer combines facts from three different sources into a single sentence and cites only one. "The policy requires encryption at rest and in transit [Source: policy.pdf]"—but "in transit" only appears in a different document entirely.

Type 3: Numerical drift. The source says "approximately 200ms." The RAG output says "200ms exactly" or "under 200ms." The citation is technically correct but the precision is fabricated.

Type 4: Temporal conflation. The model pulls a fact from a 2023 document and presents it alongside a 2025 citation, making it appear current.

Type 5: Negation inversion. The source says "this configuration is NOT recommended for production." The output says "this configuration is recommended for production" with a correct-looking citation.

⚠️ Gotcha: Types 2, 3, and 5 are nearly invisible to users who spot-check citations by verifying the source exists and is topically relevant. You need claim-level verification, not document-level verification.

---

2. Decompose Before You Verify: Atomic Claim Extraction

You can't verify a paragraph. You can verify a sentence. More precisely, you can verify a single factual assertion.

The first step in any serious citation verification pipeline is decomposing the generated answer into atomic claims—individual statements that can be independently checked against a source passage.

from pydantic import BaseModel, Field
from openai import OpenAI

class AtomicClaim(BaseModel):
    claim: str = Field(description="A single factual assertion")
    cited_source: str = Field(description="The source reference attached to this claim")
    claim_type: str = Field(
        description="One of: factual, numerical, temporal, comparative, negation"
    )

class ClaimDecomposition(BaseModel):
    claims: list[AtomicClaim]

client = OpenAI()

DECOMPOSE_PROMPT = """Decompose the following AI-generated answer into atomic claims.
Each claim must be a single verifiable factual assertion.
Preserve the citation attached to each claim.

Answer to decompose:
{answer}

Rules:
- One fact per claim
- Keep numerical values exactly as stated
- Preserve negations ("not", "never", "should not")
- Mark comparative claims explicitly ("X is faster than Y")
- If a sentence contains two facts, split into two claims
"""

def decompose_claims(answer: str) -> list[AtomicClaim]:
    response = client.beta.chat.completions.parse(
        model="gpt-4o",
        messages=[
            {"role": "user", "content": DECOMPOSE_PROMPT.format(answer=answer)}
        ],
        response_format=ClaimDecomposition,
    )
    return response.choices[0].message.parsed.claims

Why structured output matters here: you get typed, parseable claims you can route to different verification strategies. A numerical claim goes to your numeric tolerance checker. A negation claim gets flagged for NLI with special attention to polarity. A comparative claim needs both entities present in the cited source.

💡 Practical note: In production, I've found that decomposition quality degrades when answers exceed ~800 words. Chunk long answers before decomposing, or you'll get merged claims that defeat the purpose.

---

3. NLI Entailment: The Workhorse of Citation Checking

Natural Language Inference (NLI) models are trained to classify the relationship between a premise and a hypothesis as entailment, contradiction, or neutral. This maps almost perfectly onto citation verification:

• Premise = the retrieved source passage
• Hypothesis = the atomic claim from the RAG output
• Entailment = the citation is supported
• Contradiction = the citation is actively wrong
• Neutral = the source doesn't address this claim (unsupported)

The microsoft/deberta-v3-large-mnli model (or its fine-tuned variants) runs fast enough for production use and doesn't require an LLM API call:

from transformers import pipeline

nli_classifier = pipeline(
    "text-classification",
    model="cross-encoder/nli-deberta-v3-large",
    return_all_scores=True,
    device=0,  # GPU
)

def verify_claim_against_source(claim: str, source_passage: str) -> dict:
    # The NLI model expects (premise, hypothesis) pairs
    result = nli_classifier(
        f"{source_passage} </s></s> {claim}",
        top_k=None,
    )
    
    scores = {item["label"]: item["score"] for item in result[0]}
    
    # Thresholds tuned for citation verification (not general NLI)
    if scores.get("contradiction", 0) > 0.7:
        verdict = "CONTRADICTED"
    elif scores.get("entailment", 0) > 0.75:
        verdict = "SUPPORTED"
    else:
        verdict = "UNSUPPORTED"
    
    return {
        "claim": claim,
        "verdict": verdict,
        "scores": scores,
    }

The threshold asymmetry matters. I set contradiction lower (0.7) than entailment (0.75) because a false "supported" verdict is worse than a false "unsupported" flag. You'd rather over-flag for human review than silently pass a bad citation.

Where NLI struggles:

• Multi-hop reasoning ("Because X and Y, therefore Z" where X and Y are in different sentences)
• Paraphrasing that changes modality ("should" → "must")
• Claims that are implied but not stated

For these, you need the next layer.

---

4. LLM-as-Judge With Structured Rubrics

NLI catches obvious mismatches. But for nuanced verification—especially synthesis claims, hedging language, and multi-source attribution—you need a more flexible judge.

The critical mistake teams make with LLM-as-judge is asking a vague question: "Is this claim supported by the source?" The model will say yes 80% of the time because it's trying to be helpful.

Instead, use a structured rubric with explicit failure modes:

JUDGE_PROMPT = """You are a citation auditor. Your ONLY job is to determine whether
the cited source actually supports the claim. You are adversarial—assume the claim
is wrong until proven otherwise.

CLAIM: {claim}
CITED SOURCE PASSAGE: {passage}

Evaluate against these criteria:
1. Is the specific fact stated in the passage? (Not implied, not adjacent—STATED)
2. Are numerical values exact matches? (Not "approximately", not "around")
3. Is the polarity preserved? (negations, "not recommended" vs "recommended")
4. Is the temporal context correct? (dates, versions, "as of" qualifiers)
5. Does the passage support the claim at the claimed specificity level?

Respond with JSON:
{{
  "verdict": "SUPPORTED | PARTIALLY_SUPPORTED | UNSUPPORTED | CONTRADICTED",
  "reasoning": "One sentence explaining the verdict",
  "specific_issue": null | "Describe exactly what doesn't match",
  "confidence": 0.0-1.0
}}
"""

The adversarial framing ("assume the claim is wrong") measurably reduces false positives. In my testing across ~2,000 claim-passage pairs, adversarial prompting caught roughly 12% more unsupported claims than neutral prompting, at the cost of ~3% more false flags.

🔍 Why this matters: The "PARTIALLY_SUPPORTED" category is where production value lives. It catches the "200ms approximately" → "200ms exactly" drift that pure entailment misses. Route these to a human review queue rather than auto-rejecting.

---

5. Embedding Similarity as a Fast First-Pass Filter

Running NLI or LLM-as-judge on every claim is expensive. In a system processing hundreds of RAG responses per minute, you need a cheap pre-filter.

Cosine similarity between claim embeddings and source passage embeddings gives you a fast relevance gate:

import numpy as np
from sentence_transformers import SentenceTransformer

embed_model = SentenceTransformer("BAAI/bge-large-en-v1.5")

def fast_relevance_check(
    claim: str, 
    source_passage: str, 
    threshold: float = 0.45
) -> bool:
    """Returns False if the claim is likely irrelevant to the source.
    
    This does NOT verify the claim. It only checks whether the source
    is even in the right topical neighborhood.
    """
    claim_emb = embed_model.encode(claim, normalize_embeddings=True)
    source_emb = embed_model.encode(source_passage, normalize_embeddings=True)
    
    similarity = np.dot(claim_emb, source_emb)
    return similarity >= threshold

The threshold here is deliberately low (0.45). You're not verifying truth—you're filtering out cases where the retriever grabbed something completely off-topic. Claims below this threshold skip the expensive NLI/LLM checks entirely.

What this catches: Wrong-document retrievals, topic drift, and cases where the model hallucinated a citation to a document that's topically adjacent but doesn't contain the fact.

What this misses: Everything in Types 2–5 from Section 1. Two passages can be 0.92 similar and still contradict each other on a single number or negation.

Use this as Layer 0 in your pipeline. Never as the final word.

---

6. RAGAS Faithfulness Scoring in Practice

The RAGAS framework (currently at v0.2.x) provides a faithfulness metric that decomposes answers into statements and checks each against the context. It's a solid starting point if you want an off-the-shelf evaluation harness:

from ragas import evaluate
from ragas.metrics import faithfulness, context_precision
from datasets import Dataset

# Your RAG outputs, structured for RAGAS
eval_data = {
    "question": ["What is the API key rotation policy?"],
    "answer": ["API keys must be rotated every 30 days per Section 4.2."],
    "contexts": [["Section 4.2: Passwords must contain 12+ characters..."]],
    "ground_truth": ["API keys must be rotated every 90 days per Section 7.1."],
}

dataset = Dataset.from_dict(eval_data)

results = evaluate(
    dataset,
    metrics=[faithfulness, context_precision],
)

print(results)  # {'faithfulness': 0.0, 'context_precision': 0.12}

A faithfulness score of 0.0 here correctly flags that the answer isn't supported by the provided context. But RAGAS has limitations in production:

• It uses its own LLM calls internally (cost adds up at scale)
• The decomposition step sometimes merges claims you'd want separate
• It evaluates the entire answer's faithfulness, not per-citation
• It doesn't distinguish "unsupported" from "contradicted"

I treat RAGAS as a regression testing tool—run it against a golden dataset after every prompt or retriever change. For real-time per-response verification, the layered pipeline from Section 8 works better.

---

7. Programmatic Verification for Structured and Numerical Claims

Some claims are trivially verifiable without any ML. If your RAG system answers questions about a database, API documentation, or configuration files, you can check claims programmatically:

import re
from decimal import Decimal

def verify_numerical_claim(claim: str, source_text: str) -> dict:
    """Check if numbers in the claim exactly match the source."""
    # Extract numbers from claim
    claim_numbers = set(re.findall(r'\d+\.?\d*', claim))
    source_numbers = set(re.findall(r'\d+\.?\d*', source_text))
    
    missing = claim_numbers - source_numbers
    
    if missing:
        return {
            "verdict": "NUMERICAL_MISMATCH",
            "missing_in_source": list(missing),
        }
    return {"verdict": "NUMBERS_MATCH"}

def verify_negation(claim: str, source_text: str) -> dict:
    """Flag if the claim contains negation absent in the source."""
    negation_patterns = r'\b(not|never|no|neither|nor|cannot|should not|must not)\b'
    
    claim_has_negation = bool(re.search(negation_patterns, claim, re.I))
    source_has_negation = bool(re.search(negation_patterns, source_text, re.I))
    
    # Claim negates but source doesn't → likely inversion
    if claim_has_negation and not source_has_negation:
        return {"verdict": "NEGATION_MISMATCH", "flag": True}
    return {"verdict": "POLARITY_OK"}

This is cheap, deterministic, and catches the exact class of errors that make NLI models uncomfortable. Run these before the ML-based checks. If a number doesn't appear in the source passage at all, you don't need to burn an LLM call to tell you the citation is wrong.

---

8. Building a Layered Verification Pipeline

No single technique is sufficient. Production citation verification needs layers, each filtering what the next layer processes:

┌─────────────────────────────────────────────────────┐
│  Layer 0: Embedding similarity gate                 │  ← ~1ms per claim
│  (Filter topically irrelevant citations)            │
├─────────────────────────────────────────────────────┤
│  Layer 1: Programmatic checks                       │  ← ~0.5ms per claim
│  (Numbers, negations, date formats)                 │
├─────────────────────────────────────────────────────┤
│  Layer 2: NLI entailment model                      │  ← ~15ms per claim (GPU)
│  (Entailment / contradiction / neutral)             │
├─────────────────────────────────────────────────────┤
│  Layer 3: LLM-as-judge (adversarial rubric)         │  ← ~1-3s per claim
│  (Only for UNSUPPORTED or PARTIALLY_SUPPORTED)      │
├─────────────────────────────────────────────────────┤
│  Layer 4: Human review queue                        │
│  (Contradictions + low-confidence partial matches)  │
└─────────────────────────────────────────────────────┘

async def verify_rag_response(answer: str, sources: list[str]) -> VerificationResult:
    claims = await decompose_claims(answer)
    results = []
    
    for claim in claims:
        passage = retrieve_cited_passage(claim.cited_source, sources)
        
        # Layer 0
        if not fast_relevance_check(claim.claim, passage):
            results.append(FlaggedClaim(claim, "IRRELEVANT_SOURCE"))
            continue
        
        # Layer 1
        num_check = verify_numerical_claim(claim.claim, passage)
        if num_check["verdict"] == "NUMERICAL_MISMATCH":
            results.append(FlaggedClaim(claim, "NUMERICAL_MISMATCH", num_check))
            continue
        
        neg_check = verify_negation(claim.claim, passage)
        if neg_check.get("flag"):
            results.append(FlaggedClaim(claim, "NEGATION_MISMATCH"))
            continue
        
        # Layer 2
        nli_result = verify_claim_against_source(claim.claim, passage)
        if nli_result["verdict"] == "CONTRADICTED":
            results.append(FlaggedClaim(claim, "CONTRADICTED", nli_result))
            continue
        elif nli_result["verdict"] == "SUPPORTED":
            results.append(VerifiedClaim(claim))
            continue
        
        # Layer 3: Only for ambiguous cases
        judge_result = await llm_judge(claim.claim, passage)
        results.append(classify_judge_result(claim, judge_result))
    
    return VerificationResult(claims=results)

The key architectural decision: Layers 0–2 run synchronously in the response path. Layer 3 runs asynchronously. Users get their answer immediately with a "citations verified" badge only if Layers 0–2 all pass. The LLM judge runs in the background and can downgrade the response after the fact if it finds issues.

This keeps p95 latency under 200ms for the verification step while still catching subtle problems.

---

9. The Citation Drift Problem Nobody Talks About

Here's something that bites teams 6–12 months after launch: your source documents change, but your vector store doesn't.

A policy document gets updated. Section 4.2 now says something different. But your embeddings were computed three months ago. Your RAG system retrieves the old chunk, generates an answer based on it, and cites the document correctly. The citation is "valid" in the sense that the document exists and the section number is right—but the content has changed.

Mitigations:

• Track document versions alongside chunks. Store a doc_version or content_hash field in your vector metadata. At verification time, check whether the cited chunk matches the current document version.

# In your vector store metadata
{
    "chunk_text": "API keys must be rotated every 90 days...",
    "source": "security-policy.pdf",
    "section": "7.1",
    "content_hash": "sha256:a3f2...",
    "doc_version": "2025-03-14",
    "indexed_at": "2025-03-15T08:00:00Z",
}

# At verification time
def check_source_freshness(cited_source: str, current_doc_hash: str) -> bool:
    indexed_hash = vector_store.get_metadata(cited_source)["content_hash"]
    return indexed_hash == current_doc_hash

• Schedule re-indexing with the same frequency as your document update cycle. If your legal team updates policies monthly, re-index monthly. If your docs change on every merge, hook re-indexing into your CI pipeline.

• Flag stale citations explicitly. If the hash doesn't match, don't silently fail. Surface "This citation references a previous version of the document" to the user.

---

Choosing Your Verification Stack

The right verification depth depends on what you're building and what a wrong citation costs:

Use Case	Minimum Viable Verification	Recommended Stack	Latency Budget
Internal knowledge base (low risk)	Embedding gate + NLI	Layers 0–2	< 100ms
Customer-facing support bot	NLI + programmatic checks	Layers 0–2 + async Layer 3	< 200ms sync
Legal / compliance research	Full pipeline + human review	All 5 layers	Async acceptable
Medical / clinical decision support	Full pipeline + mandatory human sign-off	All layers + audit log	Async, block on contradictions
Code documentation assistant	Programmatic + NLI	Layers 0–2 + regex for API names	< 50ms

My default recommendation for most production RAG systems: Layers 0 through 2 in the synchronous path, Layer 3 async, and a weekly RAGAS regression run against a golden test set of 200–500 known claim-passage pairs. Add Layer 4 (human review) only if your domain has regulatory or safety implications.

The single highest-ROI improvement you can make today, if you're doing nothing: add the negation check. It's five lines of regex, costs nothing, and catches the most embarrassing class of citation failure—the one where your system tells a user to do something the documentation explicitly says not to do.

Start there. Layer up as your failure modes become clearer. The verification strategy that works at 100 queries per day will look very different from the one you need at 100,000. Build the pipeline interface now so you can swap layers without rewriting your RAG application.