Beyond Black-Box Compliance: Defensible LLM & RAG Explainability for the Enterprise

A Technical & Regulatory Whitepaper on Production AI Governance Published by: AgentaFlow / WhiteBoxXAI Research & Governance Team Author: WhiteBoxXAI Architecture & Compliance Working Group Target Audience: Enterprise AI Risk Officers, Chief Compliance Officers, AI Governance Boards, and Principal ML Architects Scope of Compliance: ISO/IEC 42001:2023, EU AI Act (Regulation (EU) 2024/1689), NIST AI RMF 1.0, GDPR (Articles 13–15, 22), CCPA/CPRA

---

Executive Summary

Enterprise adoption of Generative AI and Retrieval-Augmented Generation (RAG) has created a profound regulatory paradox: while international governance frameworks mandate transparency, auditability, and explainability for automated decision systems, the foundational architecture of Large Language Models (LLMs) fundamentally resists traditional explainable AI (XAI) methodologies.

When traditional techniques such as KernelSHAP or LIME are naively ported from tabular models to autoregressive generative pipelines, they fail across three dimensions:

1. Mathematical Incompatibility: Natural language generation produces high-dimensional probability distributions across token sequences rather than scalar decision boundaries, rendering standard surrogate models mathematically ungrounded.
2. Operational Failure: Combinatorial token masking creates out-of-distribution (OOD) artifacts that cause attention collapse, latency explosion ($>1,000\times$ inference overhead), and prohibitive inference costs.
3. The Closed-API Barrier: Commercial foundation models (OpenAI, Anthropic, Google) do not expose gradients, logit vectors, or hidden layer states.

In response to this impasse, a concerning industry trend has emerged: compliance theater. Vendors generate synthetic "importance scores," fabricate sensitivity metrics using pseudo-random heuristics, or present unverified visual heatmaps to satisfy audit checklists. In regulated environments, fabricated evidence is legally worse than no evidence—it constitutes false record-keeping under Article 12 of the EU AI Act and represents an audit failure under ISO/IEC 42001 Clause 8.

This whitepaper details WhiteBoxXAI's dual-tier explainability architecture designed to provide mathematically defensible, auditor-ready evidence for both open-weights models and closed-API RAG pipelines:

• Tier 3 (Open-Weights LLMs / White-Box): Exact embedding-layer backpropagation using Input×Gradient and Axiomatic Integrated Gradients, combined with multi-head attention decomposition.
• Tier 4 (Closed-API & RAG Systems / Grey-Box & Black-Box): Semantic chunk-level attribution, leave-one-chunk-out (LOCO) ablation, and sentence-level claim grounding to isolate hallucinations.
• The Honest Unmeasured State Principle: A non-negotiable governance doctrine requiring systems to explicitly record unmeasured states rather than simulating synthetic metrics.
• Plain-Language Audit Verdicts: Deterministic translation of attribution tensors into defensible prose that enterprise AI risk officers can present directly to external auditors.

---

1. The Generative AI Interpretability Crisis

1.1 Why Tabular XAI (SHAP & LIME) Fails for LLMs

The foundational literature of post-hoc explainability—principally LIME (Ribeiro et al., 2016) and KernelSHAP (Lundberg & Lee, 2017)—was conceived for low-dimensional tabular classifiers and static image classification models:

$$f: \mathbb{R}^d \to [0, 1]^K$$

In tabular regimes, features are distinct, discrete or continuous independent variables (e.g., Credit Score, Debt-to-Income, Age). Features can be masked or permuted against an empirical marginal distribution $\mathcal{D}$ without violating the underlying semantic grammar.

In contrast, autoregressive Large Language Models operate as sequence-to-sequence conditional probability distributions:

$$P(Y \mid X) = \prod_{t=1}^T P(y_t \mid y_{<t}, x_1, \dots, x_N)$$

Applying tabular XAI to this formulation causes catastrophic breakdowns:

A. The Out-of-Distribution (OOD) Perturbation Dilemma

To compute Shapley values $\phi_i(v) = \sum_{S \subseteq N \setminus {i}} \frac{|S|!(|N|-|S|-1)!}{|N|!} (v(S \cup {i}) - v(S))$, an algorithm must evaluate the model across power-set coalitions of input tokens. In natural language, replacing token $x_i$ with an arbitrary baseline token (such as [PAD], [UNK], or empty string) produces ungrammatical, nonsensical prompt strings.

Autoregressive transformer models subjected to random token dropout experience attention shock: positional encodings lose semantic continuity, attention heads allocate diffuse weights across meaningless syntax, and the resulting token probabilities reflect the model's behavior on catastrophic OOD noise rather than its true sensitivity to semantic features.

B. Combinatorial and Computational Latency Explosion

For an enterprise prompt with $N = 512$ tokens, the exact Shapley feature space contains $2^{512} \approx 1.34 \times 10^{154}$ coalitions. Even an approximation sampling $M = 2,048$ coalitions requires thousands of sequential API or model evaluations per inference call. In a production pipeline processing 100 requests per minute, computing tabular SHAP explanations is computationally intractable, incurring millions of dollars in unnecessary compute and introducing multi-minute latency per transaction.

C. The Sequence Output Space Mismatch

LIME and SHAP presuppose a scalar output target $f(x)$ (e.g., class probability or continuous score). An LLM generates an autoregressive sequence $Y = (y_1, \dots, y_T)$. Perturbing a single token in the prompt does not merely change a scalar score; it fundamentally alters the generation trajectory, shifting token lengths, syntax, semantic framing, and factual assertions. Reducing this multi-dimensional divergence to a single scalar attribution value without sequence grounding strips the explanation of legal and technical meaning.

1.2 The Closed-API Governance Barrier

Modern enterprise architectures predominantly deploy foundation models via commercial endpoints (e.g., Azure OpenAI, Anthropic Bedrock, Google Cloud Vertex AI). These closed APIs enforce strict inference boundaries:

• Loss gradients $\nabla_{\mathbf{x}} \mathcal{L}$ are inaccessible.
• Internal hidden states $\mathbf{h}_l$ and embedding spaces $\mathbf{E}$ are non-observable.
• Attention weight matrices $\mathbf{A}_{l, h}$ are strictly proprietary and unexposed.
• Logit vectors are severely truncated (typically restricted to top-5 log-probabilities, if provided at all).

Faced with this opacity, compliance platforms often resort to one of two unacceptable extremes: completely abandoning explainability for generative AI, or fabricating synthetic metrics.

---

2. Compliance Theater vs. Defensible Audit Evidence

2.1 The Danger of Fabricated Metrics

"Compliance theater" occurs when a software vendor presents synthetic, non-deterministic, or pseudo-random heuristics as formal audit artifacts. Common manifestations in generative AI include:

• Generating pseudo-random token heatmaps to mimic gradient saliency.
• Inverting simulated string transformations to fabricate "robustness" or "sensitivity" scores.
• Sampling randomized word substitutions without executing the perturbed prompt against a model, yet reporting a quantified "output change."

In an enterprise risk context, a fabricated metric is an existential liability. Under Article 12 (Technical Documentation and Logging) and Article 13 (Transparency) of the EU AI Act, deployers of high-risk AI systems must maintain accurate, verifiable records of system operation. If an internal audit, third-party assessment, or regulatory inquiry reveals that an enterprise presented a fabricated score as evidence of model governance, the organization faces:

1. Direct Regulatory Fines: Up to €35 million or 7% of total global annual turnover under the EU AI Act (Article 71), and up to €20 million or 4% of turnover under GDPR.
2. Audit Disqualification: Immediate non-conformity findings under ISO/IEC 42001 Clause 8.2 (Data for AI systems) and Clause 9 (Performance evaluation).
3. Evidentiary Invalidation: Total inadmissibility of governance logs in civil litigation or regulatory enforcement actions.

2.2 The Honest Unmeasured State Principle

WhiteBoxXAI operates under an inviolable engineering doctrine: The Honest Unmeasured State Principle.

                           ┌───────────────────────────────┐
                           │   Attribution / Evaluation    │
                           │           Request             │
                           └──────────────┬────────────────┘
                                          │
                  ┌───────────────────────┴───────────────────────┐
                  ▼                                               ▼
     ┌────────────────────────┐                      ┌────────────────────────┐
     │  Live Compute / Model  │                      │   Closed API / Model   │
     │   Execution Available  │                      │    Outputs Absent      │
     └────────────┬───────────┘                      └────────────┬───────────┘
                  │                                               │
                  ▼                                               ▼
     ┌────────────────────────┐                      ┌────────────────────────┐
     │ Exact Measured Metric  │                      │ Deterministic Candidate│
     │  (e.g., Sensitivity,   │                      │  Generation + Explicit │
     │  Input×Grad, Grounding)│                      │   UNMEASURED (null)    │
     └────────────────────────┘                      └────────────────────────┘
                  │                                               │
                  └───────────────────────┬───────────────────────┘
                                          ▼
                             ┌────────────────────────┐
                             │ Defensible Audit Trail │
                             │ (Legally Reproducible) │
                             └────────────────────────┘

When an enterprise risk officer reviews an explanation in WhiteBoxXAI:

• If a model was queried across controlled perturbations, the system reports the exact, mathematically measured divergence with cryptographic audit logs.
• If an enterprise is operating in an offline, rate-limited, or closed-API configuration where perturbations cannot be actively executed, WhiteBoxXAI produces deterministic candidate prompt mutations and explicitly sets: $$\text{score} = \text{null}, \quad \text{output} = \text{null}, \quad \text{status} = \text{"unmeasured_candidates"}$$
• The system never simulates or guesses an outcome. An explicit null accompanied by verifiable candidate inputs satisfies ISO/IEC 42001 audit standards; a fabricated 0.84 score invalidates the entire governance framework.

---

3. WhiteBoxXAI Technical Architecture

To deliver mathematically rigorous explanations across diverse deployment topologies, WhiteBoxXAI implements a tiered architecture separating white-box open-weights models from black-box closed APIs.

┌──────────────────────────────────────────────────────────────────────────────────┐
│                             WhiteBoxXAI Unified Engine                           │
└────────────────────────────────────────┬─────────────────────────────────────────┘
                                         │
        ┌────────────────────────────────┴────────────────────────────────┐
        ▼                                                                 ▼
┌────────────────────────────────────────┐       ┌────────────────────────────────┐
│   Tier 3: Open-Weights LLM Engine      │       │   Tier 4: Closed-API & RAG     │
│   (PyTorch / HuggingFace Transformers) │       │   (OpenAI, Anthropic, Gemini)  │
├────────────────────────────────────────┤       ├────────────────────────────────┤
│ • Embedding Input×Gradient Saliency    │       │ • RAG Chunk-Level Attribution  │
│ • Axiomatic Integrated Gradients (IG)  │       │ • Sentence Claim Grounding     │
│ • Multi-Head Attention Rollout         │       │ • LOCO Context Ablation        │
│ • Attention Head Sparsity / Pruning    │       │ • Deterministic Clause Ablation│
└──────────────────┬─────────────────────┘       └────────────────┬───────────────┘
                   │                                              │
                   └─────────────────────┬────────────────────────┘
                                         ▼
                 ┌────────────────────────────────────────────────┐
                 │       Unified Attribution Data Contract        │
                 │   (Standardized Features, Scores & Metadata)   │
                 └───────────────────────┬────────────────────────┘
                                         ▼
                 ┌────────────────────────────────────────────────┐
                 │          Plain-Language Audit Verdict          │
                 │  (Automated Translation for Risk & Compliance) │
                 └────────────────────────────────────────────────┘

---

4. Tier 3: Open-Weights Explainability (White-Box)

For self-hosted, sovereign, or dedicated cloud deployments running open-weights architectures (e.g., Llama 3, Mistral, Qwen, DeepSeek), WhiteBoxXAI provides exact internal attribution directly at the PyTorch tensor layer.

4.1 Embedding-Layer Input × Gradient Attribution

Token inputs in a transformer are discrete indices $w_i \in \mathcal{V}$ mapped into a continuous embedding manifold $\mathbf{e}_i = \mathbf{E}[w_i] \in \mathbb{R}^d$. While discrete tokens cannot be directly differentiated, their embedding vectors are continuous and fully differentiable.

Given an input sequence $\mathbf{E} = [\mathbf{e}_1, \dots, \mathbf{e}_N]$ and a target output loss $\mathcal{L}(F(\mathbf{E}), Y)$, WhiteBoxXAI computes the embedding gradient via backpropagation:

$$\mathbf{g}i = \nabla{\mathbf{e}_i} \mathcal{L} = \frac{\partial \mathcal{L}}{\partial \mathbf{e}_i} \in \mathbb{R}^d$$

The local attribution score $\text{Attr}_{\text{IxG}}(w_i)$ is formulated as the inner product of the input embedding and its sensitivity vector, aggregated across the hidden dimension:

$$\text{Attr}_{\text{IxG}}(w_i) = |\mathbf{e}i \odot \nabla{\mathbf{e}_i} \mathcal{L}|_2$$

This isolates the linear first-order impact of each token embedding on the generation of the target sequence $Y$, computed in a single backward pass without combinatorial sampling.

4.2 Axiomatic Integrated Gradients

While Input×Gradient provides fast local sensitivity, it suffers from the well-documented gradient saturation problem (Sundararajan et al., 2017): once an essential feature drives an activation to its asymptotic plateau, its local gradient $\frac{\partial F}{\partial x_i}$ collapses to zero, under-attributing critical tokens.

To guarantee mathematical defensibility, WhiteBoxXAI implements path-integral Integrated Gradients (IG) directly over the token embedding manifold.

Mathematical Formulation

Given a baseline embedding tensor $\mathbf{E}'$ (representing an uninformative null state, such as zero or padding embeddings) and the input embedding tensor $\mathbf{E}$, the integrated gradient for token $i$ along the straight-line interpolation path is defined as:

$$\text{IG}_i(\mathbf{E}, \mathbf{E}') = (\mathbf{e}_i - \mathbf{e}'i) \odot \int_0^1 \nabla{\mathbf{e}_i} F\left(\mathbf{E}' + \alpha (\mathbf{E} - \mathbf{E}')\right) d\alpha$$

In WhiteBoxXAI, the continuous integral is computed via a Riemannian Riemann sum across $m$ discrete interpolation steps (configurable $m \in [10, 100]$, default $m = 20$):

$$\text{IG}_i^m(\mathbf{E}, \mathbf{E}') = (\mathbf{e}i - \mathbf{e}'i) \odot \frac{1}{m} \sum{k=1}^m \nabla{\mathbf{e}_i} F\left(\mathbf{E}' + \frac{k}{m} (\mathbf{E} - \mathbf{E}')\right)$$

The scalar token importance is then obtained via vector norm:

$$\text{Score}(w_i) = \frac{|\text{IG}_i^m(\mathbf{E}, \mathbf{E}')|2}{\sum{j=1}^N |\text{IG}_j^m(\mathbf{E}, \mathbf{E}')|_2}$$

Axiomatic Compliance Proofs

Integrated Gradients is uniquely suited for regulatory compliance because it mathematically satisfies three foundational axioms of attribution theory:

1. Completeness: The sum of attributions equals the difference between the model output at the input $\mathbf{E}$ and the baseline $\mathbf{E}'$: $$\sum_{i=1}^N \sum_{d=1}^D \text{IG}_{i, d}(\mathbf{E}, \mathbf{E}') = F(\mathbf{E}) - F(\mathbf{E}')$$ Auditor Value: Proves that 100% of the prediction shift is accounted for across the input tokens, leaving no unexplained residual.
2. Implementation Invariance: Two models whose outputs are identical for all inputs have identical attributions, irrespective of architectural differences or parameter re-indexing. Auditor Value: Ensures that audit records are independent of hardware or software execution quirks.
3. Sensitivity: If an input token differs from the baseline and causes a change in output, it receives non-zero attribution; if a token does not affect the output, its attribution is identically zero. Auditor Value: Prevents false-positive attribution of inactive prompt tokens.

4.3 Multi-Head Attention Extraction and Rollout

In addition to gradient pathways, WhiteBoxXAI extracts multi-layer, multi-head attention weight tensors:

$$\mathbf{A} \in \mathbb{R}^{L \times H \times N \times N}$$

Where $L$ is layer depth, $H$ is head count, and $N$ is sequence length.

• Non-Collapsing Head Importance: Unlike simplistic implementations that average across heads prematurely, WhiteBoxXAI computes attention entropy and variance per individual head $h \in {1, \dots, H}$ before aggregation, preserving head-specific functional specialization (e.g., induction heads, syntax tracking heads).
• Attention Rollout: To account for residual stream bypass connections across layers $l = 1 \dots L$, WhiteBoxXAI implements attention rollout (Abnar & Zuidema, 2020), modeling information flow as a Markov chain across depth: $$\hat{\mathbf{A}}^{(l)} = 0.5 , \mathbf{A}^{(l)} + 0.5 , \mathbf{I}$$ $$\mathbf{R}^{(l)} = \hat{\mathbf{A}}^{(l)} \cdot \mathbf{R}^{(l-1)}$$ Where $\mathbf{I}$ represents identity residual connections, providing an uncorrupted audit map of token influence across the network depth.

---

5. Tier 4: Closed-API & RAG Explainability (Grey-Box & Black-Box)

For enterprise pipelines utilizing hosted LLMs (OpenAI GPT-4, Claude 3.5, Gemini 1.5) and Retrieval-Augmented Generation (RAG), WhiteBoxXAI provides chunk-level, claim-level, and semantic perturbation attribution.

5.1 RAG Chunk-Level Attribution

In enterprise RAG architectures, prompt inputs consist of a user query $q$ and a set of retrieved context chunks $\mathcal{C} = {c_1, c_2, \dots, c_k}$ retrieved from vector databases or hybrid indexes. The model produces an answer $\mathcal{A}$.

Evaluating token-level saliency on RAG contexts is an anti-pattern: enterprise auditors do not care whether word 127 in chunk 3 had high gradient; they need to know which reference document and chunk drove the assertion.

WhiteBoxXAI defines chunk attribution $\text{Attr}(c_j \mid q, \mathcal{A})$ through a dual-channel formulation:

$$\text{Attr}(c_j) = \beta \cdot \text{Rel}(q, c_j) + (1 - \beta) \cdot \text{Grounding}(c_j, \mathcal{A})$$

1. Query-Context Relevance $\text{Rel}(q, c_j)$: Measures semantic and lexical overlap between the user's intent and the retrieved chunk.
2. Context-Answer Grounding $\text{Grounding}(c_j, \mathcal{A})$: Measures lexical, n-gram, and entity transfer from chunk $c_j$ into the generated text $\mathcal{A}$.

Leave-One-Context-Out (LOCO) Ablation

When inference cost permits active evaluation, WhiteBoxXAI executes Leave-One-Context-Out (LOCO) counterfactual ablation. For each chunk $c_j \in \mathcal{C}$, the pipeline generates an ablated context $\mathcal{C}_{-j} = \mathcal{C} \setminus {c_j}$ and measures output divergence:

$$\Delta_j = 1 - \text{SemanticSimilarity}\left(\mathcal{A}, F(q, \mathcal{C}_{-j})\right)$$

The normalized impact $\Delta_j$ provides an empirical, causal measurement of document necessity: if removing chunk $c_j$ eliminates the core assertion in $\mathcal{A}$, chunk $c_j$ possesses high causal attribution.

5.2 Sentence-Level Claim Grounding & Hallucination Boundary Auditing

Under the EU AI Act (Article 14 Human Oversight), an AI risk officer must be able to audit whether a generative model's factual claims are grounded in provided enterprise data or fabricated by the model.

WhiteBoxXAI decomposes the completion $\mathcal{A}$ into an ordered sequence of discrete factual claims $\mathcal{S} = {s_1, s_2, \dots, s_m}$ using deterministic linguistic boundary tokenization.

For each sentence $s_i$, the engine computes its maximum grounding score across all available context chunks:

$$\text{GroundingScore}(s_i) = \max_{c_j \in \mathcal{C}} \text{OverlapMetric}(s_i, c_j)$$

Generated Answer:
┌────────────────────────────────────────────────────────────┬─────────────────────────────┐
│ Sentence Claim                                             │ Audit Verdict               │
├────────────────────────────────────────────────────────────┼─────────────────────────────┤
│ "The standard policy cancellation window is 30 days."      │ GROUNDED                    │
│                                                            │ (Source: Chunk #2, 94% match)│
├────────────────────────────────────────────────────────────┼─────────────────────────────┤
│ "Refunds are processed within 3 to 5 business days."       │ GROUNDED                    │
│                                                            │ (Source: Chunk #1, 88% match)│
├────────────────────────────────────────────────────────────┼─────────────────────────────┤
│ "All customers qualify for complimentary concierge travel."│ UNGROUNDED CLAIM            │
│                                                            │ (Grounding Score: 8% < 25%) │
│                                                            │ ⚠️ Potential Hallucination  │
└────────────────────────────────────────────────────────────┴─────────────────────────────┘

• Any sentence with $\text{GroundingScore}(s_i) < \tau$ (default threshold $\tau = 0.25$) is flagged as an Ungrounded Claim.
• This provides risk officers with an auditable hallucination boundary: evidence packages presented to regulators explicitly separate verified context derivations from ungrounded model extrapolations.

5.3 Deterministic Clause-Level Perturbation & Sensitivity

Rather than corrupting prompts with random token noise, WhiteBoxXAI performs structured semantic perturbations:

• Sentence & Clause Ablation: Removing entire systemic instructions, constraints, or guardrails one at a time.
• Entity and Attribute Inversion: Systematically swapping key nominal entities to test whether decision policies generalize or exhibit bias.
• Deterministic Word Deletion: Pruning non-essential modifiers to measure core instruction stability.

In accordance with the Honest Unmeasured State Principle, if candidate perturbations cannot be executed against the foundation model, the candidate mutations are preserved with null outcomes, ensuring that unexecuted counterfactuals are never presented as measured evidence.

---

6. The Unified Attribution Schema & Plain-Language Translation

6.1 The Unified Attribution Data Contract

Regardless of whether an explanation originates from a Tier 3 PyTorch backward pass or a Tier 4 RAG chunk attribution, WhiteBoxXAI normalizes the output into an auditor-standardized JSON payload.

{
  "id": "e9b72d1f-8a24-4f91-b384-5a76c8d20e11",
  "model_type": "rag_pipeline",
  "explanation_type": "rag_chunk_attribution",
  "explanation_method": "lexical_grounding_and_relevance",
  "scoring_method": "rag_claim_grounding",
  "target_output": "The maximum loan-to-value ratio is 80% for primary residences...",
  "measured": true,
  "plain_language_verdict": "3 of 3 answer sentences are grounded in 2 retrieved chunks (top source: underwriting-guidelines-v4, 68% attribution). No ungrounded claims detected.",
  "attribution_data": [
    {
      "feature_name": "chunk_0_underwriting-guidelines-v4",
      "attribution_score": 0.684,
      "metadata": {
        "document_id": "doc-uw-004",
        "chunk_index": 0,
        "supported_sentences": [0, 1]
      }
    },
    {
      "feature_name": "chunk_1_credit-policy-appendix",
      "attribution_score": 0.316,
      "metadata": {
        "document_id": "doc-cp-appx",
        "chunk_index": 1,
        "supported_sentences": [2]
      }
    }
  ],
  "created_at": "2026-10-27T12:00:00Z"
}

6.2 The Plain-Language Audit Verdict

Enterprise risk officers, internal legal counsel, and external regulators are rarely machine learning engineers. Presenting an auditor with an attention tensor or an array of Riemann gradients does not satisfy compliance requirements for meaningful transparency.

WhiteBoxXAI incorporates a deterministic Plain-Language Verdict Engine that translates mathematical evidence into executive statements:

---

7. Regulatory Compliance Crosswalk

The following matrix documents the precise statutory alignment between WhiteBoxXAI's explainability engine and the five recognized enterprise frameworks:

---

8. Summary for the Enterprise Buyer

When evaluating AI explainability platforms, enterprise buyers must look beyond marketing claims and verify technical integrity:

1. Demand to see the math: If a vendor claims to explain closed-API models with token-level gradient precision, ask how they calculate loss gradients without backpropagation.
2. Audit the failure modes: Inquire what the platform outputs when an API rate limit occurs or when a model cannot be reached. If it generates a score anyway, it is generating fictitious data.
3. Verify regulatory suitability: Confirm that outputs provide both rigorous technical proofs for data scientists and clear, plain-language summaries for auditors and risk officers.

WhiteBoxXAI is architected from the ground up on the principle that auditability cannot be faked. By combining PyTorch-native tensor backpropagation for open models, causal chunk attribution for RAG systems, and an unwavering commitment to the Honest Unmeasured State Principle, WhiteBoxXAI delivers the only enterprise-grade explainability solution that stands up to technical scrutiny and regulatory audit.

---

For inquiries, technical demonstrations, or architecture reviews, contact the AgentaFlow Governance Team at whiteboxxai.com.