AI Verification LandscapeSearch

Implementation · On-chip & hardware-enabled

Attestable Audits

A research prototype that runs AI safety benchmarks inside a trusted execution environment and publishes attestations binding the model, the audit and the results.

Also called Attestable Audits prototype; Verifiable AI safety benchmarks in TEEs

R2DemonstratedProvider-reported

Summary

Attestable Audits, from University of Cambridge researchers, lets a model provider and an auditor who do not trust each other run a benchmark on a confidential model. Both send their inputs, encrypted, to a trusted execution environment: the provider its weights, the auditor its test code and data. The enclave runs the audit and publishes an attestation that binds the model hash, the audit and the result. Later, users can check that the model answering them is the one that was audited. The authors' prototype on CPU-only AWS Nitro Enclaves ran a 4-bit Llama-3.1-8B on MMLU, XSum and ToxicChat. CPU inference cost 21.7 times as much per token as GPU inference, and the enclave roughly doubled the CPU cost; the authors expect an overhead as small as 5 times on confidential-computing GPUs. No code is linked. The design trusts the TEE vendor, and prompt-based model exfiltration remains an open gap.

ReadinessR2Demonstratedlow confidence · rubric v1.0 · 2026-09-23

R2. One developer paper reports end-to-end results on commercial hardware, but there is no public code and no independent reproduction, so confidence is low.

Rubric assessment
  • R1 met: the protocol, goals and adversaries are published 1.
  • R2 met through reproducible published end-to-end results. The paper specifies the protocol and its algorithms, the enclave and baseline instances, the models and the benchmarks. It reports results for Llama-3.1-8B on AWS Nitro Enclaves, which are commercially available hardware, against a stated adversary 1. The reported evaluation covers the audit step. The paper links no code, which the rubric does not require for this route.
  • R3 not met. No party other than the developers is documented relying on it for a verification decision, and the prototype is CPU-only research code rather than a production-grade system. As of September 2026 no independent security evaluation has been published.
Gaps to the next level
  • Reliance by a party other than the developers for a verification decision, or a production-grade, available implementation, for example on GPU confidential computing at realistic model scale.
  • An independent public security evaluation (audit, red-team or peer-reviewed analysis).

What it is

Attestable Audits is a protocol and prototype by four researchers in the University of Cambridge's Department of Computer Science and Technology 1. It targets a specific problem: benchmarks "typically do not offer verifiable results and lack confidentiality for model IP and benchmark datasets" 1. The design runs audits inside a TEE, so that a model provider and an auditor who do not trust each other can produce a result that users and regulators can verify 1. It is an implementation of TEE remote attestation for AI workloads and, through its inference step, of Model identity attestation.

How it works

The protocol has three steps 1:

  1. Prepare. The enclave attests its boot image and publishes a public key. The model provider sends its model, encrypted to that key. The enclave quantizes and hashes the model and posts an attestation to a transparency log.
  2. Audit. The provider sends the model and the auditor sends its audit code and data, each encrypted. The enclave runs the audit in a sandbox and publishes an attestation that binds the model hash, the audit and the result.
  3. Inference. The enclave loads the earlier attestations and checks that the served model's hash matches. It then answers each encrypted prompt with an attestation that links the model, prompt, output and audit result.

Verifiers compare the enclave's measurement registers with known images and check the vendor's signature 1. The paper considers two adversaries: network attackers, and privileged or physical attackers who may take RAM snapshots, roll back VMs or use side channels 1.

Evidence

The authors report the following results:

  • Setup. The prototype ran on CPU-only AWS Nitro Enclaves with a 4-bit Llama-3.1-8B-Instruct and three benchmarks: MMLU, XSum and ToxicChat. The reported evaluation covers the audit step 1.
  • Throughput. The enclave produced 1.84 tokens per second, against 202 on an NVIDIA L40S GPU baseline 1.
  • Cost. The enclave cost $5.80 per 100K tokens, against $2.61 on a plain CPU instance with the same number of cores and $0.12 on the GPU baseline. The authors put the cost of CPU inference at 21.7 times that of GPU inference. They attribute the roughly twofold enclave cost to needing a larger instance, or giving up cores 1.
  • Accuracy. Zero-shot MMLU accuracy was 51.4% at 4-bit, against 54.6% for the unquantized model on the GPU 1.
  • Loading. Copying a model into the enclave took at most 2 minutes 1.
  • Outlook. The authors expect a production version on confidential-computing GPUs to have "an overhead as small as 5×" 1.
  • Code. The paper links no code repository 1.

Limitations

  • Vendor trust. The approach assumes the TEE vendor is trusted 1.
  • Attacks on confidential VMs. The authors cite memory aliasing (BadRAM), ciphertext side channels (CIPHER-LEAKS) and malicious interrupts (Heckler), and propose revoking affected images when such attacks are found 1. Physical attacks on the underlying TEEs, such as memory-bus interposition, are discussed under TEE remote attestation for AI workloads.
  • Exfiltration. The authors name prompt-based model exfiltration during user interaction as a residual gap 1.
  • Model size. Memory limits forced 4-bit quantization, which lowered MMLU accuracy 1.
  • Multi-GPU. The authors wrote that H100 confidential computing had no multi-GPU support 1. NVIDIA's August 2025 white paper describes an eight-GPU protected-PCIe mode on Hopper 2.

Known flaws

Published flaws, with their severity, kind and status. How flaws are rated.

  • Relies on the TEE vendor and inherits TEE attacksSignificantTheoretical argumentOpen

    The design depends on trusting the TEE vendor, AWS in the prototype 1. The authors cite memory-aliasing, ciphertext side-channel and malicious-interrupt attacks on confidential VMs (BadRAM, CIPHER-LEAKS, Heckler). Their answer is to revoke vulnerable base images once such attacks are discovered 1.

  • Prompt-based model exfiltration is a residual gapSignificantOpen questionOpen

    The authors state that "prompt-based model exfiltration during the user interaction step remains a residual gap" 1.

  • CPU-only enclaves force small, quantized models and high costMinorOpen questionOpen

    Memory limits required 4-bit quantization, which lowered zero-shot MMLU accuracy from 54.6% to 51.4%. CPU inference cost 21.7 times as much per token as GPU inference, and the enclave roughly doubled the CPU cost 1. The authors wrote that H100 confidential computing had no multi-GPU support 1. NVIDIA's white paper of August 2025 describes a protected-PCIe mode that passes all eight GPUs of a Hopper HGX node to one confidential VM, with NVLink traffic unencrypted 2.

Blockers

  • The prototype needs porting to GPU confidential computing to handle larger models; the authors expect an overhead as small as 5 times there.

    Performance & compatibilityBlocked by: TEE remote attestation for AI workloads[1]

  • As of September 2026 no code has been released for the prototype.

    Adversarial validation[1]

Technical detail

Show technical detail
  • Security goals. G1 model verifiability, G2 audit verifiability, G3 confidentiality of model IP and audit data, G4 transparency of artifacts, G5 statelessness, and G6 output verifiability 1.
  • Adversaries. A1 is a network adversary that can intercept, tamper with or spoof messages, with denial of service excluded. A2 is a physical or privileged adversary able to take RAM snapshots, roll back VMs and run side-channel attacks 1.
  • PREPARE. The enclave generates a KEM key pair and attests its boot image. The provider sends the encrypted model, which the enclave quantizes and hashes. The enclave publishes an attestation linking model and quantized model to a transparency log 1.
  • ATTESTABLEAUDIT. The provider and the auditor send the encrypted model and the encrypted audit code and data. The enclave runs the audit in a sandbox and publishes an attestation binding model, audit and result 1.
  • INFERENCE. The enclave loads the earlier attestations, checks that the provider's model hash matches, and returns each encrypted response with an attestation linking model, prompt, output and audit result 1.
  • Verification. Verifiers check platform configuration registers (PCRs) against known images and check signatures with the TEE vendor's key or attestation service. Including the base image in the measurement allows revocation when vulnerabilities are found 1.
  • Prototype. The implementation is written in Rust with bindings to llama.cpp. A 4-core enclave on an m5.2xlarge instance is compared with CPU baselines on m5.xlarge and m5.2xlarge instances and an NVIDIA L40S GPU baseline. Enclave throughput was 1.84 tokens/s at $5.80 per 100K tokens, against 202 tokens/s and $0.12 on the GPU baseline. Zero-shot MMLU accuracy was 51.4% at 4-bit (57.4% excluding unparsable responses), against 54.6% for the unquantized model on the GPU. Copying models into the enclave took at most 2 minutes 1.

Sources

  1. BC. Schnabl et al. (2025). Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments. ICML 2025 Workshop on Technical AI Governance. Source recordSupports: all protocol, prototype, results and limitations statements · §2 (background, GPU CC), §3 (goals, adversaries, Algorithms 1-3), §5 (Table 2), §7 (limitations)
  2. BNVIDIA (2025). NVIDIA Secure AI with Blackwell and Hopper GPUs (White Paper). NVIDIA documentation. Source recordSupports: Hopper protected-PCIe multi-GPU mode with unencrypted NVLink · p. 13

I-0007JSONSource-checked 2026-09-23 · changed 2026-09-24Suggest a correction

Drafted with AI assistance and checked against the cited sources by an independent verifier. Expert review pending. Readiness assessed by the AI draft on 2026-09-23. How records are made.