AI Verification LandscapeSearch

Mechanism · Cryptographic & computational

Model identity attestation

Establishes that responses come from a specific, committed set of model weights, using enclave measurements or recomputation of sampled outputs.

Also called Model integrity verification; Proving which model is served; Weight commitment and attestation; Model provenance attestation

R2Demonstrated⚠ Open critical flawProvider-reported

Summary

Users, auditors and regulators often need to know that a provider is serving the model it declared, not a cheaper, altered or unevaluated one. There are two routes. First, the weights are committed to a hash, and the attestation from a trusted execution environment shows that the measured software enforced that commitment while the model ran. Second, a verifier that holds the declared weights recomputes a sample of logged outputs, which can also detect weights smuggled out in responses. Tinfoil reports running the enclave route in a commercial service, and its Modelwrap tool is open source. Research prototypes bind evaluations and inference to model hashes, and recomputation has been tested on models of up to 30 billion parameters. The enclave route inherits the weaknesses of TEEs, including published attacks that forge Intel TDX and AMD SEV-SNP attestations. Recomputation needs trusted logging and randomness, and must tolerate numerical nondeterminism.

ReadinessR2Demonstratedmedium confidence · rubric v1.0 · 2026-09-23

R2. Public code runs on production hardware, but the independent attacks published so far left flaws open, including a critical flaw in the trusted execution environments the enclave route relies on.

Rubric assessment
  • R1 met: designs and assumptions are published for both routes 1 2 4.
  • R2 met through Tinfoil's Modelwrap chain. Its code is open source 14, it runs on NVIDIA H100, H200 or B200 GPUs with AMD SEV-SNP or Intel TDX (provider-reported) 11, and it has been reported on models of up to 554 GB 1. Rinberg et al. publish code and results on models from 3B to 30B parameters 4.
  • R3 not met. Criterion (a) is met only on the provider's account: Tinfoil reports offering the feature in a production service 12 13. Criterion (b) fails, because the independent evaluations that exist left flaws open. TEE.fail used physical access to forge the Intel TDX attestation that the enclave route relies on. By pairing the forgeries with relayed H100 attestations, it made a vLLM proxy running outside TEE protection pass both the TDX and the NVIDIA confidential-computing checks 7. That flaw is critical and open. Kezins, at Delft University of Technology, attacked the exfiltration bound of Rinberg et al.'s recomputation check with chosen prompts. On a 30B mixture-of-experts model this raised leakage from 0.119 to 0.286 bits per token. The attack widens the covert channel and does not target the check that outputs match the declared model 5.
Gaps to the next level
  • Independent security evaluation of a deployed model-identity scheme that leaves no critical flaw open.
  • Resistance of the enclave variant to physical attackers (see TEE remote attestation for AI workloads).
  • Third-party verification for private models beyond consistency across requests.
  • Tooling for audit-time checking of transparency records.

How it works

Two routes exist, and they can be combined.

Commit and attest

This route runs inside a TEE (TEE remote attestation for AI workloads). It closes a gap: attestation "measures launch state, not runtime state", while weights are read from disk after boot 1. Three designs bind the weights to the attestation:

  • Tinfoil's Modelwrap packs the weights into a read-only image and computes a Merkle-tree root hash. It places that hash on the kernel command line, which the enclave measurement covers, and the kernel's dm-verity layer checks every block read against it 1. Clients compare the attestation's measurements with values that Tinfoil's build pipeline publishes to the Sigstore transparency log 12. For a public model, anyone can rebuild the image from the same Hugging Face commit and compare root hashes 1.
  • PAL*M puts hashes of each operation's inputs, including the model, and of its outputs into the Intel TDX report, for single prompts and multi-turn sessions 2.
  • Attestable Audits records the model hash when an audit runs. At inference time it checks that the served model matches, then returns each response with an attestation that links model, prompt, output and audit result 3.

Recompute and compare

This route uses sampled recomputation. A verifier that holds the declared weights scores logged outputs by the likelihood that each token was sampled from that model under a known seed. The estimators allow for legitimate numerical nondeterminism 4. Rinberg et al. frame the check as a way to catch weights hidden steganographically in responses, and it also shows whether outputs are consistent with the declared model 4. Deterministic inference is covered in Deterministic and bit-exact inference.

What it establishes

The two routes establish different things:

  • Commit and attest can show that the bytes served match a commitment 1.
  • Recompute and compare can show that logged outputs are consistent with the declared model's sampling procedure 4.

Neither shows:

  • What a private model can do. With unpublished weights, a user can confirm only that the same weights are served each time 1. An attested evaluation bound to the same hash can close part of that gap 3.
  • Everything else that shapes behaviour. Tinfoil measures the firmware, kernel, initrd and a configuration file as well as the weights 12. Gloria Z warns that unmeasured runtime flags can undermine integrity 6.
  • Properties a weak hashing scheme leaves out. Gloria Z also warns that a scheme that sequences values loosely, or checks the weights too early in loading, could let two models with significantly different properties share a hash "without breaking the hash function itself" 6.

Threat model

The enclave route trusts the hardware vendor and treats the host software as hostile. Tinfoil states "You trust the hardware manufacturer", but not the hypervisor or host OS 11. PAL*M excludes side channels and physical attacks 2.

The recomputation route assumes the following 4:

  • the verification code is trusted;
  • every input, output and seed is logged correctly;
  • the attacker can neither predict nor manipulate which messages are sampled for verification;
  • the verifier sees prompts and responses and holds the weights.

Evidence

  • Tinfoil. Modelwrap is published under an MIT license 14. Tinfoil reports a storage overhead of 0.8%, build times of up to 13 min 25 s for a 554 GB model, and slower cold loading but no inference slowdown 1.
  • PAL*M. It reports inference attestation on an H100 across three models, with total observed overhead of 3.8–11.4% for multi-turn sessions and 45.5–66.4% for single prompts 2.
  • Attestable Audits. Its protocol binds inference to the audited model hash. The reported evaluation covers the audit step, for a 4-bit Llama-3.1-8B on CPU-only AWS Nitro Enclaves 3.
  • Rinberg et al. They tested Llama-3.1-8B, Llama-3.2-3B and two Qwen mixture-of-experts models, and publish their code. On the 30B model under benign prompt traffic, the detector cut exfiltratable information to under 0.5% at a false-positive rate under 0.01% 4. An independent study found that prompt control roughly doubles leakage per token 5.

Limitations

  • Inherited TEE attacks. Using physical access, TEE.fail forged Intel TDX attestations and, by pairing them with relayed H100 attestations, passed a workload outside TEE protection 7. Battering RAM and RMPocalypse forged AMD SEV-SNP attestations, the second without physical access 8 9. Tinfoil's documentation acknowledges physical-access and side-channel risks 11.
  • Audit-time checking. Tinfoil does not provide a supported tool for querying its transparency records at audit time 13.
  • Recomputation slack. Nondeterminism concentrates at a few token positions, and very slow leaks remain possible 4. An adversary who controls prompts widens the leak 5.
  • Recomputation dependencies. The check depends on complete and trusted logging, and on the verifier holding the weights 4.

Known flaws

Published flaws, with their severity, kind and status. How flaws are rated.

  • Underlying attestation can be forged or relayedCriticalDemonstrated attackOpen

    The enclave variant is only as sound as the attestation. With physical access and root privileges, TEE.fail extracted a per-CPU Intel attestation signing key and forged TDX attestations. Pairing the forgeries with genuine H100 attestations relayed from a rented server, the researchers made a workload outside TEE protection pass both checks 7. Other independent research forged AMD SEV-SNP attestations. Battering RAM did so with a DDR4 memory interposer, and RMPocalypse did so from a malicious hypervisor without physical access 8 9. AMD reports firmware fixes for RMPocalypse 10. Tinfoil's documentation lists physical attacks among the limitations of enclaves 11.

  • Launch-state attestation does not by itself cover weights loaded laterSignificantTheoretical argumentMitigated

    Attestation measures launch state, and weights are read from disk after boot. A signature checked at load time does not stop a malicious hypervisor from altering the disk afterwards 1. Tinfoil reports mitigating this with dm-verity checks on every read 1. Unmeasured runtime configuration remains a general risk 6.

  • For private models, a user can confirm consistency but not contentSignificantOpen questionOpen

    When weights are not published, users can check that the same root hash is served each time, but not what the model is 1. Pairing the hash with an attested evaluation, as in Attestable Audits, is one proposed remedy 3.

  • Recomputation depends on trusted logging and randomness, and its tolerance leaves a covert channelSignificantDemonstrated attackOpen

    The recomputation variant assumes that every input, output and seed is logged correctly, and that the attacker can neither predict nor manipulate which messages are sampled for verification. Legitimate nondeterminism concentrates at a few token positions, and slow leaks within the tolerated slack remain possible 4. An independent study showed that an adversary who controls the prompts roughly doubles the bits leaked per token, reducing the exfiltration slowdown from more than 200 times to 60–118 times. The attack targets the exfiltration bound, not the check that outputs match the declared model 5.

Blockers

Technical detail

Show technical detail
  • Modelwrap build (Tinfoil's description). The tool downloads a pinned model revision, normalizes its directory structure and builds an EROFS image. It computes a dm-verity root hash with veritysetup. The Merkle tree gives a 32-byte commitment to, for example, 140 GB of weights 1.
  • Binding and enforcement. The root hash is passed on the kernel command line, which the enclave measurement covers. dm-verity then checks every block read against it and returns an error on any mismatch 1.
  • Reported costs. Tinfoil reports a hash-tree overhead of about 0.8% of image size, and build times from 5 s for a 549 MB model to 13 min 25 s for a 554 GB model. Cold-cache loading takes about 80% longer, and inference is unaffected once the weights are in GPU memory 1.
  • PAL*M inference attestation. PAL*M sets the Intel TDX REPORTDATA field to the concatenation of the operation, a challenge, and hashes of the inputs and outputs. It supports single-prompt and multi-turn inference attestations. Across three models on an H100, total observed overhead was 3.8–11.4% for multi-turn sessions and 45.5–66.4% for single prompts 2.
  • Fixed-seed sampling likelihood. Rinberg et al. score the likelihood that each token was honestly sampled from the claimed model under a known seed, with estimators for inverse-probability-transform and Gumbel-max samplers. On a 30B mixture-of-experts model under benign prompt traffic, the check limits exfiltratable information to under 0.5% at a false-positive rate under 0.01%, a slowdown of more than 200 times for the adversary 4. An independent study found that an adversary who controls the prompts roughly doubles leakage per token, reducing the slowdown to 60–118 times 5.

Sources

  1. CTinfoil Team (2026). How Tinfoil Proves Exactly What Model Is Running. Tinfoil. Source recordSupports: Modelwrap design, launch-state problem, signing comparison, private models, overheads (provider-reported) · sections on the challenge, the three phases, performance, private models
  2. BP. Chantasantitam et al. (2026). PAL*M: Property Attestation for Large Generative Models. arXiv. Source recordSupports: inference attestation binding hashes to TDX report; overheads · §4, Table 6
  3. BC. Schnabl et al. (2025). Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments. ICML 2025 Workshop on Technical AI Governance. Source recordSupports: audit-to-inference model-hash binding; prototype evaluation of the audit step · §3, Algorithms 1-3, §5
  4. BR. Rinberg et al. (2025). Verifying LLM Inference to Detect Model Weight Exfiltration. arXiv. Source recordSupports: recomputation-based verification, assumptions, results, limitations, code · Abstract; §1, §3.3-3.4, §4.2, §5-§7
  5. BN. Kezins (2026). Adversarial Entropy Inflation Against Gumbel-Based Inference Verification. arXiv. Source recordSupports: independent prompt-control attack that widens the exfiltration bound; 0.119 to 0.286 bits per token on the 30B MoE model · Abstract; Table 1
  6. CGloria Z (2026). On TEEs for Privacy-Preserving Monitoring in AI Governance. MIRI Technical Governance Team. Source recordSupports: measurement incompleteness; hashing-scheme warning
  7. AJ. Chuang et al. (2026). TEE.fail: Breaking Trusted Execution Environments via DDR5 Memory Bus Interposition. 2026 IEEE Symposium on Security and Privacy (SP). Source recordSupports: Intel TDX attestation forgery and H100 attestation relay to a vLLM proxy outside TEE protection · Abstract; §1.1, §8.3
  8. AJ. De Meulemeester et al. (2026). Battering RAM: Low-Cost Interposer Attacks on Confidential Computing via Dynamic Memory Aliasing. 47th IEEE Symposium on Security and Privacy (S&P 2026). Source recordSupports: SEV-SNP attestation breach with a DDR4 interposer (Battering RAM) · Abstract; site FAQ
  9. AB. Schlüter & S. Shinde (2025). RMPocalypse: How a Catch-22 Breaks AMD SEV-SNP. 2025 ACM SIGSAC Conference on Computer and Communications Security (CCS '25). Source recordSupports: software-only SEV-SNP attestation forgery by a malicious hypervisor (RMPocalypse) · Abstract; site
  10. BAMD (2025). SEV-SNP RMP Initialization Vulnerability (AMD-SB-3020). AMD product security bulletin. Source recordSupports: AMD firmware fixes for RMPocalypse (vendor-reported) · Mitigation tables
  11. BTinfoil (2026). A primer on secure enclaves. Tinfoil documentation. Source recordSupports: Tinfoil hardware, trust model and documented limitations (provider-reported)
  12. BTinfoil (2026). Backend infrastructure. Tinfoil documentation. Source recordSupports: measured boot chain, Sigstore publication, production model volumes (provider-reported)
  13. BTinfoil (2026). How verification works in Tinfoil. Tinfoil documentation. Source recordSupports: production deployment; no supported audit-time tool (provider-reported) · In-band vs. out-of-band verification
  14. BTinfoil (2026). modelwrap: Reproducible dm-verity read-only image of Huggingface models. GitHub. Source recordSupports: open-source implementation, release v0.3.0

M-0012JSONSource-checked 2026-09-23 · changed 2026-09-24Suggest a correction

Drafted with AI assistance and checked against the cited sources by an independent verifier. Expert review pending. Readiness assessed by the AI draft on 2026-09-23. How records are made.