AI Verification LandscapeSearch

Claim · Negative

Model weights or data have not left the facility

No copy of specified model weights or sensitive data has left a designated facility through networks, physical media or other channels.

Agreements and security commitments may require that certain model weights stay inside a secured facility, for example to protect them against theft by well-resourced attackers. Verification itself creates a related concern, since monitoring equipment could leak the prover's secrets. Verifying that weights have not left is a negative claim over every possible channel: network links, physical media, people and physical side channels. Approaches include checking that outgoing traffic is explained by legitimate computation so that hidden data stands out, capping the bandwidth of external links, suppressing side channels, and physical security with inspection. Checking inference outputs against a reference cut the information an attacker could hide in responses to under 0.5% in one test; in an independent study, an attacker who controlled the prompts roughly doubled the leak. Physical and insider channels remain harder to bound.

State of verificationeditors' synthesis

Only the network channel has a demonstrated (R2) bound, and only for inference outputs. Physical side channels rely on proposed (R1) designs, and insiders and physical media on facility security.

Bounding unexplained information (R2) measures how much of a facility's output the declared computation cannot explain. Sampled recomputation (R2) supplies the reference, deterministic inference (R2) would remove the tolerance that hidden data can exploit 10, and network taps (R1) record the traffic. Caps on external bandwidth (Communication between compute groups is bounded) bound the total that can leave 9. Side-channel suppression (R1) targets physical leakage 6, and RAND's secure inference data centre (R1) is designed to protect weights against state-backed attackers 11.

Checking inference outputs cut the information an attacker could hide to under 0.5% on a 30-billion-parameter model, at a false-positive rate below 0.01% 2. In an independent study, an attacker who controlled the prompts roughly doubled the leak per token 3.

People and physical media are left to facility security 1 and personnel measures such as whistleblowing 7. RAND's design does not describe how an outside party would verify it 11. The designs aim for a stated bound on leakage, not an absolute guarantee 5.

Mechanisms

Why it matters

Model weights are a primary target for theft 1.

  • Security of frontier models. RAND researchers write that protecting frontier models from theft and misuse will become more important as they grow more capable 1. They identify 38 distinct attack vectors, and define five security levels for defending against actors ranging from opportunistic criminals to well-resourced nation-states 1.
  • Agreements. Scher and Thiergart describe one approach as using strong security to prevent model weights from leaving a data centre, and then monitoring that data centre closely 4.
  • Verification itself. RAND's framework treats confidentiality, meaning protection of models, data and code from unauthorised access and especially theft, as a requirement on verification 7. A low-trust system overview assumes the verifier may try to exfiltrate the prover's secrets 5. It therefore sends only cryptographic commitments out of the facility, and checks challenged data only in an auditing environment inside the prover's facility 5.

Why it is hard

The claim must hold for every channel. The RAND weight-security study's 38 attack vectors span far more than network transfer 1.

  • Hidden data in ordinary outputs. An attacker who controls an inference server could hide weights inside ordinary model responses using steganography 2. Rinberg and colleagues formalise this as a security game and verify outputs against a reference 2. On a 30-billion-parameter mixture-of-experts model, their detector reduced exfiltratable information to under 0.5% at a false-positive rate below 0.01%, which they equate to a slowdown of more than 200 times for an adversary 2. An independent study found that an attacker who chooses the prompts roughly doubles the bits leaked per token, cutting the slowdown to 60–118 times 3.
  • Capping egress. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB 9. Bounding a facility's links in this way is the subject of Communication between compute groups is bounded.
  • Explaining all egress. Making all egress traffic explainable by its ingress is an open design question, including which engineering compromises it would require 5. Active network taps on front-end links could scrub headers to reduce covert channels 8.
  • Physical side channels. These offer routes around network monitoring altogether. Proposed defences include shielding, jamming, inspection and filtering 6, with a design target of kilobits per second rather than zero 5. Residual copies in memory are addressed by memory wiping 5.
  • Insiders and physical media. These are outside the reach of network-level mechanisms. RAND's verification framework includes whistleblower programmes and personnel interviews among its layers 7, and the weight-security study treats defence as requiring comprehensive measures rather than isolated ones 1.

Sources

  1. BS. Nevo et al. (2024). Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models. RAND Corporation. Source recordSupports: importance of protecting frontier weights; 38 attack vectors; five security levels; adversaries up to nation-states; comprehensive defences · summary
  2. BR. Rinberg et al. (2025). Verifying LLM Inference to Detect Model Weight Exfiltration. arXiv. Source recordSupports: steganographic exfiltration via inference server responses; security game; <0.5% exfiltratable information at <0.01% FPR on MoE-Qwen-30B; >200x adversary slowdown · abstract
  3. BN. Kezins (2026). Adversarial Entropy Inflation Against Gumbel-Based Inference Verification. arXiv. Source recordSupports: prompt-controlling adversary roughly doubles bits leaked per token; slowdown falls to 60–118x · abstract; results
  4. BA. Scher & L. Thiergart (2025). Mechanisms to Verify International Agreements About AI Development. arXiv. Source recordSupports: strong security to keep weights in a data centre plus close monitoring · data-centre security discussion
  5. BN. Cankaya (2026). A System Overview for Near-Term, Low-Trust AI Compute Verification. Machine Intelligence Research Institute. Source recordSupports: verifier aims to exfiltrate prover secrets; only commitments leave the facility; egress explainable by ingress; memory wiping; side-channel target · threat model; architecture; open problems
  6. CN. Cankaya (2026). Suppressing Side Channels in an Untrusted Data Center via Retrofitted Defenses. MIRI Technical Governance Team. Source recordSupports: physical side channels can bypass network monitoring; defences · channels of concern; defences
  7. BM. Baker et al. (2025). Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment. RAND Corporation. Source recordSupports: confidentiality as protecting models, data and code from theft; whistleblower and interview layers · §1; §4
  8. CN. Cankaya (2026). The Fundamentals and Feasibility of Secure Network Taps for Verifying AI Datacenter Use. The Datacenter Lie Detector. Source recordSupports: active taps scrubbing headers against covert channels on front-end links · frontend vs backend
  9. BR. Rinberg et al. (2026). Haiku to Opus in Just 10 bits: LLMs Unlock Large Compression Gains. arXiv. Source recordSupports: egress limits cap what can be stolen · §5.1
  10. BN. Cankaya (2026). Bit-Exact AI Inference Verification Without Performance Tradeoffs. ICML 2026 Workshop on Technical AI Governance Research. Source recordSupports: approximate output matching leaves degrees of freedom, including steganography, that covert adversaries can exploit · abstract
  11. BS. F. Comer et al. (2026). Highly Secure Inference Data Centers: A Vertically Integrated Strategy for Security Engineering. RAND Corporation (Research Report RR-A4827-1). Source recordSupports: secure inference data centre design against state-backed attackers; no external verification path described · Summary; ch. 1