This compute runs inference, not training

A declared cluster is used only to run existing models to produce outputs, and not to train new or more capable models.

Several agreement proposals would let existing AI models keep serving users while restricting further training. That requires verifying that a data centre runs inference only. The claim combines a positive part (the declared inference is happening) with a negative part (nothing else, especially training, is). It matters because it could let most AI compute stay in productive use under a training restriction. It is hard because the same chips can do both, workloads can be disguised, and the negative part requires accounting for all of a facility's activity. Proposed approaches include capturing and recomputing the traffic that enters and leaves a facility, limiting bandwidth between groups of chips so that large training cannot be coordinated, and classifying workloads from telemetry. Each rests on open assumptions about numerical nondeterminism, hidden capacity and side channels.

State of verificationeditors' synthesis

Declared inference can be checked with demonstrated (R2) components, but the absence of training cannot yet be verified. That half rests on full-stack designs (AI 2040 inference-only verification stack, Low-trust AI compute verification system overview, SASH confidential network logger) that are proposed (R1) and not built beyond small prototypes.

Network taps (R1) capture front-end traffic, and sampled recomputation (R2) checks it against the declared model; reproducible computation packets (R1) would extend this to whole workloads. Bandwidth limits, memory wiping (both R1) and bounding unexplained information (R2) constrain what else could run or leave. Workload classification and on-chip telemetry (both R2) are lighter-touch alternatives.

Recomputation tolerates numerical noise on open-weight models of 8 to 30 billion parameters ¹¹. A telemetry classifier detects training with 98.2% accuracy on its own corpus, but 43–87% on the most challenging disguised workloads held out from its training ⁸. A team building the components reports nothing past a proof-of-principle prototype ¹³.

Training traffic runs on back-end fabric that is harder to tap, and whether all outgoing traffic can be explained by incoming traffic is an open question ⁵. Telemetry is trustworthy only with protections, such as a tamper-resistant read path, that current GPUs lack ⁸ ¹⁴.

Mechanisms

R2On-chip telemetry from timing, memory and performance counters⚠primary
Classifying training versus other workloads from counters ³ or timing ¹.
R2Workload classification from telemetry and side channelsprimary
Classifies observed activity as training, inference or non-ML work.
R1AI 2040 inference-only verification stackprimary
The stack's stated purpose: retrofitted data centres run only inference.
R1Low-trust AI compute verification system overviewprimary
Challenged records are screened for inference versus training.
R1Network taps and certifiersprimary
Supplies the committed record of cluster I/O that recomputation checks against declared inference ⁴ ¹.
R1Reproducible computation packetsprimary
Proposed as the correctness check for an inference-only retrofit ¹.
R1SASH confidential network loggerprimary
SASH describes the aim as distinguishing inference from training in data centres ¹.
R2Bounding unexplained information in outputssupporting
In the compute-agreement framing, outputs must be predictable by policy-compliant computation, which limits what undeclared workloads can export ¹.
R2Deterministic and bit-exact inferencesupporting
Bit-exact recomputation of declared inference removes the tolerance an operator could hide other work in ¹.
R2Sampled inference recomputationsupporting
Proposed as the correctness check in inference-only retrofit plans. Completeness needs other mechanisms ⁶ ³.
R2Tamper evidence for verifier devicessupporting
Protects the integrity of taps, gateways and recomputation hardware used for inference-only verification ⁹ ¹⁰.
R2TEE remote attestation for AI workloads⚠supporting
Attests a declared inference deployment, but not that the same chips ran no other workloads ¹¹.
R2Zero-knowledge proofs of inferencesupporting
Attestable proposes using proofs to show accounted workloads used an approved, unchanged model.
R1Attestable zero-knowledge inference proversupporting
Proposed use: showing an accounted workload used an approved, unchanged model.
R1Bandwidth limits and compartmentalizationsupporting
Intended to leave inference workable while making large training impractical ¹ ².
R1Memory wiping and proofs of secure erasuresupporting
Periodic wipes are proposed so that only verified inference outputs persist ¹; Amodo frames wipes as verifying completeness of declared workloads ⁵.
R1Proofs of useful work and resource exhaustionsupporting
Attestable's pacing proposal pairs work accounting with ZK inference proofs.
R1RAND secure inference data center (SIDC) designsupporting
Scoped to serving already-trained models.
R1Side-channel suppression for isolated facilitiessupporting
Inference-only designs count on suppressing unmonitored physical channels so that all significant traffic passes the taps ².
R1Timed challenge-response and memory-occupation challengessupporting
Listed as an alternative inference-verification direction that may not need a hardware retrofit ²; bounds spare memory ¹.

Why it matters

Several proposals to restrict frontier AI development target training while preserving the use of existing models ¹ ². Sastry and colleagues note that most AI compute is now used for inference ³.

A draft international agreement restricts the scale of AI training. Its chip-use verification is meant to distinguish inference on existing systems from the training of new models ¹.
The AI 2040 verification plan would convert data centres to inference-only operation, preventing training runs while models continue to serve users ².
RAND's framework treats accurate declaration of AI inference as its own verification subgoal, separate from declared training ⁴. It lists deterministic replication of neural-network inference as a research problem ⁴.
A low-trust system overview names distinguishing inference from training, and deploying only approved models, among its core objectives ⁵.

Why it is hard

The same hardware can do both. Scher and Thiergart expect that some current inference-specialised chips could be repurposed for training without significant effort ⁶. Shavit notes that there is no straightforward way to determine whether an ML chip is running a training job or an unrelated one ⁷.
Classifiers invite evasion. The Open Problems survey notes that adversarial customers may obfuscate their activities, for example by adding noise to how they use computational resources ⁹. One adversarial study of GPU-telemetry classifiers reports 98.2% accuracy at identifying training across its corpus, but 43–87% on the most challenging disguised workloads held out from its training ⁸.
Training traffic is harder to observe. Inference produces token-level input and output data on the front-end links between a data centre and its users ⁵. Training traffic runs over back-end fabric that has much higher bandwidth, is latency-sensitive, and is harder to tap ⁵. Cankaya describes front-end tapping as the most viable option, and notes that back-end tapping may require sampling rather than full capture ¹⁰. The system overview leaves open how far a facility can be required to make all egress traffic explainable by its ingress ⁵.
Recomputation must cope with numerical noise. Checks on sampled outputs handle it either statistically ¹¹ or by exact reproduction ¹².
Bandwidth limits may erode. Limits between pods of chips could prevent the gradient exchange that distributed training needs ⁶. Sastry and colleagues note that more viable decentralised training could undermine the detectability of training ³.
Hidden capacity and channels. The negative part of the claim requires that no capacity or channel is hidden. The system overview addresses this with memory wiping and side-channel suppression, and lists open problems for both ⁵.

Sources

BA. Scher et al. (2025). An International Agreement to Prevent the Premature Creation of Artificial Superintelligence. Machine Intelligence Research Institute. Source recordSupports: restricting the scale of training; chip use verification distinguishing inference on existing systems from training · abstract; Article VII (as summarised)
CR. Dean (2026). Verification Plan. AI 2040. Source recordSupports: data centres converted to inference-only operation; network taps, recomputation and reproducible packets · phases; verification mechanisms
BG. Sastry et al. (2024). Computing Power and the Governance of Artificial Intelligence. arXiv. Source recordSupports: majority of AI compute used for inference; decentralised training could undermine detectability · training vs inference; limitations
BM. Baker et al. (2025). Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment. RAND Corporation. Source recordSupports: declared inference (1.A.2) as a distinct subgoal; deterministic replication of inference as an R&D problem · §3.2; Appendix A.9
BN. Cankaya (2026). A System Overview for Near-Term, Low-Trust AI Compute Verification. Machine Intelligence Research Institute. Source recordSupports: distinguishing inference from training; token-level front-end evidence; back-end harder to tap; egress explainable by ingress as open question; memory wiping; side channels · verification goals; inference vs training; open problems
BA. Scher & L. Thiergart (2025). Mechanisms to Verify International Agreements About AI Development. arXiv. Source recordSupports: inference-specialised chips repurposable for training; pods with limited external bandwidth · Verifying that known compute is not being used for a large training run
BY. Shavit (2023). What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring. arXiv. Source recordSupports: no straightforward way to tell whether a chip is running training or another job · open problems
BR. Rahman & S. Tajdari (2026). Detecting Hidden ML Training With Zero-Overhead Telemetry. ICML 2026 Workshop on Technical AI Governance Research. Source recordSupports: telemetry classifier accuracy overall and on adversarially disguised workloads; required telemetry protections · abstract; §5.2; deployment requirements
AA. Reuel et al. (2025). Open Problems in Technical AI Governance. Transactions on Machine Learning Research. Source recordSupports: workload classification; adversarial customers may obfuscate by adding noise · §3.2.2 / §5.2.2 open problems
CN. Cankaya (2026). The Fundamentals and Feasibility of Secure Network Taps for Verifying AI Datacenter Use. The Datacenter Lie Detector. Source recordSupports: front-end tapping most viable; back-end requires sampling · frontend vs backend
BA. Karvonen et al. (2025). DiFR: Inference Verification Despite Nondeterminism. arXiv. Source recordSupports: recomputation despite benign numerical noise on 8–30B open-weight models · abstract; §5
BN. Cankaya (2026). Bit-Exact AI Inference Verification Without Performance Tradeoffs. ICML 2026 Workshop on Technical AI Governance Research. Source recordSupports: bit-exact inference verification across GPU variants · abstract
CT. Milton et al. (2026). Verifying international AI deals: Plan A, the state-of-play, and what you can do to help. Amodo (Substack). Source recordSupports: no verification component past a proof-of-principle prototype · introduction
BS. K. Monfared et al. (2026). Timing and Memory Telemetry on GPUs for AI Governance. arXiv. Source recordSupports: current GPUs expose limited trusted telemetry · abstract