Claim · Mixed

This compute runs inference, not training

The claim is that a declared cluster is used only to run existing models to produce outputs, not to train new or more capable models.

Several agreement proposals would let existing AI models keep serving users while restricting further training. The claim combines a positive part (the declared inference is happening) with a negative part (nothing else, especially training, is). It matters because it could let most AI compute stay in productive use under a training restriction.

It is hard because the same chips can do both, workloads can be disguised, and the negative part requires accounting for all of a facility's activity. Proposed approaches include capturing and recomputing the traffic that enters and leaves a facility, limiting bandwidth between groups of chips so that large training cannot be coordinated, and classifying workloads from telemetry. Each rests on open assumptions about numerical nondeterminism, hidden capacity and side channels.

State of verification

Declared inference can be checked with components at research demonstration (R2). Sampled recomputation and deterministic inference are in production (R3) for checking untrusted inference providers. No production use for a declared cluster is documented. Ruling out training cannot yet be verified, because it needs the rest of the cluster accounted for. Full-stack designs that would do so (AI 2040 inference-only verification stack, Low-trust AI compute verification system overview, SASH confidential network logger) are proposed (R1). The Amodo team, which is building verification components, reports that none of the core components is past a proof-of-principle prototype ¹³.

For example, a verifier may want to know that a cluster declared for inference is not training a new model.

Network taps (R1) record the cluster's front-end traffic with its users, and sampled recomputation re-runs sampled requests on the declared model. DiFR (R2) tolerates numerical noise in these re-runs on open-weight models of 8 to 30 billion parameters ¹¹.
The Singapore AI Safety Hub's network logger (R1) is a public prototype of both. It passes every request through a logger and re-runs it on a separate cluster, with a 270-million-parameter model and no stated adversary ¹⁷.
Network taps and sampled recomputation cover at most the claim's positive half, that the declared inference is happening. They show that sampled outputs match the declared model. A tap on the cluster's external links does not stop covert workloads. It aims only to stop their results leaving over those links ¹⁸.
For the negative half, training traffic runs on back-end fabric that is harder to tap ⁵. Bandwidth limits (R2 for software monitoring) between groups of chips target that fabric, since distributed training must exchange gradients ⁶. Reuter and colleagues specify proofs of compartmentalization (PoComps), protocols that combine gateway records, periodic memory erasure and sampled proofs of small deterministic tasks to constrain frontier training ¹⁹. The paper notes that training very small models can still fit within their limits, and it reports no end-to-end PoComp implementation ¹⁹. The company Attestable proposes proofs of useful work (R1) to keep declared hardware busy with approved or protocol-defined work, leaving little spare capacity ¹⁵.
Known routes remain. Reinforcement-learning rollouts are inference, so declared servers could generate them while hidden compute updates the model ¹⁶. In one scenario, a March 2026 Alignment Forum post by joshc gives three alternative countermeasures ¹⁶. One is proof of work or memory that accounts for more than 95% of computation ¹⁶. Workload classification (R2) is a lighter alternative. It detects training with 98.2% accuracy on its own corpus, but 43–87% on the most challenging disguised workloads held out from its training, and current GPUs lack the protections its telemetry needs to be trusted ⁸ ¹⁴.

On this page

Why it matters

Several proposals to restrict frontier AI development target training while preserving the use of existing models ¹ ². Sastry and colleagues note that most AI compute is now used for inference ³.

A draft international agreement restricts the scale of AI training. Its chip-use verification is meant to distinguish inference on existing systems from the training of new models ¹.
The AI 2040 verification plan would convert data centres to inference-only operation, preventing training runs while models continue to serve users ².
RAND's verification framework treats accurate declaration of AI inference as its own verification subgoal, separate from declared training ⁴. It lists deterministic replication of neural-network inference as a research problem ⁴.
A low-trust system overview names distinguishing inference from training, and deploying only approved models, among its core objectives ⁵.

Why it is hard

The same hardware can do both. Scher and Thiergart expect that some current inference-specialised chips could be repurposed for training without significant effort ⁶. Shavit notes that there is no straightforward way to determine whether an ML chip is running a training job or an unrelated one ⁷.
Classifiers invite evasion. The Open Problems in Technical AI Governance survey notes that adversarial customers may obfuscate their activities, for example by adding noise to how they use computational resources ⁹. One adversarial study of GPU-telemetry classifiers reports 43–87% accuracy on the most challenging disguised workloads held out from its training ⁸.
Training traffic is harder to observe. Inference produces token-level input and output data on the front-end links between a data centre and its users ⁵. Training traffic runs over back-end fabric that has much higher bandwidth, is latency-sensitive, and is harder to tap ⁵. In a research note on secure network taps, Cankaya describes front-end tapping as the most viable option, and notes that back-end tapping may require sampling rather than full capture ¹⁰. The system overview leaves open how far a facility can be required to make all egress traffic explainable by its ingress ⁵.
Recomputation must cope with numerical noise. Checks on sampled outputs handle it either statistically ¹¹ or by exact reproduction ¹².
Bandwidth limits may erode. Limits between pods of chips could prevent the gradient exchange that distributed training needs ⁶. Sastry and colleagues note that more viable decentralised training could undermine the detectability of training ³.
Hidden capacity and channels. The negative part of the claim requires that no capacity or channel is hidden. The system overview addresses this with memory wiping and side-channel suppression, and lists open problems for both ⁵.
Inference can be part of training. A March 2026 Alignment Forum post by joshc analyzes a covert reinforcement-learning strategy that generates rollouts on declared inference servers while updating the model on hidden compute ¹⁶. Observing inference outputs alone would not rule it out.