{
  "schema_version": "1.0.0",
  "rubric_version": "1.0",
  "license": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
  "record": {
    "id": "C-0004",
    "slug": "inference-not-training",
    "title": "This compute runs inference, not training",
    "aliases": [
      "inference-only verification",
      "training vs inference"
    ],
    "status": "draft",
    "last_reviewed": "2026-09-23",
    "review_interval_days": 90,
    "steward": null,
    "provenance": {
      "drafted_by": "ai",
      "reviewed_by": []
    },
    "risk_flags": [],
    "flags": [],
    "one_liner": "A declared cluster is used only to run existing models to produce outputs, and not to train new or more capable models.",
    "summary": "Several agreement proposals would let existing AI models keep serving users while restricting further training. That requires verifying that a data centre runs inference only. The claim combines a positive part (the declared inference is happening) with a negative part (nothing else, especially training, is). It matters because it could let most AI compute stay in productive use under a training restriction. It is hard because the same chips can do both, workloads can be disguised, and the negative part requires accounting for all of a facility's activity. Proposed approaches include capturing and recomputing the traffic that enters and leaves a facility, limiting bandwidth between groups of chips so that large training cannot be coordinated, and classifying workloads from telemetry. Each rests on open assumptions about numerical nondeterminism, hidden capacity and side channels.",
    "claim_class": "mixed",
    "editors_synthesis": {
      "assessment": true,
      "markdown": "Declared inference can be checked with demonstrated (R2) components, but the absence of training cannot yet be verified. That half rests on full-stack designs ([[I-0011]], [[I-0012]], [[I-0008]]) that are proposed (R1) and not built beyond small prototypes.\n\n[[M-0013|Network taps]] (R1) capture front-end traffic, and [[M-0001|sampled recomputation]] (R2) checks it against the declared model; [[M-0003|reproducible computation packets]] (R1) would extend this to whole workloads. [[M-0014|Bandwidth limits]], [[M-0015|memory wiping]] (both R1) and [[M-0024|bounding unexplained information]] (R2) constrain what else could run or leave. [[M-0021|Workload classification]] and [[M-0010|on-chip telemetry]] (both R2) are lighter-touch alternatives.\n\nRecomputation tolerates numerical noise on open-weight models of 8 to 30 billion parameters [[S-0016]]. A telemetry classifier detects training with 98.2% accuracy on its own corpus, but 43–87% on the most challenging disguised workloads held out from its training [[S-0037]]. A team building the components reports nothing past a proof-of-principle prototype [[S-1512]].\n\nTraining traffic runs on back-end fabric that is harder to tap, and whether all outgoing traffic can be explained by incoming traffic is an open question [[S-0018]]. Telemetry is trustworthy only with protections, such as a tamper-resistant read path, that current GPUs lack [[S-0037]] [[S-0033]].",
      "text": "Declared inference can be checked with demonstrated (R2) components, but the absence of training cannot yet be verified. That half rests on full-stack designs (AI 2040 inference-only verification stack, Low-trust AI compute verification system overview, SASH confidential network logger) that are proposed (R1) and not built beyond small prototypes. Network taps (R1) capture front-end traffic, and sampled recomputation (R2) checks it against the declared model; reproducible computation packets (R1) would extend this to whole workloads. Bandwidth limits, memory wiping (both R1) and bounding unexplained information (R2) constrain what else could run or leave. Workload classification and on-chip telemetry (both R2) are lighter-touch alternatives. Recomputation tolerates numerical noise on open-weight models of 8 to 30 billion parameters [S-0016]. A telemetry classifier detects training with 98.2% accuracy on its own corpus, but 43–87% on the most challenging disguised workloads held out from its training [S-0037]. A team building the components reports nothing past a proof-of-principle prototype [S-1512]. Training traffic runs on back-end fabric that is harder to tap, and whether all outgoing traffic can be explained by incoming traffic is an open question [S-0018]. Telemetry is trustworthy only with protections, such as a tamper-resistant read path, that current GPUs lack [S-0037] [S-0033]."
    },
    "sources": [
      {
        "source": "S-0063",
        "supports": "restricting the scale of training; chip use verification distinguishing inference on existing systems from training",
        "locator": "abstract; Article VII (as summarised)"
      },
      {
        "source": "S-0067",
        "supports": "data centres converted to inference-only operation; network taps, recomputation and reproducible packets",
        "locator": "phases; verification mechanisms"
      },
      {
        "source": "S-0053",
        "supports": "majority of AI compute used for inference; decentralised training could undermine detectability",
        "locator": "training vs inference; limitations"
      },
      {
        "source": "S-0002",
        "supports": "declared inference (1.A.2) as a distinct subgoal; deterministic replication of inference as an R&D problem",
        "locator": "§3.2; Appendix A.9"
      },
      {
        "source": "S-0018",
        "supports": "distinguishing inference from training; token-level front-end evidence; back-end harder to tap; egress explainable by ingress as open question; memory wiping; side channels",
        "locator": "verification goals; inference vs training; open problems"
      },
      {
        "source": "S-0005",
        "supports": "inference-specialised chips repurposable for training; pods with limited external bandwidth",
        "locator": "Verifying that known compute is not being used for a large training run"
      },
      {
        "source": "S-0029",
        "supports": "no straightforward way to tell whether a chip is running training or another job",
        "locator": "open problems"
      },
      {
        "source": "S-0037",
        "supports": "telemetry classifier accuracy overall and on adversarially disguised workloads; required telemetry protections",
        "locator": "abstract; §5.2; deployment requirements"
      },
      {
        "source": "S-0001",
        "supports": "workload classification; adversarial customers may obfuscate by adding noise",
        "locator": "§3.2.2 / §5.2.2 open problems"
      },
      {
        "source": "S-0031",
        "supports": "front-end tapping most viable; back-end requires sampling",
        "locator": "frontend vs backend"
      },
      {
        "source": "S-0016",
        "supports": "recomputation despite benign numerical noise on 8–30B open-weight models",
        "locator": "abstract; §5"
      },
      {
        "source": "S-0020",
        "supports": "bit-exact inference verification across GPU variants",
        "locator": "abstract"
      },
      {
        "source": "S-1512",
        "supports": "no verification component past a proof-of-principle prototype",
        "locator": "introduction"
      },
      {
        "source": "S-0033",
        "supports": "current GPUs expose limited trusted telemetry",
        "locator": "abstract"
      }
    ],
    "concepts": [
      "K-0025",
      "K-0009",
      "K-0008",
      "K-0014",
      "K-0021",
      "K-0017",
      "K-0013"
    ],
    "order": 4,
    "type": "claim",
    "url": "https://trustbutveri.fyi/claims/inference-not-training/",
    "source_file": "content/claims/inference-not-training.md",
    "flags_all": [
      "ai-drafted"
    ],
    "body_markdown": "## Why it matters\nSeveral proposals to restrict frontier AI development target training while preserving the use of existing models [[S-0063]] [[S-0067]]. Sastry and colleagues note that most AI compute is now used for inference [[S-0053]].\n\n- **A draft international agreement** restricts the scale of AI training. Its chip-use verification is meant to distinguish inference on existing systems from the training of new models [[S-0063]].\n- **The AI 2040 verification plan** would convert data centres to inference-only operation, preventing training runs while models continue to serve users [[S-0067]].\n- **RAND's framework** treats accurate declaration of AI inference as its own verification subgoal, separate from declared training [[S-0002]]. It lists deterministic replication of neural-network inference as a research problem [[S-0002]].\n- **A low-trust system overview** names distinguishing inference from training, and deploying only approved models, among its core objectives [[S-0018]].\n\n## Why it is hard\n- **The same hardware can do both.** Scher and Thiergart expect that some current inference-specialised chips could be repurposed for training without significant effort [[S-0005]]. Shavit notes that there is no straightforward way to determine whether an ML chip is running a training job or an unrelated one [[S-0029]].\n- **Classifiers invite evasion.** The Open Problems survey notes that adversarial customers may obfuscate their activities, for example by adding noise to how they use computational resources [[S-0001]]. One adversarial study of GPU-telemetry classifiers reports 98.2% accuracy at identifying training across its corpus, but 43–87% on the most challenging disguised workloads held out from its training [[S-0037]].\n- **Training traffic is harder to observe.** Inference produces token-level input and output data on the front-end links between a data centre and its users [[S-0018]]. Training traffic runs over back-end fabric that has much higher bandwidth, is latency-sensitive, and is harder to tap [[S-0018]]. Cankaya describes front-end tapping as the most viable option, and notes that back-end tapping may require sampling rather than full capture [[S-0031]]. The system overview leaves open how far a facility can be required to make all egress traffic explainable by its ingress [[S-0018]].\n- **Recomputation must cope with numerical noise.** Checks on sampled outputs handle it either statistically [[S-0016]] or by exact reproduction [[S-0020]].\n- **Bandwidth limits may erode.** Limits between pods of chips could prevent the gradient exchange that distributed training needs [[S-0005]]. Sastry and colleagues note that more viable decentralised training could undermine the detectability of training [[S-0053]].\n- **Hidden capacity and channels.** The negative part of the claim requires that no capacity or channel is hidden. The system overview addresses this with memory wiping and side-channel suppression, and lists open problems for both [[S-0018]].",
    "body_text": "Why it matters Several proposals to restrict frontier AI development target training while preserving the use of existing models [S-0063] [S-0067]. Sastry and colleagues note that most AI compute is now used for inference [S-0053]. - A draft international agreement restricts the scale of AI training. Its chip-use verification is meant to distinguish inference on existing systems from the training of new models [S-0063]. - The AI 2040 verification plan would convert data centres to inference-only operation, preventing training runs while models continue to serve users [S-0067]. - RAND's framework treats accurate declaration of AI inference as its own verification subgoal, separate from declared training [S-0002]. It lists deterministic replication of neural-network inference as a research problem [S-0002]. - A low-trust system overview names distinguishing inference from training, and deploying only approved models, among its core objectives [S-0018]. Why it is hard - The same hardware can do both. Scher and Thiergart expect that some current inference-specialised chips could be repurposed for training without significant effort [S-0005]. Shavit notes that there is no straightforward way to determine whether an ML chip is running a training job or an unrelated one [S-0029]. - Classifiers invite evasion. The Open Problems survey notes that adversarial customers may obfuscate their activities, for example by adding noise to how they use computational resources [S-0001]. One adversarial study of GPU-telemetry classifiers reports 98.2% accuracy at identifying training across its corpus, but 43–87% on the most challenging disguised workloads held out from its training [S-0037]. - Training traffic is harder to observe. Inference produces token-level input and output data on the front-end links between a data centre and its users [S-0018]. Training traffic runs over back-end fabric that has much higher bandwidth, is latency-sensitive, and is harder to tap [S-0018]. Cankaya describes front-end tapping as the most viable option, and notes that back-end tapping may require sampling rather than full capture [S-0031]. The system overview leaves open how far a facility can be required to make all egress traffic explainable by its ingress [S-0018]. - Recomputation must cope with numerical noise. Checks on sampled outputs handle it either statistically [S-0016] or by exact reproduction [S-0020]. - Bandwidth limits may erode. Limits between pods of chips could prevent the gradient exchange that distributed training needs [S-0005]. Sastry and colleagues note that more viable decentralised training could undermine the detectability of training [S-0053]. - Hidden capacity and channels. The negative part of the claim requires that no capacity or channel is hidden. The system overview addresses this with memory wiping and side-channel suppression, and lists open problems for both [S-0018].",
    "addressed_by": [
      {
        "id": "M-0014",
        "title": "Bandwidth limits and compartmentalization",
        "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/",
        "role": "supporting",
        "note": "Intended to leave inference workable while making large training impractical (S-0067, S-1301)."
      },
      {
        "id": "M-0024",
        "title": "Bounding unexplained information in outputs",
        "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/",
        "role": "supporting",
        "note": "In the compute-agreement framing, outputs must be predictable by policy-compliant computation, which limits what undeclared workloads can export (S-0019)."
      },
      {
        "id": "M-0002",
        "title": "Deterministic and bit-exact inference",
        "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/",
        "role": "supporting",
        "note": "Bit-exact recomputation of declared inference removes the tolerance an operator could hide other work in (S-0020)."
      },
      {
        "id": "M-0015",
        "title": "Memory wiping and proofs of secure erasure",
        "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/",
        "role": "supporting",
        "note": "Periodic wipes are proposed so that only verified inference outputs persist (S-0067); Amodo frames wipes as verifying completeness of declared workloads (S-1302)."
      },
      {
        "id": "M-0013",
        "title": "Network taps and certifiers",
        "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/",
        "role": "primary",
        "note": "Supplies the committed record of cluster I/O that recomputation checks against declared inference (S-0067, S-1300)."
      },
      {
        "id": "M-0010",
        "title": "On-chip telemetry from timing, memory and performance counters",
        "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/",
        "role": "primary",
        "note": "Classifying training versus other workloads from counters (S-0037) or timing (S-0033)."
      },
      {
        "id": "M-0007",
        "title": "Proofs of useful work and resource exhaustion",
        "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/",
        "role": "supporting",
        "note": "Attestable's pacing proposal pairs work accounting with ZK inference proofs."
      },
      {
        "id": "M-0003",
        "title": "Reproducible computation packets",
        "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/",
        "role": "primary",
        "note": "Proposed as the correctness check for an inference-only retrofit (S-0067)."
      },
      {
        "id": "M-0001",
        "title": "Sampled inference recomputation",
        "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/",
        "role": "supporting",
        "note": "Proposed as the correctness check in inference-only retrofit plans. Completeness needs other mechanisms (S-0067, S-0017)."
      },
      {
        "id": "M-0022",
        "title": "Side-channel suppression for isolated facilities",
        "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/",
        "role": "supporting",
        "note": "Inference-only designs count on suppressing unmonitored physical channels so that all significant traffic passes the taps (S-0018)."
      },
      {
        "id": "M-0017",
        "title": "Tamper evidence for verifier devices",
        "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/",
        "role": "supporting",
        "note": "Protects the integrity of taps, gateways and recomputation hardware used for inference-only verification (S-0018, S-0067)."
      },
      {
        "id": "M-0008",
        "title": "TEE remote attestation for AI workloads",
        "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/",
        "role": "supporting",
        "note": "Attests a declared inference deployment, but not that the same chips ran no other workloads (S-0014)."
      },
      {
        "id": "M-0016",
        "title": "Timed challenge-response and memory-occupation challenges",
        "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/",
        "role": "supporting",
        "note": "Listed as an alternative inference-verification direction that may not need a hardware retrofit (S-0067); bounds spare memory (S-0018)."
      },
      {
        "id": "M-0021",
        "title": "Workload classification from telemetry and side channels",
        "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/",
        "role": "primary",
        "note": "Classifies observed activity as training, inference or non-ML work."
      },
      {
        "id": "M-0004",
        "title": "Zero-knowledge proofs of inference",
        "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/",
        "role": "supporting",
        "note": "Attestable proposes using proofs to show accounted workloads used an approved, unchanged model."
      },
      {
        "id": "I-0011",
        "title": "AI 2040 inference-only verification stack",
        "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/",
        "role": "primary",
        "note": "The stack's stated purpose: retrofitted data centres run only inference."
      },
      {
        "id": "I-0005",
        "title": "Attestable zero-knowledge inference prover",
        "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/",
        "role": "supporting",
        "note": "Proposed use: showing an accounted workload used an approved, unchanged model."
      },
      {
        "id": "I-0012",
        "title": "Low-trust AI compute verification system overview",
        "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/",
        "role": "primary",
        "note": "Challenged records are screened for inference versus training."
      },
      {
        "id": "I-0010",
        "title": "RAND secure inference data center (SIDC) design",
        "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/",
        "role": "supporting",
        "note": "Scoped to serving already-trained models."
      },
      {
        "id": "I-0008",
        "title": "SASH confidential network logger",
        "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/",
        "role": "primary",
        "note": "SASH describes the aim as distinguishing inference from training in data centres (S-1320)."
      }
    ],
    "referenced_by": [
      {
        "id": "M-0014",
        "title": "Bandwidth limits and compartmentalization",
        "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/"
      },
      {
        "id": "M-0024",
        "title": "Bounding unexplained information in outputs",
        "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/"
      },
      {
        "id": "M-0002",
        "title": "Deterministic and bit-exact inference",
        "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/"
      },
      {
        "id": "M-0015",
        "title": "Memory wiping and proofs of secure erasure",
        "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/"
      },
      {
        "id": "M-0013",
        "title": "Network taps and certifiers",
        "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/"
      },
      {
        "id": "M-0010",
        "title": "On-chip telemetry from timing, memory and performance counters",
        "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/"
      },
      {
        "id": "M-0007",
        "title": "Proofs of useful work and resource exhaustion",
        "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/"
      },
      {
        "id": "M-0003",
        "title": "Reproducible computation packets",
        "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/"
      },
      {
        "id": "M-0001",
        "title": "Sampled inference recomputation",
        "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/"
      },
      {
        "id": "M-0022",
        "title": "Side-channel suppression for isolated facilities",
        "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/"
      },
      {
        "id": "M-0017",
        "title": "Tamper evidence for verifier devices",
        "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/"
      },
      {
        "id": "M-0008",
        "title": "TEE remote attestation for AI workloads",
        "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/"
      },
      {
        "id": "M-0016",
        "title": "Timed challenge-response and memory-occupation challenges",
        "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/"
      },
      {
        "id": "M-0021",
        "title": "Workload classification from telemetry and side channels",
        "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/"
      },
      {
        "id": "M-0004",
        "title": "Zero-knowledge proofs of inference",
        "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/"
      },
      {
        "id": "I-0011",
        "title": "AI 2040 inference-only verification stack",
        "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/"
      },
      {
        "id": "I-0005",
        "title": "Attestable zero-knowledge inference prover",
        "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/"
      },
      {
        "id": "I-0012",
        "title": "Low-trust AI compute verification system overview",
        "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/"
      },
      {
        "id": "I-0010",
        "title": "RAND secure inference data center (SIDC) design",
        "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/"
      },
      {
        "id": "I-0008",
        "title": "SASH confidential network logger",
        "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/"
      },
      {
        "id": "C-0002",
        "title": "Chips are where they are declared to be",
        "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/"
      },
      {
        "id": "C-0003",
        "title": "Declared hardware is idle or shut down",
        "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/"
      },
      {
        "id": "C-0010",
        "title": "There is no undeclared relevant compute",
        "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/"
      },
      {
        "id": "K-0003",
        "title": "Positive and negative claims",
        "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/"
      },
      {
        "id": "O-0205",
        "title": "Oxford Martin AI Governance Initiative",
        "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/"
      }
    ]
  }
}