AI Verification LandscapeSearch

Implementation · Isolation & system architectures

AI 2040 inference-only verification stack

A proposed retrofit that isolates data-centre inference units, taps their front-end traffic and recomputes random samples to check that only declared inference runs.

Also called AI 2040 verification plan (inference-only retrofit); Inference-only retrofit

R1ProposedProvider-reported

Summary

The AI Futures Project's AI 2040 verification plan proposes a technical stack for checking that retrofitted data centres run only inference, not training or experiments. Data centres would be split into isolated inference units by removing much of their back-end networking. Passive network taps would copy all front-end traffic to a recomputation server, which re-runs random samples of workloads organized into reproducible packets. Strong physical security would protect the equipment, and side-channel limits plus periodic memory wiping are meant to stop hidden work from leaving results. As of July 2026, passive optical taps worked at 400G and recomputation schemes were being tested, but the integrated stack had not been built; one team working on it says nothing is past a proof-of-principle prototype. The main obstacles are reproducible inference and networking, securing the recomputation server, and scaling side-channel defences. The plan does not verify that spare compute is unused.

ReadinessR1Proposedmedium confidence · rubric v1.0 · 2026-09-23

R1: the six-layer design, its claim and its assumptions are published, but the integrated stack has not been built.

Rubric assessment
  • R1 met: the plan publishes the design of the stack, the claim it would verify (only inference runs on retrofitted compute) and its key assumptions, including a trusted recomputation server and strong physical security 1.
  • R2 not met: there is no public implementation or end-to-end result for the integrated stack. The plan's companion page lists a reproducible inference stack as not started and gives no plan yet for scaling side-channel defences 2. A team working on the components says that nothing is past a proof-of-principle prototype 3. Some components are further along: passive optical taps work at 400G, and recomputation schemes such as TOPLOC and DiFR are being tested on relevant hardware 2; see Sampled inference recomputation.

Confidence is medium: the design is clearly published, but component status comes from pages that are updated over time 2 4.

Gaps to the next level
  • A public working implementation or reproducible end-to-end results for the integrated stack (isolated inference unit, taps, packetization and recomputation) at realistic scale or against a stated adversary.
  • A hardened recomputation server and a method for checking that taps are correctly installed and remain in place.
  • Red-teaming of recomputation and of the completeness measures (side channels, memory wiping).

What it is

The AI 2040 verification plan, written by Romeo Dean and published by the AI Futures Project, includes a technical stack for verifying that retrofitted data centres run only inference workloads, not training or experiments 1 2. The plan calls it an inference-only retrofit and lists six layers 1. It is a proposal: the plan's companion page tracks which components exist and which are still open problems 2.

How it works

The six layers, as the plan lists them 1:

  • Isolated inference units. Data centres are split into groups of isolated inference units by removing much of their back-end (east–west) networking 1. The plan argues that inference needs much less scale-out interconnect than training, so the overhead is relatively small 1. See Bandwidth limits and compartmentalization.
  • Passive taps. Simple, passive taps on the front-end (north–south) network copy all input and output traffic to a recomputation server 1. The plan favours optical splitters, "essentially a piece of glass", because they are probably the easiest to trust 1. See Network taps and certifiers.
  • Reproducible packets. All AI workloads, including agentic tool use, are organized into discrete, reproducible packets that the recomputation server can inspect 1. Inference is not reproducible by default, and achieving it may cost some overhead 1. See Reproducible computation packets.
  • Partial recomputation. The server recomputes random samples of packets to check their correctness 1. The plan argues that this can verify all outputs with very high probability using a small fraction of the original compute, provided the server can be trusted 1. See Sampled inference recomputation.
  • Physical security. Strong physical security protects the verification equipment against tampering 1. See Tamper evidence for verifier devices.
  • Completeness. Side-channel communication is bounded, and memory is wiped periodically, probably by forced memorization 1. The aim is that the only results that persist are verified inference outputs 1. See Side-channel suppression for isolated facilities and Memory wiping and proofs of secure erasure.

Evidence

The plan's companion page, updated in July 2026, and a practitioner status page report the state of each component 2 4:

  • Taps. Passive optical taps work at 400G; the 800G and 1600G line rates now arriving in data centres look feasible but are undemonstrated 2. Capturing all front-end traffic needs no new research, but needs a lot of infrastructure plus work on network topologies and sampling 2.
  • Recomputation. TOPLOC and DiFR are well described and are being tested on relevant hardware and models 2. Recomputation red-teaming has not started 4.
  • Reproducibility. Demonstrations work around non-determinism, but work on a fully reproducible inference stack has not started 2.
  • Completeness. Early hardware testing of memory wiping is under way 2. For side-channel mitigation, only early theoretical pieces exist 2.
  • Overall. The Amodo team reports that the core components are being de-risked, but that nothing is past a proof-of-principle prototype 3.

Limitations

  • Trusted recomputation server. The assurance argument assumes that the server's computations and outputs can be trusted 1. Amodo rates recomputation-server security as not on track 4.
  • Spare compute. Idle or spare compute is not directly verified 1.
  • Tap installation. Checking that taps are correctly installed and stay in place is not yet solved 2.
  • Network reproducibility. Making individual packets reproducible across the network may need considerable software, firmware and possibly hardware work 2.
  • Recomputation attacks. When the adversary controls the prompts, Gumbel-based inference verification, the family that includes Token-DiFR 5, leaks roughly twice as many bits per token as under benign prompts 6. See Bounding unexplained information in outputs.
  • Side-channel defence. There is no plan yet for scaling side-channel defences quickly on a frontier cluster 2.

Known flaws

Published flaws, with their severity, kind and status. How flaws are rated.

  • The recomputation server must be trustedSignificantTheoretical argumentOpen

    The plan calls the integrity of the recomputation server an extremely important aspect, and its argument that sampling verifies all outputs assumes that the server's computations and outputs can be trusted. The companion page notes that the server sits inside the prover's facility, possibly under the prover's physical control, and that hardening it against integrity attacks needs significant research. Amodo rates recomputation-server security as not on track.

    Sources: [1] [2] [4]
  • Spare compute is not verifiedSignificantTheoretical argumentOpen

    The plan states that it does not verify that spare compute is unused for unapproved workloads, because this seems very challenging. It relies instead on side-channel bounds and memory wipes, so that the only results that persist are verified inference outputs.

    Sources: [1]
  • A recomputation family degrades against prompt-controlling adversariesSignificantDemonstrated attackOpen

    The plan's companion page names DiFR among the recomputation schemes being tested. An independent study found that Gumbel-based inference verification, the family that includes Token-DiFR, leaks roughly twice as many bits per token when the adversary chooses prompts that disrupt the text's structure, across six models of 1 to 32 billion parameters. The slowdown it imposes on a hidden-signalling adversary falls from over 200 times to 60–118 times. The attack weakens the bound on hidden information in outputs, which the plan relies on to keep undeclared results from leaving.

    Sources: [2] [5] [6] [1]

Blockers

Technical detail

Show technical detail
  • Secure network gateways. The plan names more capable gateways as a possible replacement for the passive taps. Splitters on the input and output paths log data flow into packets, which are randomly sampled and sent to a secure recomputing server. At boot, the gateways would also check that only whitelisted model weights are uploaded to the inference unit 1.
  • Physical security measures. The plan lists tamper-evident enclosures, security cameras, perimeter controls and air-gapping 1.
  • Other options the plan names. Zero-knowledge proofs could preserve privacy and need less hardware retrofit, but are "currently somewhat speculative" for lack of efficient enough algorithms. Memory-challenge verification might also avoid a hardware retrofit, but "seems likely to face some difficulties with verifying completeness". For an initial phase, removing major scale-out interconnect and installing simple sensors might suffice 1.

Sources

  1. CR. Dean (2026). Verification Plan. AI 2040. Source recordSupports: the six-layer stack; isolation rationale; passive optical taps; packets and reproducibility; recomputation and its trust assumption; physical security measures; completeness measures; spare-compute scope; gateway variant; other options · Summary of the plan; inference-only retrofit description; Appendix reference
  2. CAI Futures Project (2026). Get Involved in Verification. AI 2040. Source recordSupports: component status and open problems as of 9 July 2026; recomputation-server hardening · network taps; reproducible packets; partial recomputation; physical security; completeness
  3. CT. Milton et al. (2026). Verifying international AI deals: Plan A, the state-of-play, and what you can do to help. Amodo (Substack). Source recordSupports: overall maturity statement; description of taps and recomputation · introduction
  4. CAmodo Design (2026). AI 2040 Plan A — Verification SITREP. Amodo Design. Source recordSupports: status of recomputation server security and red-teaming · status items
  5. BR. Rinberg et al. (2025). Verifying LLM Inference to Detect Model Weight Exfiltration. arXiv. Source recordSupports: Token-DiFR as a Gumbel-Max estimator · §6.3
  6. BN. Kezins (2026). Adversarial Entropy Inflation Against Gumbel-Based Inference Verification. arXiv. Source recordSupports: prompt-controlling attack on Gumbel-based verification; models tested; bits per token; slowdown factor · abstract

I-0011JSONSource-checked 2026-09-23 · changed 2026-09-24Suggest a correction

Drafted with AI assistance and checked against the cited sources by an independent verifier. Expert review pending. Readiness assessed by the AI draft on 2026-09-23. How records are made.