Naci Cankaya's system overview, published by the Machine Intelligence Research Institute's Technical Governance Team, proposes a privacy-preserving verification system that can be added to existing AI data centres, for an agreement between states that distrust each other.
Network taps hash all traffic crossing the monitored boundary, and only these hashes are shared with the verifier. Later, the verifier challenges random records. The operator supplies the matching data, and at least three independently sourced computing clusters, each air-gapped inside the operator's data centre, re-run the workload and screen it, for example for training versus inference or banned uses.
One-way optical links, cross-checks of every output and re-transmission by the operator limit what either side can leak. The document is a working draft, and key parts are open research questions.
The main obstacles are bit-exact replay, physical security and side-channel defence. The author says the threat model is under-developed.
A detailed end-to-end design states the claim and threat model, but there is no integrated implementation and there are no results.
Assessed use: screening challenged records to show declared inference compute is not training
On this page
What it is
Naci Cankaya's overview, from the Machine Intelligence Research Institute's Technical Governance Team, is a working draft 1. It describes the author's current best idea for a privacy-preserving, retrofittable AI compute verification system, meant to build confidence in an arms-control-style AI agreement 1. It offers example rules the system should support: monitorable chains of thought, cryptographic commitment of checkpoints, whitelisting of models before deployment, and a blacklist of illicit uses 1.
The author argues that checking aggregate properties, such as total training FLOPs, is not enough, and sets collision-resistant evidence capture of individual workloads as a requirement 1. He assumes the worst case on both sides: the prover may covertly break the agreement's rules, and the verifier may try to exfiltrate the prover's secrets 1.
How it works
The design separates capturing evidence from evaluating it 1.
Evidence capture
- Network taps. Taps on the monitored links hash traffic in groups of packets at line rate 1. See Network taps and certifiers.
- Shared hash stream. The hashes leave over a split, sanitized fibre, so prover and verifier see identical copies. The prover recomputes the hashes from its own copy of the traffic, confirming that the tap sent nothing but legitimate commitments 1.
- Prover-held data. Beyond hashes, no information is shared with the verifier; the prover keeps the plaintext and a manifest 1.
- Memory challenges and wiping. These are an optional second form of evidence 1. See Data-centre memory challenging, Timed challenge-response and memory-occupation challenges and Memory wiping and proofs of secure erasure.
A companion preprint specifies a secure gateway device for the taps that needs no processor trusted by both parties 2.
Evidence evaluation
- The verifier issues a challenge over a one-way link enforced by a data diode 1.
- The prover supplies the matching data, manifest entries and replay metadata 1.
- Each party's trusted computing base checks the hashes, re-runs the computation under the declared metadata and screens it for compliance 1. See Deterministic and bit-exact inference.
- The trusted computing bases cross-check their verdicts, and nothing is released unless they agree; by default, disclosure is minimal 1.
A missing or mismatched record fails the challenge, while a failed replay or a disagreement triggers investigation 1. The evaluation hardware is at least three distinct trusted computing bases from different supply chains, trusted respectively by the prover, the verifier and a third party, each inside its own air-gap in the prover's data centre 1. Zero-knowledge proofs are a tentative "plan B" for evaluation, if they mature 1. See Zero-knowledge proofs of inference.
Subsystems
The design avoids the need for mutually trusted silicon with these building blocks 1:
- Analog data movement control. Fused fibre splitters duplicate traffic, optical isolators act as data diodes, and enclosures protect the equipment 1.
- Controlled ingress. A one-way path carries the verifier's inputs into the auditing environment 1.
- Output cross-checks. The prover clones the output of each verifier device with a fibre splitter, to confirm it sends nothing but the correctly computed result 1.
- Sanitized egress. The prover re-transmits verifier outputs at batched or randomized times, blocking optical and timing side channels 1.
- Secure performance offshoring. Untrusted fast hardware re-runs workloads, while small trusted devices compare the results 1.
Side-channel defence and resource accounting support the rest of the system 1. See Side-channel suppression for isolated facilities.