Sampling and assurance
Checking a random sample of accelerators, workload segments or outputs rather than all of them, so that violations are caught with a calculable probability.
Sampling-based assurance checks a random sample of accelerators, workload segments or outputs instead of all of them, and chooses the sample size so that a violation is caught with a desired probability 1.
Shavit gives a formula for how many accelerators a verifier must inspect in each monitoring period to find at least one accelerator used in a rule-violating training run with a chosen probability 1. The required number falls as the run occupies a larger share of the prover's accelerators, so larger runs need fewer inspections 1. Sampling works only if the prover cannot predict what will be checked 1 or change its records once it knows; in one scheme, the prover commits a hash of sampled weights at each training step before it learns whether that step will be audited 2. The same logic applies to recomputation of random training segments in proof-of-learning 1 and of random workload samples in reproducible computation packets 3. Against a covert adversary, sampling works through deterrence: one system overview notes that such an adversary is caught if it fails to stay hidden even once, and treats randomly sampled inspections as a primary defence 4.
Related
Used in
- R2Bounding unexplained information in outputs
- R1Chip registries and manufacturing records
- R1Network taps and certifiers
- R2On-chip telemetry from timing, memory and performance counters⚠
- R2Proof-of-learning and training-transcript verification⚠
- R1Reproducible computation packets
- R2Sampled inference recomputation
- R1Timed challenge-response and memory-occupation challenges
- R2Zero-knowledge proofs of inference
- R2Zero-knowledge proofs of training constraints
- R1AI 2040 inference-only verification stack
- R1Attestable zero-knowledge inference prover
- R2DiFR (Divergence From Reference)
- R1Low-trust AI compute verification system overview
- R1SASH confidential network logger
- Compute stock is at most a declared amount
- There is no undeclared relevant compute
- A training run stayed within declared limits
Sources
- BY. Shavit (2023). What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring. arXiv. Source recordSupports: number of chips to sample per monitoring period to catch at least one chip from a violating run with probability p; fewer samples for larger runs; the Prover cannot predict which chips are inspected; sampled segment recomputation · §3.1–3.2, Equation 1, Table 1; §5.1
- CAmodo Design (2026). Example Schemes for Verifying High-Stakes AI Agreements. Amodo Design. Source recordSupports: hash commitment to sampled weights before the prover learns whether a step will be audited · pre-training scheme
- CR. Dean (2026). Verification Plan. AI 2040. Source recordSupports: recomputation server re-runs random samples of workload packets · Concrete inference-only retrofitting proposal
- BN. Cankaya (2026). A System Overview for Near-Term, Low-Trust AI Compute Verification. Machine Intelligence Research Institute. Source recordSupports: a covert adversary is caught if it fails to stay hidden even once; random sampling needs to catch only a single instance of cheating; sampled inspections as primary defences · threat model; defence layers