Mechanism · Sampled inference recomputation
Technical detail
On this page
The reference protocol has five steps 1:
- The serving stack writes every request and response to an append-only log, with the model release, code version and sampling seed 1.
- An isolated verification server draws a private random sample of logged pairs. Its isolation can range from access controls and containers to a full air gap 1.
- The server runs one prefill pass over the prompt and claimed output with the trusted model and code. This gives the next-token distribution at every position 1 2.
- The server scores how far the claimed output diverges from the reference. Token-DiFR uses the clipped Gumbel-Max logit margin under a shared seed. Activation-DiFR uses the distance between random projections of activations 2. TOPLOC counts exponent mismatches and measures mantissa differences in the top-128 last-layer activations 7.
- Scores beyond a threshold calibrated on trusted hardware are escalated for review 1 2.
Verification is cheap because it needs only the prefill pass. Decoding typically runs at 3–5 times lower hardware utilization than prefill 1. Amodo's prototype measured a 2–8 times performance advantage for the verifier with models of 1.5 to 120 billion parameters on H100 and H200 GPUs 11.
The number of samples sets the chance of detection. For a false-output rate p, the chance of catching at least one false output in k independent samples is 1 − (1 − p)^k 5. With 3,000 samples, a 0.1% rate is caught with 95.03% probability 4.
Recomputation can also match bit for bit, which makes the check pass/fail. This exact variant needs a known hardware model, deployed weights, parallelism topology, software versions and per-pass batch size, and no atomic operations 4 5.