DiFR (Divergence From Reference)

What it is

DiFR (Divergence From Reference) is a pair of methods for checking that an inference provider performed inference correctly. The checks work despite the benign numerical noise that makes re-runs of the same model differ ¹. Authors at the ML Alignment and Theory Scholars (MATS) programme, FAR AI, Anthropic and Harvard University introduced it in a 2025 preprint ¹.

• Token-DiFR compares the tokens a provider generated with the predictions of a trusted reference run that uses the same random seed ¹.
• Activation-DiFR compares compressed fingerprints of internal activations ¹.

A companion paper uses the same family of estimators to detect model weights hidden in inference outputs ². The code is public under an MIT licence ³.

How it works

The provider commits to a specification: model and weights, numerical precision, sampling settings and random seed ¹. Token-DiFR relies on Gumbel-Max sampling, which picks each token by adding seeded random noise to the model's scores and taking the largest. With a shared seed, the verifier knows the random noise the provider used ¹. It can therefore compute which token the reference model would have chosen at each position, and how far the claimed token falls short of it ¹.

The verifier pools these margins over many tokens into a score. It compares the score with a threshold calibrated on trusted hardware ¹. The verifier feeds the prompt and output through the model once, which gives logits and activations for every position at once ¹. Because the seed pins down sampling, over 98% of tokens match exactly between provider and verifier ¹.

Activation-DiFR uses a shared random orthogonal projection to compress activations into short fingerprints. The verifier compares distances between its fingerprints and the provider's ¹. This checks the forward pass but not the sampling step ¹.

Evidence

• The authors tested Llama 3.1 8B-Instruct, Qwen3-8B and Qwen3-30B-A3B on 2,000 UltraChat prompts ¹. The four inference configurations were H200 with four-way tensor parallelism, A100 with and without it, and H200 without it running Hugging Face ¹.
• The faults tested were FP8 key-value cache quantization, 4-bit model quantization, a temperature of 1.1 instead of 1.0, an incorrect seed, and a simulated sampling bug that picks uniformly among the top-k tokens 1% of the time ¹.
• The authors report that Token-DiFR detects 4-bit quantization with an AUC above 0.999 within 300 output tokens ¹. Activation-DiFR does so within 2 tokens, while cutting communication by 25–75% relative to existing methods ¹.
• The authors report that Activation-DiFR Pareto-dominates TOPLOC in communication cost against detection accuracy ¹. They also cite a report that a distributional test called RUT failed to reliably separate 8-bit quantized models. Token-DiFR detects such differences within a few thousand tokens ¹.
• The companion paper's detector limited exfiltratable information to under 0.5% at a false-positive rate below 0.01% on a mixture-of-experts Qwen 30B model, under benign prompt traffic ². An independent study found that choosing the prompts roughly doubles the leakage per token ⁸.
• Amodo Design, a separate team, re-implemented DiFR in an asynchronous recomputation prototype ^{4 5}. It tested the prototype on models of 1.5 to 120 billion parameters on H100 and H200 GPUs ⁴.

Limitations

• The methods need the model weights ¹. Providers can check their own traffic and users can check open-weights models, but the methods cannot verify closed-weights providers ¹.
• Deployment requires knowing the provider's sampling procedure ¹. In Amodo's prototype, vLLM v0.25.1 used a different random-number implementation from the one DiFR expected. This produced large logit differences until Amodo forced the older model runner ⁴.
• For Qwen3-30B-A3B, mixing A100 and H200 reference runs made the smallest tested change harder to separate from honest runs ¹.
• Only single-model sampling was studied. Speculative decoding was not evaluated ¹.
• A statistical tolerance can bound covert bandwidth but cannot close it ⁶.
• Adversarial testing is limited. The authors show that simple adversarial temperature tuning defeats a cross-entropy baseline, while Token-DiFR still detects the change ¹. An independent study reports that attacker-chosen prompts roughly double the leakage allowed by a weight-exfiltration detector built on the same Gumbel-margin statistic. This cut the detector's slowdown to 60–118 times ⁸. Amodo rates red-teaming of recomputation schemes as not started ⁷.