Output nondeterminism leaves covert capacity

Hashing cannot remove information hidden in the outputs themselves. The Secure Gateway Device paper estimates that about 0.1 bit per token remains even with seed-synchronized replay checks. For a 200k-GPU inference cluster at full load (2,000 tokens per GPU per second), that is about 40 Mbit/s of covert egress, enough to move a 1 TB model in under three days. The paper names this the core remaining challenge and points to deterministic replay or active scrubbing of hardware-induced entropy ¹. An independent study found that an adversary who chooses the prompts roughly doubles the bits leaked per token under Gumbel-based inference verification ¹³; see Bounding unexplained information in outputs.