The claim is that data flowing between specified groups of chips, or out of a facility, stays below a declared rate.
Restricting communication between compute groups could limit joint training while allowing existing models to keep serving users 1. A bound on a facility's external links also limits how much weights or data could leave it 14 15. The claim covers every channel, including paths outside the monitored network.
Proposed designs cap the external bandwidth of small groups of chips, called pods, or restrict chip networking in hardware. A pod design aims to allow inference tokens but restrict training gradients, provided the devices that serve each copy of the model share one pod 1 16.
Network taps observe the links. The hard parts are finding every path, including physical side channels; choosing bounds that stay meaningful as training methods improve; and monitoring high-speed accelerator fabrics that are difficult to tap.
A software bandwidth monitor has reached research demonstration (R2). It runs on the operator's nodes, and its authors say the measurements are trivially spoofable 13. No bandwidth cap that a verifier can check has been demonstrated publicly.
Bandwidth limits and compartmentalization (R2 for software monitoring) proposes to cap or remove links between accelerator groups. Its pod-cap design aims to allow inference tokens but restrict training gradients, provided each copy of the served model, including any split of the model or its experts across devices, stays within one pod 1 16. It needs no access to a facility's code 1. Hardware versions include RAND's fixed-set design 4 and guarantee processors (proposed, R1), and the AI 2040 stack (R1) removes back-end networking 10. Side-channel suppression (R1) addresses paths outside the network 7. Tamper evidence (R2 for probing detection on servers and protected electronics) would protect the enforcing devices, and bounding unexplained information (R2) limits what the permitted bandwidth can carry.
Lucid Computing's pod-cap design is "not yet implemented or red-teamed" 11, and Amodo Design's rate limiting on 400G links was for weight security with a cooperating operator 12.
Copper scale-up links and encrypted interconnects are hard to observe 6, and the designs aim to cut side channels to low rates, not to eliminate them 5 7. Sastry and colleagues note that decentralised training, if it becomes more viable, might undermine the detectability of training runs 2, and training methods that need less communication could likewise erode a bound that separates training from inference today.
On this page
Why it matters
Distributed training depends on fast communication between chips, so limiting communication limits training 1.
- Scher and Thiergart describe pods of chips with high-bandwidth interconnect inside each pod but very low external bandwidth: enough for inference tokens, not for training gradients 1. They suggest this could be implemented with modest changes to existing technology, physical access and monitoring, without access to code 1.
- That token-only description applies when the served model and its expert routing fit within a pod. Expert-parallel mixture-of-experts inference can require substantial all-to-all traffic between devices 16. A pod boundary must therefore be sized and tested for the declared serving workload.
- Compute-governance analyses list physical limits on chip-to-chip networking as a way to enforce compute caps 2. RAND's "fixed set" design restricts networking so that small, fixed sets of GPUs cannot be aggregated into large clusters 4.
- A low-trust system overview asks how far a facility can be required to make all egress traffic explainable by its ingress 5. It assumes that the information bandwidth of inference is not more than two orders of magnitude above expected API traffic at full utilisation 5. In an Alignment Forum post on upload limits, Greenblatt proposes limiting the total data uploaded from inference servers so that weights cannot be exfiltrated through them 14. Rinberg and colleagues restate the rationale: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB 15. This links the claim to keeping weights inside the facility.
Why it is hard
The claim requires knowing every path.
- Taps see only the links they are on. The network-tap layer of RAND's verification framework observes data exchanged between chips 9. The cost and feasibility of tapping differ by link: in a research note on secure network taps, Cankaya describes front-end links to the outside world as the most viable to monitor, while the back-end compute fabric has far higher bandwidth, is latency-critical, and may require sampling rather than full capture 6.
- Some links are hard to observe at all. As scale-up domains grow, more traffic runs over copper links that passive optical taps cannot see, and encrypted interconnects complicate inspection 6. Shavit notes that methods for verifiers to confirm, after the fact, the topology in which chips were configured in a data centre were beyond the scope of his 2023 compute-monitoring paper 8.
- Physical side channels. Data could bypass network monitoring altogether. A Technical Governance Team post from the Machine Intelligence Research Institute surveys electromagnetic, acoustic, conducted and fibre-optic channels, and proposes shielding, active jamming with sensing, inspection and filtering 7. The low-trust system overview's target is to suppress covert side-channel bandwidth to kilobits per second, not to zero 5.
- Choosing the bound. A threshold that separates inference from training today may not do so later. On the widely used C4 dataset, DiLoCo, a low-communication training method, on 8 workers matched fully synchronous training while communicating 500 times less 3. Sastry and colleagues note that decentralised training across many data centres, or with lower-quality compute, could undermine compute detectability if it becomes more viable 2.