{ "schema_version": "1.0.0", "rubric_version": "1.0", "license": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)", "record": { "id": "C-0008", "slug": "bandwidth-is-bounded", "title": "Communication between compute groups is bounded", "aliases": [ "bandwidth limits", "interconnect limits" ], "status": "draft", "last_reviewed": "2026-09-23", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [] }, "risk_flags": [], "flags": [], "one_liner": "Data flowing between specified groups of chips, or out of a facility, stays below a declared rate, so the groups cannot jointly run large workloads.", "summary": "Large-scale training requires chips to exchange large volumes of data, while serving an existing model needs much less between groups of chips. A verified bound on communication between compute groups could therefore restrict training while allowing inference. A bound on a facility's external links also limits how much weights or data could leave it. The claim is negative: no channel exceeds the bound, including channels outside the monitored network. Proposed designs cap the external bandwidth of small pods of chips or restrict chip networking in hardware, and use network taps to observe the links. The hard parts are finding every path, including physical side channels; choosing bounds that stay meaningful as distributed training methods improve; and monitoring high-speed accelerator fabrics that are difficult to tap.", "claim_class": "negative", "editors_synthesis": { "assessment": true, "markdown": "No bandwidth cap that a verifier can check has been built, and the mechanisms aimed at this claim are proposed (R1). Bounds appeal because they constrain a facility without access to its code [[S-0005]].\n\n[[M-0014|Bandwidth limits and compartmentalization]] (R1) cap or remove the links between groups of accelerators, leaving enough for inference but not for training [[S-0005]]. Hardware versions include RAND's fixed-set design [[S-0057]] and [[M-0009|guarantee processors]] (R1), and the [[I-0011|AI 2040 stack]] (R1) removes back-end networking [[S-0067]]. [[M-0022|Side-channel suppression]] (R1) addresses paths outside the network [[S-0038]]. [[M-0017|Tamper evidence]] (R2) would protect the enforcing devices, and [[M-0024|bounding unexplained information]] (R2) limits what the permitted bandwidth can carry.\n\nLucid Computing's pod-cap design is \"not yet implemented or red-teamed\" [[S-1301]]. Amodo has run rate limiting on 400G links, but for weight security with a cooperating operator [[S-1313]].\n\nCopper scale-up links and encrypted interconnects are hard to observe [[S-0031]]. The designs aim to cut side channels to low rates, not to eliminate them [[S-0018]] [[S-0038]]. Sastry and colleagues note that decentralised training, if it becomes more viable, might undermine the detectability of training runs [[S-0053]]. Training methods that need less communication could likewise erode a bound that separates training from inference today.", "text": "No bandwidth cap that a verifier can check has been built, and the mechanisms aimed at this claim are proposed (R1). Bounds appeal because they constrain a facility without access to its code [S-0005]. Bandwidth limits and compartmentalization (R1) cap or remove the links between groups of accelerators, leaving enough for inference but not for training [S-0005]. Hardware versions include RAND's fixed-set design [S-0057] and guarantee processors (R1), and the AI 2040 stack (R1) removes back-end networking [S-0067]. Side-channel suppression (R1) addresses paths outside the network [S-0038]. Tamper evidence (R2) would protect the enforcing devices, and bounding unexplained information (R2) limits what the permitted bandwidth can carry. Lucid Computing's pod-cap design is \"not yet implemented or red-teamed\" [S-1301]. Amodo has run rate limiting on 400G links, but for weight security with a cooperating operator [S-1313]. Copper scale-up links and encrypted interconnects are hard to observe [S-0031]. The designs aim to cut side channels to low rates, not to eliminate them [S-0018] [S-0038]. Sastry and colleagues note that decentralised training, if it becomes more viable, might undermine the detectability of training runs [S-0053]. Training methods that need less communication could likewise erode a bound that separates training from inference today." }, "sources": [ { "source": "S-0005", "supports": "pods with high internal and very low external bandwidth; enough for inference tokens not training gradients; implementable with modest changes, physical access and monitoring, without code access", "locator": "Interconnect bandwidth limits" }, { "source": "S-0053", "supports": "compute caps via physical limits on chip-to-chip networking; decentralised training could undermine detectability", "locator": "enforcement; §3.B.1 Detectability" }, { "source": "S-1314", "supports": "DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less", "locator": "abstract" }, { "source": "S-0057", "supports": "fixed-set HEM restricting networking of small GPU sets", "locator": "p. viii" }, { "source": "S-0018", "supports": "front-end vs back-end bandwidth; inference bandwidth assumption; covert side-channel target in kilobits per second; egress explainable by ingress", "locator": "inference vs training; open problems" }, { "source": "S-0031", "supports": "front-end vs back-end tapping costs and feasibility; copper links and scale-up domains; encrypted interconnect", "locator": "frontend vs backend; open problems" }, { "source": "S-0038", "supports": "physical side channels can bypass network monitoring; defences; tolerable low rate", "locator": "channels of concern; defences" }, { "source": "S-0029", "supports": "retroactively confirming data-centre topology is beyond scope", "locator": "open problems" }, { "source": "S-0002", "supports": "network taps intercepting inter-chip data", "locator": "§4.2" }, { "source": "S-0067", "supports": "removing back-end networking to create isolated inference units", "locator": "verification mechanisms" }, { "source": "S-1301", "supports": "pod-level traffic cap design; not yet implemented or red-teamed", "locator": "summary; status" }, { "source": "S-1313", "supports": "DPU-enforced rate limiting on 400G links for weight security, with the limits set by a trusted operator's controller", "locator": "whole note" }, { "source": "S-1508", "supports": "egress limits cap what can be stolen", "locator": "§5.1" } ], "concepts": [ "K-0021", "K-0017", "K-0014", "K-0013" ], "order": 8, "type": "claim", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/", "source_file": "content/claims/bandwidth-is-bounded.md", "flags_all": [ "ai-drafted" ], "body_markdown": "## Why it matters\nDistributed training depends on fast communication between chips, so limiting communication limits training [[S-0005]].\n\n- **Pods with narrow external links.** Scher and Thiergart describe pods of chips with high-bandwidth interconnect inside each pod but very low external bandwidth: enough for inference tokens, not for training gradients [[S-0005]]. They suggest this could be implemented with modest changes to existing technology, physical access and monitoring, without access to code [[S-0005]].\n- **Hardware limits on networking.** Compute-governance analyses list physical limits on chip-to-chip networking as a way to enforce compute caps [[S-0053]]. RAND's \"fixed set\" design restricts networking so that small, fixed sets of GPUs cannot be aggregated into large clusters [[S-0057]].\n- **Limits on what leaves a facility.** A low-trust system overview assumes that the information bandwidth of inference is not more than two orders of magnitude above expected API traffic at full utilisation [[S-0018]]. It asks how far a facility can be required to make all egress traffic explainable by its ingress [[S-0018]]. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [[S-1508]]. This links the claim to [[C-0009]].\n\n## Why it is hard\nThe claim requires knowing every path.\n\n- **Taps see only the links they are on.** RAND's network-tap layer observes data exchanged between chips [[S-0002]]. The cost and feasibility of tapping differ by link: Cankaya describes front-end links to the outside world as the most viable to monitor, while the back-end compute fabric has far higher bandwidth, is latency-critical, and may require sampling rather than full capture [[S-0031]].\n- **Some links are hard to observe at all.** As scale-up domains grow, more traffic runs over copper links that passive optical taps cannot see, and encrypted interconnects complicate inspection [[S-0031]]. Shavit notes that methods for verifiers to confirm, after the fact, the topology in which chips were configured in a data centre were beyond the scope of his work [[S-0029]].\n- **Physical side channels.** Data could bypass network monitoring altogether. A MIRI Technical Governance Team post surveys electromagnetic, acoustic, conducted and fibre-optic channels, and proposes shielding, active jamming with sensing, inspection and filtering [[S-0038]]. The system overview's target is to suppress covert side-channel bandwidth to kilobits per second, not to zero [[S-0018]].\n- **Choosing the bound.** A threshold that separates inference from training today may not do so later. On the widely used C4 dataset, DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less [[S-1314]]. Sastry and colleagues note that decentralised training across many data centres, or with lower-quality compute, could undermine compute detectability if it becomes more viable [[S-0053]].", "body_text": "Why it matters Distributed training depends on fast communication between chips, so limiting communication limits training [S-0005]. - Pods with narrow external links. Scher and Thiergart describe pods of chips with high-bandwidth interconnect inside each pod but very low external bandwidth: enough for inference tokens, not for training gradients [S-0005]. They suggest this could be implemented with modest changes to existing technology, physical access and monitoring, without access to code [S-0005]. - Hardware limits on networking. Compute-governance analyses list physical limits on chip-to-chip networking as a way to enforce compute caps [S-0053]. RAND's \"fixed set\" design restricts networking so that small, fixed sets of GPUs cannot be aggregated into large clusters [S-0057]. - Limits on what leaves a facility. A low-trust system overview assumes that the information bandwidth of inference is not more than two orders of magnitude above expected API traffic at full utilisation [S-0018]. It asks how far a facility can be required to make all egress traffic explainable by its ingress [S-0018]. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [S-1508]. This links the claim to Model weights or data have not left the facility. Why it is hard The claim requires knowing every path. - Taps see only the links they are on. RAND's network-tap layer observes data exchanged between chips [S-0002]. The cost and feasibility of tapping differ by link: Cankaya describes front-end links to the outside world as the most viable to monitor, while the back-end compute fabric has far higher bandwidth, is latency-critical, and may require sampling rather than full capture [S-0031]. - Some links are hard to observe at all. As scale-up domains grow, more traffic runs over copper links that passive optical taps cannot see, and encrypted interconnects complicate inspection [S-0031]. Shavit notes that methods for verifiers to confirm, after the fact, the topology in which chips were configured in a data centre were beyond the scope of his work [S-0029]. - Physical side channels. Data could bypass network monitoring altogether. A MIRI Technical Governance Team post surveys electromagnetic, acoustic, conducted and fibre-optic channels, and proposes shielding, active jamming with sensing, inspection and filtering [S-0038]. The system overview's target is to suppress covert side-channel bandwidth to kilobits per second, not to zero [S-0018]. - Choosing the bound. A threshold that separates inference from training today may not do so later. On the widely used C4 dataset, DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less [S-1314]. Sastry and colleagues note that decentralised training across many data centres, or with lower-quality compute, could undermine compute detectability if it becomes more viable [S-0053].", "addressed_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/", "role": "primary", "note": "Caps or removes communication between declared groups of accelerators (S-0067, S-1301)." }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/", "role": "supporting", "note": "Bounds effective unexplained output bandwidth rather than raw link bandwidth." }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/", "role": "supporting", "note": "Interlocks on NVLink or NICs, and RAND's fixed-set pods, would bound communication (S-1204, S-0057)." }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/", "role": "primary", "note": "Bounds the capacity of physical covert channels out of an enclosure, so that monitored links carry all significant traffic." }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/", "role": "supporting", "note": "Protects network devices that enforce or monitor bandwidth boundaries (S-0018)." }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/", "role": "supporting", "note": "Removing back-end networking limits communication between inference units." } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0009", "title": "Model weights or data have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" } ] } }