Bandwidth limits and compartmentalization

How it works

Training and serving use the network differently ^{1 2}. The AI 2040 plan notes that AI servers "have a lot of high bandwidth communication between them, but this is (mostly) only necessary for training" ¹. Lucid Computing gives the same rationale: inference sends little external traffic, while training moves gradients, model state and activations among many machines ².

Two families of design use this asymmetry ^{1 2}:

• Removing links. The AI 2040 plan would "convert datacenters into groups of isolated inference-units" by removing a significant amount of back-end networking ¹. Its illustration retrofits 40% of US compute into inference units of four server racks ¹.
• Capping links. Lucid's Traffic Shaping design installs a network shaper at the north-south uplink of each pod, modelled as one GB200 NVL72 rack ². It recommends a cap of 1 Gbps per direction per pod, averaged over 60 seconds ². The parties would configure and agree on the shaping devices, built on commodity hardware or open standards through a mediated co-design process ².

The MIRI system overview frames the same boundary as a perimeter: untrusted devices are grouped so that only monitored links communicate in and out ⁴. For inference, that perimeter can be as small as pods of a few hundred accelerators ⁴. Monitored links are covered in Network taps and certifiers.

Amodo has built a related enforcement point for weight security ⁵. It replaced a node's NICs with NVIDIA BlueField-3 DPUs, reasoning that "all the traffic in and out of a node goes through a NIC", which creates "a clean bandwidth boundary" ⁵.

For proofs of compartmentalization, a related approach to constraining training, see Zero-knowledge proofs of training constraints.

What it establishes

Lucid aims to let a third party verify that frontier training either is not happening in a declared facility or would cost so much more that training a model ten times the current frontier becomes economically infeasible ². It estimates a lower bound of about 350x on the inefficiency of covert frontier training, with central estimates of 1,000x to 10,000x, if the auditor controls pod routing and per-pod storage is capped ². This addresses Communication between compute groups is bounded, and indirectly This compute runs inference, not training and A training run stayed within declared limits.

It does not establish:

• What the pods compute. A cap alone does not show this, so the AI 2040 plan pairs inference units with taps and recomputation (Sampled inference recomputation, Reproducible computation packets) ¹.
• Absence of training inside one pod. Lucid places training that fits within one pod, fine-tuning and reinforcement-learning post-training within one pod outside its modelled threat ².
• Anything outside declared facilities. Rahman judges it "infeasible to prevent evaders from getting fast internet connections" ⁷.

Threat model

• Favourable adversary. Lucid's analysis grants the adversary perfect overlap of communication and computation, zero pipeline bubbles and "the best published communication-reduction techniques", among other advantages ².
• Auditor-controlled routing. The bound holds only if the auditor randomizes the assignment of pods to routers and periodically re-randomizes it ².
• Declared storage. Per-pod storage must be declared, capped and inspected ².
• Complete perimeter. The shaper sits at each pod's uplink ². The verifier must also know that no other path leaves the pod, which the MIRI overview handles with a perimeter where only monitored links communicate in and out ⁴; see Network taps and certifiers and Tamper evidence for verifier devices.

Evidence

• Designs. The AI 2040 plan, Lucid's design brief and the MIRI overview are designs and analysis ^{1 2 4}. Lucid describes its design as "still at the design stage and not yet implemented or red-teamed" ². It reports that its engineers are building a proof of concept for red-teaming at a partner cluster ².
• Node-level limiter. Amodo reports 390 Gbps unencrypted and 193 Gbps encrypted throughput between two DPUs on a 400G link ⁵. That demonstration targets weight security with a cooperating operator, and Amodo has "not yet fully analysed resilience to a single compromised DPU" ⁵.

Limitations

• Low-communication training. DiLoCo matched fully synchronous training on 8 workers while communicating 500 times less ⁶. Rahman writes that this family theoretically allows large-scale training at under 100 Mbps ⁷. Lucid's bounds include DiLoCo- and SWARM-family methods, but it notes that extreme activation compression or modular paradigms could erode the margin ².
• Routing control. If the operator can group pods freely behind routers, the bound falls to about 90–220x uncompressed and as low as about 25x with compression ².
• Hidden storage. Undeclared per-pod storage weakens the bound ².
• Scale-up bypass. In GB200 topologies, GPUs reach other nodes through NVSwitches without a NIC on the path, and compromising one or two parallel switches would bypass a limit ⁵.
• Durability. Lucid positions the design as "a short-to-medium-term deterrent and Phase-1 milestone" ².