Compartmentalization

Compartmentalization divides a facility's accelerators into groups and restricts communication between the groups, so that no group can take part in a larger workload than it is allowed ^{1 2}.

Scher and Thiergart describe pods with high-bandwidth interconnect inside and very little bandwidth between them: between pods, inference needs to pass only tokens, whereas training exchanges gradients or activations ¹. Efficient inference fits within dozens to low hundreds of closely connected accelerators, while large-scale training links thousands ¹. RAND's "fixed set" design likewise restricts networking so that small, fixed sets of GPUs cannot be combined into large clusters ², and Sastry and colleagues list physical limits on chip-to-chip networking as a way to enforce compute caps ³; these ideas underlie bandwidth limits and compartmentalization.

Compartments can also separate trust domains: one low-trust design air-gaps its evaluation environments and uses optical splitters and data diodes, simple components that can be inspected for tampering, to enforce one-way data movement ⁴. The boundaries can be checked by observing traffic between accelerators with network taps ⁵, while physical channels that could bypass monitored links are the target of side-channel suppression ⁶.