Network taps and certifiers

How it works

A network tap is a device that "mirror[s] network traffic without disrupting normal operation" ². Verification proposals use taps to build a record of what crossed a link, which the verifier can check later ^{2 3}. The MIRI system overview states the principle as "capture everything, challenge and audit random samples after" ³.

In the most detailed designs, the verifier never receives plaintext ^{1 3}. The tap hashes payloads over groups of packets, without knowing where objects such as prompts begin or end, and sends only hashes out; the operator (the prover) keeps the plaintext ^{1 3}. Later the verifier picks stored hashes at random, the prover supplies the matching data, and a verification facility replays the computation and checks that the hashes match ¹. The prover fails the challenge if it supplies no data, or data that does not match the stored hash ³. A replay that does not reproduce the recorded output points to false metadata, a technical fault or a workload other than the one declared ³.

The AI 2040 plan proposes a simpler variant: passive taps on the frontend network copy all input and output traffic to a recomputation server (Sampled inference recomputation, Whole-workload recomputation (reproducible packets)) ⁴.

Taps can sit at three levels ²:

• North-south (frontend) links to the outside world, at an estimated material cost under 0.01% of the monitored data centre ².
• The storage fabric, which carries model loads, dataset reads and checkpoint saves, at an estimated 0.3–0.5% of data-centre cost ².
• The compute (east-west) fabric, at an estimated 0.2–1.5% of data-centre cost for its optical levels. Volumes there are so high that Cankaya proposes sampling, for example 0.01% of traffic, about 20 Gb/s, through an optical circuit switch ².

For inference, the monitored perimeter could be as small as pods of a few hundred accelerators; monitoring training needs larger perimeters or taps on the compute and storage fabrics ³.

A key aim of the "Secure Gateway Device" design is that no processor has to be trusted by both sides ¹. It uses passive optical splitters, so that each party can watch the signals it must check, and coin-flip protocols to generate shared random values ¹. It also acts as an "active warden", rewriting free header fields and re-timing packets to close timing, analogue and header channels that checks on hashes cannot address ¹.

SASH's planned next prototype would place an FPGA between the cluster and its gateway to issue "cryptographic certificates attesting to the encrypted traffic it has seen" ¹²; see SASH confidential network logger.

What it establishes

The Secure Gateway Device paper aims to make it "infeasible to covertly exfiltrate the results of undisclosed workloads in the cluster through the tapped wires" ¹. Combined with replay, it would show that recorded outputs match declared workloads ^{1 4}; see inference-only use and weight exfiltration.

It does not establish:

• Taps capture flows, not device state, and the MIRI system overview treats memory probes as complementary ³.
• A north-south tap does not directly prevent covert workloads, only the exfiltration of their results ¹. The paper judges it futile to block every single-bit leak. It argues that the residual risk, covert computation plus low-bandwidth signalling of results, needs other checks such as east-west taps and throughput accounting ¹.
• The paper does not decide which workloads should be allowed, or whether declared workloads comply ¹.

Threat model

• The operator and the verifier distrust each other. The operator must not hide information in the traffic, and the verifier must learn nothing beyond hashes ¹.
• The facility is physically monitored, so that all traffic passes through the gateway ¹.
• The operator stores all hashed data for the agreement's duration ¹.
• Replay needs reproducible inference (Deterministic and bit-exact inference) ^{1 3}.
• The MIRI system overview models a covert adversary, deterred when the likelihood of detection times the cost of being caught exceeds the expected gain ³.