SASH confidential network logger

What it is

The Singapore AI Safety Hub (SASH) describes an international collaboration to build inference-verification mechanisms that distinguish model inference from training in data centres ¹. Its early partners include the Oxford Martin AI Governance Initiative and the Future of Life Institute ¹. The researchers span Singapore, the US, the UK, Hungary, Canada and Germany ¹.

The core component is a Confidential Network Logger (CNL): "specialised traffic analysis systems which split and copy network traffic coming out of a cable" ¹. SASH argues that, unlike on-chip mechanisms, CNLs "can be installed on most datacenters" ¹. The first prototype is published as open-source code under the MIT licence, and its README calls it "a prototype demo for verifying LLM inference" ². It implements network logging combined with recomputation. SASH's planned next prototype samples requests, while the published code re-runs all of them ^{1 2}.

How it works

SASH describes the flow planned for its next prototype as follows ¹:

1. An FPGA placed between the host's cluster and its output gateway issues "cryptographic certificates attesting to the encrypted traffic it has seen" as traffic passes through ¹.
2. A random sample is sent to a recomputation cluster "which has been mutually inspected by both parties" and houses another copy of the model ¹.
3. The cluster recomputes the received inputs, and an alarm is triggered if the outputs do not match ¹.

SASH says the verifier would see only "encrypted hashes" of inputs and outputs, so it can tell whether they match without reading them ¹. SASH motivates the approach by the different traffic patterns of the two workloads ¹. It describes inference as many small bursts of activity on relatively few GPUs, and training of large models as sustained, cyclical activity on many GPUs ¹.

The published code splits the system into a gateway, a network logger, a main cluster, a recomputation cluster and a dashboard ²:

• The gateway encrypts each request and passes it to the logger ².
• The logger forwards it to the main cluster, then sends the request and response to the recomputation cluster, and releases the response only if the check passes ².
• The recomputation cluster re-runs the request and compares the response text for an exact match ².

In this version, every request is forwarded for verification and no hashes of the traffic are computed; the random sampling and hashes described in the blog post belong to the planned next prototype ^{2 1}. Reproducible inference, which exact matching needs, is covered in Deterministic and bit-exact inference.

Evidence

• Hardware demonstration. In SASH's demonstration, "a pair of DGX Spark GPUs sit in for the audited datacenter and recomputation cluster", and Raspberry Pis stand in for the logger and the data-centre frontend ¹.
• Public code. The code's container setup serves the Gemma 3 270M instruction-tuned model on a CPU inference image, and a demo switch simulates covert training by replacing output with random characters ².
• Next prototype. SASH says a further prototype with an FPGA is planned ¹.

Limitations

SASH lists the following ¹:

• "There is no means to monitor the security of the recomputation cluster" ¹.
• Using a Raspberry Pi 5 as the logger locks the design into Broadcom and Arm supply chains ¹.
• Future versions would add zero-knowledge proofs to the recomputation method, eliminate a wider range of side-channel vulnerabilities, and scale the design to production traffic volumes ¹.

The general limitations of network taps, such as covert capacity in model outputs and line-rate hashing, are covered in Network taps and certifiers.