On-chip telemetry from timing, memory and performance counters

How it works

Three families of signals are proposed.

Built-in counters and management telemetry. RAND notes that GPUs already have on-device counters for floating-point operations, instructions executed, NVLink and PCIe bandwidth, memory accesses and power. These can be reported per streaming multiprocessor or for the whole chip ⁴. O'Gara et al. list such counters as possible meters for compute accounting and licensing ⁵. Rahman and Tajdari train a classifier to spot training workloads from nine standard NVML counters sampled once a second. They describe these signals as content-agnostic ³.

Timing and memory challenges. Monfared et al. have a challenger send puzzles to the GPU and time the answers ¹:

• memory-hard proof-of-work puzzles probe parallel effort;
• verifiable delay functions probe sequential execution;
• matrix-multiplication puzzles probe the tensor cores;
• a residency test checks whether challenge data stayed in the GPU's own memory.

Co-running workloads compete for the same units, so contention shows up as shifts in the timing distributions ¹. Gloria Z similarly suggests memory-residency and random challenges, which force an operator to keep false accounts consistent across every signal ⁶. Timed challenge-response in general is covered in Timed challenge-response and memory-occupation challenges.

New memory-side hardware. Guaranteeable Memory would place an open-source "guarantee chiplet" beneath the HBM stacks. The chiplet would observe memory traffic and attest to memory snapshots, GPU instructions, or random subsets of claimed computations ². This connects to the guarantee processors of Hardware-enabled guarantees (flexHEG) and guarantee processors.

What it establishes

These signals give statistical indicators of utilisation and workload type. Examples are whether a GPU is training ³ and whether data is resident in its memory ¹. They bear most directly on This compute runs inference, not training, and more weakly on Declared hardware is idle or shut down and There is no undeclared relevant compute.

They cannot establish:

• Correct execution. Monfared et al. state that their measurements are "not designed to deliver cryptographic proof of correct execution" ¹.
• Which device answered. The timing challenges do not identify the individual GPU ¹.
• The content of the computation. The counters do not access model weights or training data ³.
• That all activity was declared. Gloria Z treats telemetry as one partial answer to resource accounting, and judges that it "seems extremely difficult to verify the completeness of workload declarations" ⁶.

Classifying workloads from these and other signals is covered in Workload classification from telemetry and side channels.

Threat model

The schemes trust different parts of the stack:

• NVML classifier. It assumes the counters are genuine. Rahman and Tajdari list the protections this would need: a tamper-resistant read path, an authenticated channel and secure boot of the monitoring software. They model two evaders: a cloud tenant with user-space control, and a hardware operator who controls the full software stack ³.
• Timing challenges. Monfared et al.'s most adversarial case lets the host and GPU firmware be modified, virtualized or colluding. Only challenge generation, timing and verification are trusted, and the challenges run on the GPU being measured ¹.
• Guaranteeable Memory. It aims not to rely on any other component, including the main accelerator die ².

Confidentiality is only partial. Counters are content-agnostic ³, but they are also a known side channel ⁶. NVIDIA disables them in full confidential-computing mode because they could provide an avenue for side-channel attacks ⁷, which puts this mechanism in tension with TEE remote attestation for AI workloads.

Evidence

• Rahman and Tajdari. They tested nine NVIDIA GPU models from four architecture generations, including A100, H100, H200 and B200, with up to eight GPUs per node. The classifier reached 98.2% binary accuracy at identifying training across their whole corpus. After five monitor-evader rounds covering 20 evasion strategy families, it reached 43–87% accuracy on the most challenging disguised workloads held out from training. The authors state that code for the workloads is in the project repository, but give no URL ³.
• Monfared et al. They measured T4 and H100 GPUs running language models from 1.1B to 72B parameters. With 60 GB of challenge data on an H100, the residency test separated on-device from off-device data by more than 350 ms. Continuous puzzles caused noticeable losses in power and throughput, while the residency test's losses were negligible. The paper mentions no code release ¹.
• Guaranteeable Memory. It is a workshop proposal with no reported prototype ².

Limitations

• Forgery. If the operator controls the software that reads the counters, the counters prove little ³.
• Disguise. Adversarial disguise works in part: white-box LoRA fine-tuning was detected less than half the time ³.
• Outsourcing and virtualization. Timing challenges can be answered by identical GPUs, and virtualization confounds attribution ¹.
• Checking matrix results. Floating-point rounding complicates the check ¹.
• Error rates. The timing primitives have no calibrated error rates ¹.
• Evaluation scope. The classifier was evaluated only on single nodes, with evasion at the PyTorch level and sampling at about 1 Hz ³.
• Power sampling. Yang and colleagues found that on A100 and H100 GPUs the built-in power reading, which nvidia-smi obtains through NVML, samples only 25% of runtime. The GPU can draw very different power in the other 75% without the reading showing it ⁸. They also found the reading's error to be within about ±5% in most cases, against the ±5 W that NVIDIA claims ⁸.
• Leakage. Richer counters risk leaking secrets ⁶.