FLOP accounting

FLOP accounting estimates or verifies the total number of floating-point operations (FLOP) that a training run or other workload performs, usually to compare it with a threshold set by a rule ^{1 2}.

Shavit lists total training compute among the rules a verifier might enforce, noting that it has proven to be an indicator of model capabilities ¹. US Executive Order 14110 required reporting for models trained with more than 10^26 operations ² until its revocation in January 2025 ⁸, and a proposed international agreement sets a prohibited threshold of 10^24 FLOP and a monitored threshold of 10^22 FLOP ³. Proposed ways to count or cap FLOP include:

• Hardware time. Shavit converts a FLOP threshold into accelerator-days by assuming that every accelerator runs at its full rate with perfect parallelization, a conservative assumption that gives the fewest accelerator-days a run of that size could occupy ¹.
• Energy. Energy use can be converted into an approximate FLOP count ⁴.
• Telemetry. RAND lists estimating a workload's model FLOPs utilization (MFU) and physical signature, such as power, as a research problem ⁵, the aim of on-chip telemetry.
• On-chip budgets. Offline licensing ties chip use to a renewable licence carrying a compute budget ⁶, as in hardware performance throttling and licensing.

Training compute is only a high-level proxy for capability, and algorithmic progress means thresholds may need to change ²; one system overview argues that coarse totals such as FLOP counts are not enough and seeks evidence about individual workloads ⁷.